CSV operations helper classes

In order to ease the CSV related operations, there are two helper classes to assist.

Module Core is directly calling the Base class. Process class is for manipulating CSV files and should be called explictly.

Base Class CSV

Base class for csv operations

class ozcore.core.data.csv.base.Base[source]

Base class for csv operations.

Usage:

from ozcore import core

csv = core.csv

csv.read("path_to_csv_file", **kwargs)

csv.save(df_altered, index=False, verbose=False, overwrite=False, **kwargs)
# warning: this saves over the read csv file!

csv.search(q="something")
# searches every column

# but if certain columns desired to be searched than define it before action
csv.searchable = ["col1"]
csv.search(q="something in col1")
read(path, verbose=True, **kwargs)[source]

read a csv file

Parameters
  • path – str|posixpath, path to csv file

  • kwargs – pandas read_csv arguments

Returns

fills self.actual_df and self.df

save(df_altered, index=False, verbose=False, overwrite=False, **kwargs)[source]

saves altered dataframe on the csv file

paramaters:

df_altered: Dataframe index: bool, default False verbose: bool, default False overwrite: bool, default False kwargs: pandas to_csv arguments

Returns

logging if verbose is True

Warning

if overwrite is True, dumps given df into file without checking its len with actual_df

search(q)[source]

search records based on self._searchable argument (col name to search given in child classes)

Returns

dataframe slice from search results

Process Class CSV

Processing and marking a csv file.

class ozcore.core.data.csv.process.Process(path)[source]

Class for processing and marking csv files. Drived from Base class. ! And, cannot be initiated if validation fails.

Hint

This module can be used to modify fields in a csv file and tick mark completed rows…

Warning

Processing CSV assumes that the csv file has no index defined. Also, no kwargs are passed to read_csv method of Pandas.

Notes

You need to define a path to the csv file in order to initiate the Process class.

Warning

If no process header found in CSV, this class raises an Exception

usage:

from ozcore.core.data.csv.process import Process as csv_process

csv = csv_process(path="path_to_csv")

csv.focus(0)
# focus first record

csv.processed()
# mark first record as processed and save it to csv files

csv.next()
# focus on next unprocessed record (marked as 0) if exists

create a processed column:

# to use this class, a processed column with 0 values should be available
# warning: there may be similar column, please check before proceeding

from ozcore import core

df = core.csv.read(file)

df = df.assign(processed=0)
df.processed = df.processed.astype(int)
df.loc[:,"processed"] = 0

core.save(df, index=False, overwrite=True)

# now you can initiate the Process class

deleting the processed column:

# when you are finished with marking csv file you can remove it
# warning: be sure to remove

from ozcore import core

df = core.csv.read(file)

df.drop(columns="processed", inplace=True)

core.save(df, index=False, overwrite=True)

# column is removed
property validate_csv

Validates the CSV if fits the strict rules for processing a CSV file.

focus(index, verbose=True)[source]
the record which is in progress

all write actions is applied to this Serie

paramaters:

index: int, index number of the record being focused verbose: bool, default True

Returns

fills self.focus_index and displays the Series if verbose displays a warning msg

next()[source]

next unprocessed item where processed==0 is displayed and assigned as focused

Returns

Series

processed(revert=False)[source]

marks a records as processed by assigning 1

Parameters

revert – boolean, default False

usage:

when a records focused by focus(index) or next() methods assigns 1 to processed column (0 if revert is True) saves the file