CSV operations helper classes¶
In order to ease the CSV related operations, there are two helper classes to assist.
Module Core is directly calling the Base class. Process class is for manipulating CSV files and should be called explictly.
Base Class CSV¶
Base class for csv operations
Base class for csv operations.
from ozcore import core csv = core.csv csv.read("path_to_csv_file", **kwargs) csv.save(df_altered, index=False, verbose=False, overwrite=False, **kwargs) # warning: this saves over the read csv file! csv.search(q="something") # searches every column # but if certain columns desired to be searched than define it before action csv.searchable = ["col1"] csv.search(q="something in col1")
read(path, verbose=True, **kwargs)¶
read a csv file
path – str|posixpath, path to csv file
kwargs – pandas read_csv arguments
fills self.actual_df and self.df
save(df_altered, index=False, verbose=False, overwrite=False, **kwargs)¶
saves altered dataframe on the csv file
df_altered: Dataframe index: bool, default False verbose: bool, default False overwrite: bool, default False kwargs: pandas to_csv arguments
logging if verbose is True
if overwrite is True, dumps given df into file without checking its len with actual_df
search records based on self._searchable argument (col name to search given in child classes)
dataframe slice from search results
Process Class CSV¶
Processing and marking a csv file.
Class for processing and marking csv files. Drived from Base class. ! And, cannot be initiated if validation fails.
This module can be used to modify fields in a csv file and tick mark completed rows…
Processing CSV assumes that the csv file has no index defined. Also, no kwargs are passed to read_csv method of Pandas.
You need to define a path to the csv file in order to initiate the Process class.
If no process header found in CSV, this class raises an Exception
from ozcore.core.data.csv.process import Process as csv_process csv = csv_process(path="path_to_csv") csv.focus(0) # focus first record csv.processed() # mark first record as processed and save it to csv files csv.next() # focus on next unprocessed record (marked as 0) if exists
create a processed column:
# to use this class, a processed column with 0 values should be available # warning: there may be similar column, please check before proceeding from ozcore import core df = core.csv.read(file) df = df.assign(processed=0) df.processed = df.processed.astype(int) df.loc[:,"processed"] = 0 core.save(df, index=False, overwrite=True) # now you can initiate the Process class
deleting the processed column:
# when you are finished with marking csv file you can remove it # warning: be sure to remove from ozcore import core df = core.csv.read(file) df.drop(columns="processed", inplace=True) core.save(df, index=False, overwrite=True) # column is removed
Validates the CSV if fits the strict rules for processing a CSV file.
- the record which is in progress
all write actions is applied to this Serie
index: int, index number of the record being focused verbose: bool, default True
fills self.focus_index and displays the Series if verbose displays a warning msg
next unprocessed item where processed==0 is displayed and assigned as focused
marks a records as processed by assigning 1
revert – boolean, default False
when a records focused by focus(index) or next() methods assigns 1 to processed column (0 if revert is True) saves the file