data_io module

class data_io.DataSet(data_path: List[str] | str)

Bases: object

Methods

evaluate_regex_path(data_path)	Take the given regular expression to generate a list of paths that match it.
find_base_path_in_regex(path_regex)	Takes a regular expression for a file path and determines the static part of the given path regex.

expand_data_path
get_common_root
get_data_path
get_file_list

static evaluate_regex_path(data_path: str)

Take the given regular expression to generate a list of paths that match it.

Parameters:

data_pathstr

The regular expression that is used to select files.

Returns:

List[str]

A list of paths matching the regular expression.

expand_data_path()

static find_base_path_in_regex(path_regex: str) → str

Takes a regular expression for a file path and determines the static part of the given path regex.

Parameters:

path_regexstr

The regex pattern for the file path.

Returns:

str

Static parts of the path that do not contain any regex.

get_common_root() → str

get_data_path() → List[str] | str

get_file_list() → List[str]

data_io.read_from_file(path, file_format='feather', sample: float | None = None, sample_seed: int = 23, filter_query: str | None = None)