omics_utils.Splitter#

class omics_utils.Splitter(*args, converter, seed, n_partitions, test_size=0.1, **kwargs)#

Bases: Step

Splits the data into a train and validation set.

__init__(*args, converter, seed, n_partitions, test_size=0.1, **kwargs)#
Parameters:
  • converter (Partial[AnnDataConverter]) – Converter used to convert the DataFrame format back to AnnData.

  • seed (int) – Seed to use for shuffling of the dataset.

Methods

__init__(*args, converter, seed, n_partitions)

cluster_context()

Context manager for cluster lifecycle management.

read()

Read the data from the previous step.

run()

Run the stratification and splitting step.

run_step()

Run the pipeline step.

transform(data_df)

Extract the metadata and perform splitting.

write(train_df, val_df, metadata_df, categories)

Save the folds and the case metadata separately.

write_df_to_anndata(df, output_dir, fold_name)

Write a DataFrame to an anndata file.

Attributes

input_step

Name of the preceding preprocessing step.

output_step

Name of this preprocessing step.

cluster_context()#

Context manager for cluster lifecycle management.

Yields:

A dask Client connected to the configured cluster.

Return type:

Generator[Client, None, None]

property input_step: str#

Name of the preceding preprocessing step.

property output_step: str#

Name of this preprocessing step.

read()#

Read the data from the previous step.

Return type:

DataFrame

Returns:

The data ready for transformation.

run()#

Run the stratification and splitting step.

Return type:

None

run_step()#

Run the pipeline step.

Return type:

None

transform(data_df)#

Extract the metadata and perform splitting.

Return type:

tuple[dict[int, DataFrame], DataFrame, dict[str, list[str]]]

Returns:

A tuple of a dict containing a mapping of the fold index to the data split and the metadata DataFrame.

write(train_df, val_df, metadata_df, categories)#

Save the folds and the case metadata separately.

Parameters:
  • folds – A dict containing a mapping of the fold index to the data split.

  • metadata_df (DataFrame) – The metadata DataFrame.

  • categories (dict[str, list[str]]) – Categories of the metadata columns.

Return type:

None

write_df_to_anndata(df, output_dir, fold_name, partition_info=None)#

Write a DataFrame to an anndata file.

Parameters:
  • df (DataFrame) – Partition as a pandas DataFrame to save as an AnnData .h5ad file.

  • output_dir (Path) – Folder to save the anndata file in.

  • fold_name (str) – Name of the fold the partition belongs to. Usually train or val.

  • partition_info (dict[str, Any] | None) – Dictionary containing metadata about the partition.

Return type:

None