omics_utils.Splitter#
- class omics_utils.Splitter(*args, converter, seed, n_partitions, test_size=0.1, **kwargs)#
Bases:
StepSplits the data into a train and validation set.
- __init__(*args, converter, seed, n_partitions, test_size=0.1, **kwargs)#
- Parameters:
converter (
Partial[AnnDataConverter]) – Converter used to convert the DataFrame format back to AnnData.seed (
int) – Seed to use for shuffling of the dataset.
Methods
__init__(*args, converter, seed, n_partitions)Context manager for cluster lifecycle management.
read()Read the data from the previous step.
run()Run the stratification and splitting step.
run_step()Run the pipeline step.
transform(data_df)Extract the metadata and perform splitting.
write(train_df, val_df, metadata_df, categories)Save the folds and the case metadata separately.
write_df_to_anndata(df, output_dir, fold_name)Write a DataFrame to an anndata file.
Attributes
Name of the preceding preprocessing step.
Name of this preprocessing step.
- cluster_context()#
Context manager for cluster lifecycle management.
- Yields:
A dask Client connected to the configured cluster.
- Return type:
Generator[Client,None,None]
- property input_step: str#
Name of the preceding preprocessing step.
- property output_step: str#
Name of this preprocessing step.
- read()#
Read the data from the previous step.
- Return type:
DataFrame- Returns:
The data ready for transformation.
- run()#
Run the stratification and splitting step.
- Return type:
None
- run_step()#
Run the pipeline step.
- Return type:
None
- transform(data_df)#
Extract the metadata and perform splitting.
- Return type:
tuple[dict[int,DataFrame],DataFrame,dict[str,list[str]]]- Returns:
A tuple of a dict containing a mapping of the fold index to the data split and the metadata DataFrame.
- write(train_df, val_df, metadata_df, categories)#
Save the folds and the case metadata separately.
- Parameters:
folds – A dict containing a mapping of the fold index to the data split.
metadata_df (
DataFrame) – The metadata DataFrame.categories (
dict[str,list[str]]) – Categories of the metadata columns.
- Return type:
None
- write_df_to_anndata(df, output_dir, fold_name, partition_info=None)#
Write a DataFrame to an anndata file.
- Parameters:
df (
DataFrame) – Partition as a pandas DataFrame to save as an AnnData .h5ad file.output_dir (
Path) – Folder to save the anndata file in.fold_name (
str) – Name of the fold the partition belongs to. Usually train or val.partition_info (
dict[str,Any] |None) – Dictionary containing metadata about the partition.
- Return type:
None