omics_utils.Selector#

class omics_utils.Selector(*args, input_dir, filterer, var_cols, n_debug=None, subset=None, n_highly_variable=None, gene_file=None, **kwargs)#

Bases: Step

Selects the genes used in consecutive preprocessing steps.

__init__(*args, input_dir, filterer, var_cols, n_debug=None, subset=None, n_highly_variable=None, gene_file=None, **kwargs)#
Parameters:
  • input_dir (str) – Folder where the input CellXGene AnnData files are saved.

  • filterer (Partial[AnnDataFilterer]) – Filter to apply to each individual AnnData file.

  • var_cols (dict[str, str]) – Subset of columns in the var DataFrame to keep and their respective dtype.

  • n_debug (int | None) – Take the first n input files only for debugging purposes.

  • subset (list[str] | None) – Subset of .h5ad file names to process.

  • n_highly_variable (int | None) – Number of highly variable genes to take per sub-dataset.

  • gene_file (str | None) – File to load the selected genes from. Ignores all other kwargs.

Methods

__init__(*args, input_dir, filterer, var_cols)

cluster_context()

Context manager for cluster lifecycle management.

read(input_files)

Read and transform the data from the the input data.

run()

Run the whole preprocessing step by sequentially running read and write.

run_step()

Run the pipeline step.

transform(file_path)

Preprocess a single AnnData file.

write(data_df)

Rank the genes by number of batches they are HVG and break ties by dispersion norm.

Attributes

input_step

Has no preceding preprocessing step.

meta_df

Empty DataFrame used by Dask to determine the output of transform.

output_step

Name of this preprocessing step.

cluster_context()#

Context manager for cluster lifecycle management.

Yields:

A dask Client connected to the configured cluster.

Return type:

Generator[Client, None, None]

property input_step: str | None#

Has no preceding preprocessing step.

property meta_df: DataFrame#

Empty DataFrame used by Dask to determine the output of transform.

property output_step: str#

Name of this preprocessing step.

read(input_files)#

Read and transform the data from the the input data.

NOTE: In this case the read step contains the transform step for compatibility with dask.

Parameters:

input_files (list[Path]) – File paths to process in parallel.

Return type:

DataFrame

Returns:

The data ready for transformation.

run()#

Run the whole preprocessing step by sequentially running read and write.

Return type:

None

run_step()#

Run the pipeline step.

Return type:

None

transform(file_path)#

Preprocess a single AnnData file.

Parameters:

file_path (Path) – Path to the AnnData file to be transformed.

Return type:

DataFrame

Returns:

A list of Delayed tasks for preprocessing.

write(data_df)#

Rank the genes by number of batches they are HVG and break ties by dispersion norm.

Parameters:

data_df (DataFrame) – DataFrame containing a gene id, a highly variable gene flag and the dispersion norm.

Return type:

None