omics_utils.Pipeline#
- class omics_utils.Pipeline(steps, output_dir=None, force_recompute=False, cluster=None, scale=None)#
Bases:
objectHandles a sequence of preprocessing steps in an ETL style pipeline.
Can be used for the preprocessing, concatenation, stratification, splitting and saving of tabular omics datasets.
- __init__(steps, output_dir=None, force_recompute=False, cluster=None, scale=None)#
Initializes an instance of a preprocessing pipeline.
- Parameters:
steps (
list[Partial[Step]]) – Preprocessing steps to perform in sequenceoutput_dir (
str|None) – Output path to save intermediate and final preprocessing outputs in. By default, this is the hydra output directory (“outputs”).force_recompute (
bool) – Whether to recompute already existing intermediate results.cluster (
Optional[Partial[JobQueueCluster|LocalCluster]]) – Configured cluster to use for distributed preprocessing using dask. This config will be used for all steps in the pipeline.scale (
int|None) – Number of jobs to start on the cluster.
Methods
__init__(steps[, output_dir, ...])Initializes an instance of a preprocessing pipeline.
run()Run the preprocessing pipeline on the data.
- run()#
Run the preprocessing pipeline on the data.
- Return type:
None