dance.pipeline

dance.pipeline.get_additional_sweep(entity, project, sweep_id)

Recursively retrieve all related sweep IDs from a given sweep.

Given a sweep ID, this function recursively finds all related sweep IDs by examining the command arguments of the runs within each sweep. It handles cases where sweeps may have prior runs or additional sweep references.

dance.pipeline.get_step3_yaml(conf_save_path='config_yamls/params/', conf_load_path='step3_default_params.yaml', result_load_path='results/pipeline/best.csv', metric='acc', ascending=False, step2_pipeline_planer=None, required_funs=['SetConfig'], required_indexes=[9223372036854775807], root_path=None)

Generate the configuration file of step 3 based on the results of step 2.

Parameters:

• conf_save_path – Directory to save the configuration file generated in step 3.

• conf_load_path – Parameter search range of all preprocessing functions under a specific algorithm task.

• result_load_path – Path for the result of step 2.

• metric – Evaluation criteria.

• ascending – The order of the results of step 2.

• top_k – The number of steps 2 selected.

• required_funs – Required functions in step 3.

• required_indexes – Location of required functions in step 3.

• root_path – root path of all paths, defaults to the directory where the script is called.

dance.pipeline.run_step3(root_path, evaluate_pipeline, step2_pipeline_planer, tune_mode='params')

Run step 3 by default.

Parameters:

• root_path – root path of all paths, defaults to the directory where the script is called.

• evaluate_pipeline – Evaluation function

• step2_pipeline_planer (PipelinePlaner) – Pipeline_planer of step2

• tune_mode – tune_mode can only be set to params

dance.pipeline.save_summary_data(entity, project, sweep_id, summary_file_path, root_path, additional_sweep_ids=None, save=True)

Download sweep summary data from wandb and save to file.

The returned dataframe includes running time, results and corresponding hyperparameters, etc.

-----------------------------------------------------------------------------------------------------------
| id         | _runtime    | _timestamp      | acc   | _step      | _wandb_runtime   | pipeline.0         |
-----------------------------------------------------------------------------------------------------------
| 9hwsyumy   | 42.445390   | 1706685331.70  | 0.707  | 0          | 41               | WeightedFeaturePCA |
| sgsr5mw4   | 38.906304   | 1706685272.85  | 0.707  | 0          | 37               | WeightedFeaturePCA |
| czvfm197   | 48.190104   | 1706685217.47  | 0.331  | 0          | 47               | CellPCA            |
-----------------------------------------------------------------------------------------------------------

Return type:

DataFrame