Frequently Ask Questions

Q: How to use GPU training?

You can specify the devices in the trainer config. PyTorch Lightning support automatic multiple GPU training. Please refer to the PyTorch Lightning documentation for more details.

Q: How to run all example pipelines?

You can run the following command to run all example pipelines.

Continual learning:

clarena pipeline=CL_MAIN_EXPR index=example_cl_main_expr

# after running the above command, run main evaluation
clarena pipeline=CL_MAIN_EVAL index=example_cl_main_eval main_model_path=... output_dir=...

clarena pipeline=CL_REF_JOINT_EXPR index=example_cl_main_expr
clarena pipeline=CL_REF_INDEPENDENT_EXPR index=example_cl_main_expr
clarena pipeline=CL_REF_RANDOM_EXPR index=example_cl_main_expr

# after running the above command, run full evaluation
clarena pipeline=CL_FULL_EVAL index=example_cl_full_eval main_acc_csv_path=... refjoint_acc_csv_path=... refindependent_acc_csv_path=... refrandom_acc_csv_path=... output_dir=...

clarena pipeline=CL_FULL_EXPR index=example_cl_main_expr

Continual unlearning:

clarena pipeline=CUL_MAIN_EXPR index=example_cul_main_expr

# after running the above command, run main evaluation
clarena pipeline=CUL_MAIN_EVAL index=example_cl_main_eval main_model_path=... output_dir=...

clarena pipeline=CUL_REF_RETRAIN_EXPR index=example_cul_main_expr
clarena pipeline=CUL_REF_ORIGINAL_EXPR index=example_cul_main_expr

# after running the above command, run full evaluation
clarena pipeline=CUL_FULL_EVAL index=example_cul_full_eval main_model_path=... refretrain_model_path=... reforiginal_model_path=... output_dir=...

clarena pipeline=CUL_FULL_EXPR index=example_cul_main_expr

Multi-task learning:

clarena pipeline=MTL_EXPR index=example_mtl_expr

# after running the above command, run evaluation
clarena pipeline=MTL_EVAL index=example_mtl_eval model_path=... output_dir=...

Single-task learning:

clarena pipeline=STL_EXPR index=example_stl_expr

# after running the above command, run evaluation
clarena pipeline=STL_EVAL index=example_stl_eval model_path=... output_dir=...

Q: What are the best practices for organizing configs?

• Create only one YAML file for each class as its default config.
• Override in the index config if you want to change field values.
• Override in the command line for those runtime configs, especially like trainer.

Q: How to reproduce an experiment?

In each pipeline that involves uncertainty, there is a global_seed config field that set the random seed for all random operations. You can set it to a fixed value to reproduce the experiment.

Do we support resume training from checkpoints?

Unfortunately not yet. Resume training in continual learning makes things messy; that’s why we don’t support it for now. We may consider adding this feature in the future.

Q: Many logging messages are output to the console. How can we add them in the source code?

We use the Hydra logging system to output logging messages. To add logging messages in your module implementation, you need to first include the following code snippet:

# always get logger for built-in logging in each module
pylogger = logging.getLogger(__name__)

And then call pylogger.info(), pylogger.debug(), pylogger.warning(), etc., to output logging messages.