Configure Callbacks (CUL)

Modified

August 16, 2025

Under the framework of PyTorch Lightning, we use callbacks to add additional actions and functionalities integrated in different timing of the experiment. This includes before, during, or after training, validating, or testing process.

Each type of callback is designed for one set of actions, such as save sample images, generating logging information, etc. we can apply multiple callbacks to enables different sets of actions at the same time.

This section is only about configuring callbacks other than metric callbacks. To configure metric callbacks, please refer to Configure Metrics (CL Main) section.

Callbacks is a sub-config under the experiment index config (CUL Main) and experiment index config (CUL full evaluation). To configure custom callbacks, you need to create a YAML file in callbacks/ folder. At the moment, we only support uniform callbacks setting across all tasks. Below shows examples of the callbacks config.

Example

configs
├── __init__.py
├── entrance.yaml
├── experiment
│   ├── example_clmain_train.yaml
│   └── ...
├── callback
│   ├── cul_default.yaml
...
configs/experiment/example_culmain_train.yaml
defaults:
  ...
  - /callbacks: cul_default.yaml
  ...

The callbacks config is a list of callback objects:

configs/callbacks/cul_default.yaml
- _target_: clarena.callbacks.CULPyloggerCallback
- _target_: clarena.callbacks.CLRichProgressBar
- _target_: clarena.callbacks.SaveFirstBatchImagesCallback
  save_dir: ${output_dir}/samples/
- _target_: clarena.callbacks.SaveModelCallback
  save_dir: ${output_dir}/saved_models/

Supported Callbacks & Required Config Fields

All Lightning built-in callbacks are supported. In CLArena, we also implemented many callbacks in clarena.callbacks module that you can use for your CUL Main experiment.

The _target_ field of each callback must be assigned to the corresponding class name, such as lightning.pytorch.callbacks.EarlyStopping for EarlyStopping. Each callback has its own required fields, which are the same as the arguments of the class specified by _target_. The arguments of each callback class can be found in PyTorch Lightning documentation and CLArena API documentation.

PyTorch Lightning Documentation (Built-In Callbacks) API Reference (Callbacks) Source Code (Callbacks)

Below is the full list of supported callbacks that can be applied to CUL Main experiment. Note that the “Callback” is exactly the class name that the _target_ field is assigned.

Callback Description Required Config Fields
SaveModelCallback Saves the model at the end of each training task. Please refer to Save and Evaluate Model (CLMain) section. Same as SaveModelCallback class arguments
CULPyloggerCallback Provides additional logging messages for during continual unlearning progress. For example, at the start and end of each training and testing task, log the task ID and other relevant information. Same as CULPyloggerCallback class arguments
SaveFirstBatchImagesCallback Saves images and labels of the first batch of training data into files. Applies to all tasks. Same as SaveFirstBatchImagesCallback class arguments
CLRichProgressBar Customised RichProgressBar for continual learning. Same as CLRichProgressBar class arguments

The callbacks that can be applied to CUL full evaluation experiment include CULPyloggerCallback and CLRichProgressBar.

Back to top