Shawn’s Blog
  • πŸ—‚οΈ Collections
    • πŸ–₯️ Slides Gallery
    • πŸ’» LeetCode Notes
    • πŸ§‘β€πŸ³οΈ Cooking Ideas
    • 🍱 Cookbook
    • πŸ’¬ Language Learning
    • 🎼 Songbook
  • βš™οΈ Projects
    • βš› Continual Learning Arena
  • πŸ“„ Papers
    • AdaHAT
    • FG-AdaHAT
    • AmnesiacHAT
  • πŸŽ“ CV
    • CV (English)
    • CV (Mandarin)
  • About
  1. Components
  2. Callbacks
  • Welcome to CLArena
  • Getting Started
  • Configure Pipelines
  • Continual Learning (CL)
    • CL Main Experiment
    • Save and Evaluate Model
    • Full Experiment
    • Output Results
  • Continual Unlearning (CUL)
    • CUL Main Experiment
    • Full Experiment
    • Output Results
  • Multi-Task Learning (MTL)
    • MTL Experiment
    • Save and Evaluate Model
    • Output Results
  • Single-Task Learning (STL)
    • STL Experiment
    • Save and Evaluate Model
    • Output Results
  • Components
    • CL Dataset
    • MTL Dataset
    • STL Dataset
    • CL Algorithm
    • CUL Algorithm
    • MTL Algorithm
    • STL Algorithm
    • Backbone Network
    • Optimizer
    • Learning Rate Scheduler
    • Trainer
    • Metrics
    • Lightning Loggers
    • Callbacks
    • Other Configs
  • Custom Implementation
    • CL Dataset
    • MTL Dataset
    • STL Dataset
    • CL Algorithm
    • CUL Algorithm
    • MTL Algorithm
    • STL Algorithm
    • Backbone Network
    • Callback
  • API Reference
  • FAQs

On this page

  • Example
  • Supported Callbacks & Required Config Fields
    • Continual Learning Callbacks
    • Multi-Task Learning Callbacks
    • Single-Task Learning Callbacks
  1. Components
  2. Callbacks

Configure Callbacks

Modified

October 9, 2025

Under the PyTorch Lightning framework, we use callbacks to add additional actions at different points in the experiment or evaluation, including before, during, or after training, validation, or testing.

Each callback type is designed for a specific set of actions, such as saving sample images or generating logging information. We can apply multiple callbacks to enable different sets of actions at the same time.

This section is only about configuring callbacks other than metric callbacks. To configure metric callbacks, please refer to Configure Metrics section.

Callbacks are a sub-config under the index config of:

  • Continual learning main experiment and evaluation
  • Continual learning full experiment and the reference experiments
  • Continual unlearning main experiment and evaluation
  • Continual unlearning full experiment, the reference experiments and evaluation
  • Multi-task learning experiment and evaluation
  • Single-task learning experiment and evaluation

To configure custom callbacks, create a YAML file in the callbacks/ folder. Note that callbacks are shared in different continual learning tasks. Below is an example of the callbacks config.

Example

configs
β”œβ”€β”€ __init__.py
β”œβ”€β”€ entrance.yaml
β”œβ”€β”€ index
β”‚   β”œβ”€β”€ example_cl_main_expr.yaml
β”‚   └── ...
β”œβ”€β”€ callbacks
β”‚   β”œβ”€β”€ cl_default.yaml
...
example_configs/index/example_cl_main_expr.yaml
defaults:
  ...
  - /callbacks: cl_default.yaml
  ...

The callbacks config is a list of callback objects:

example_configs/callbacks/cl_default.yaml
- _target_: clarena.callbacks.CLPylogger
- _target_: clarena.callbacks.CLRichProgressBar
- _target_: clarena.callbacks.SaveFirstBatchImages
  save_dir: ${output_dir}/samples/
- _target_: clarena.callbacks.SaveModels
  save_dir: ${output_dir}/saved_models/

Supported Callbacks & Required Config Fields

All Lightning built-in callbacks are supported. In CLArena, we have also implemented many callbacks as Python classes in the clarena.callbacks module that you can use for your experiments and evaluations.

To choose a callback, assign the _target_ field to the corresponding class name, such as lightning.pytorch.callbacks.EarlyStopping for EarlyStopping. Each callback has its own hyperparameters and configurations, which means it has its own required fields. The required fields are the same as the arguments of the class specified by _target_. The arguments for each callback class can be found in the PyTorch Lightning documentation and CLArena API documentation.

PyTorch Lightning Documentation (Built-In Callbacks) API Reference (Callbacks) Source Code (Callbacks)

Below is the full list of supported callbacks. Note that the names in the β€œCallback” column are the exact class names that you should assign to _target_.

Continual Learning Callbacks

These callbacks can be applied to continual learning and unlearning.

Callback Description Required Config Fields
SaveModels Saves the model at the end of each training task. See Save and Evaluate Model (CL Main) Same as SaveModels class arguments
CLPylogger Provides additional logging messages for during continual learning progress Same as CLPylogger class arguments
SaveFirstBatchImages Saves images and labels of the first batch of training data into files. Applies to all tasks Same as SaveFirstBatchImages class arguments
CLRichProgressBar Customised RichProgressBar for continual learning Same as CLRichProgressBar class arguments

Multi-Task Learning Callbacks

These callbacks can be applied to multi-task learning.

Callback Description Required Config Fields
SaveModels Saves the model at the end of training. See Save and Evaluate Model (MTL) Same as SaveModels class arguments
SaveFirstBatchImages Saves images and labels of the first batch of training data into files Same as SaveFirstBatchImages class arguments
MTLPylogger Provides additional logging messages for during multi-task learning progress Same as MTLPylogger class arguments

Single-Task Learning Callbacks

These callbacks can be applied to single-task learning.

Callback Description Required Config Fields
SaveModels Saves the model at the end of training. See Save and Evaluate Model (STL) Same as SaveModels class arguments
STLPylogger Provides additional logging messages for during single-task learning progress Same as STLPylogger class arguments
SaveFirstBatchImages Saves images and labels of the first batch of training data into files Same as SaveFirstBatchImages class arguments
Back to top
Lightning Loggers
Other Configs
 
 

©️ 2025 Pengxiang Wang. All rights reserved.