Continual Learning (CL)

Continual Learning (CL) is a machine learning paradigm that involves training on multiple tasks sequentially while maintaining performance across all previously learned tasks. Please refer to my article Continual Learning Beginners’ Guide.

Definition

In CLArena, continual learning is specifically designed for classification problems and follows the formal definition below.

Definition 1 (Continual Learning Classification) Given:

• An initialized neural network model $f$ consisting of:
  • A shared backbone network $B$
  • Task-specific output heads
• Sequential tasks: $t=t_1,t_2,\dots$ with task IDs $t_k\in {\mathbb{N}}^{+}$
• For each task $t$, we have:
  • Training data: ${\mathcal{D}}_{\text{train}}^{(t)}=\{({\mathbf{x}}_i,y_i){\}}_{i=1}^{N_t}\in ({\mathcal{X}}^{(t)},{\mathcal{Y}}^{(t)})$
  • Validation data: ${\mathcal{D}}_{\text{val}}^{(t)}\in ({\mathcal{X}}^{(t)},{\mathcal{Y}}^{(t)})$
  • Test data: ${\mathcal{D}}_{\text{test}}^{(t)}\in ({\mathcal{X}}^{(t)},{\mathcal{Y}}^{(t)})$

Objective: Develop an algorithm that updates the model from $f^{(t-1)}$ to $f^{(t)}$ when learning task $t$, such that:

• Only current task data ${\mathcal{D}}_{\text{train}}^{(t)}$ and ${\mathcal{D}}_{\text{val}}^{(t)}$ are accessible
• Maintain good performance on test datasets of all seen tasks: ${\mathcal{D}}_{\text{test}}^{(t_1)},\cdots ,{\mathcal{D}}_{\text{test}}^{(t)}$

Note

Continual learning by definition has no knowledge of future tasks, however, the task IDs to train is set beforehand in an experiment (see config field train_tasks in Experiment Index Config of CL_MAIN_EXPR). The continual learning algorithm must not access any information about future tasks during training, for example, allocate space preemptively.

Supported Paradigms

CLArena supports two primary continual learning paradigms for classification:

Task-Incremental Learning (TIL)

In TIL, each new task introduces an independent classification head. When a new task arrives, the model adds a separate output head that operates independently from previous tasks.

Class-Incremental Learning (CIL)

In CIL, new classes are incrementally added to a single, growing classification head. The output head concatenates new classes to the existing classification space.

The key architectural difference between these paradigms lies in their output head structure, as illustrated below. For detailed explanations, see my continual learning beginners’ guide and the source code.

Figure 1: Architectural comparison of TIL and CIL output heads.

Supported Pipelines

CLArena supports the following experiment and evaluation pipelines for continual learning:

• Continual Learning Main Experiment: The primary experiment for training and evaluating continual learning models. See Continual Learning Main Experiment.
• Continual Learning Main Evaluation: The evaluation phase for assessing the performance of the trained continual learning models. See Save and Evaluate Model.
• Continual Learning Full Experiment: A comprehensive experiment that evaluates more metrics for continual learning. This involves the main experiment, additional reference experiments and continual learning full evaluation based on these results. See Continual Learning Full Experiment.
• Reference Joint Learning Experiment (Continual Learning): The reference joint learning experiment for full evaluation. See Reference Joint Learning Experiment.
• Reference Independent Learning Experiment (Continual Learning): The reference independent learning experiment for full evaluation. See Reference Independent Learning Experiment.
• Reference Random Learning Experiment (Continual Learning): The reference random learning experiment for full evaluation. See Reference Random Learning Experiment.
• Continual Learning Full Evaluation: The evaluation phase of the full experiment. See Full Evaluation.