Configure Backbone Network (STL)

Modified

August 26, 2025

The backbone network refers to the feature extractor that precedes the output heads. Typically, the backbone is shared and fixed: a neural network whose head is truncated so it outputs a feature vector. However, in some CL approaches (particularly architecture-based approaches), the backbone is dynamic: it can expand, incorporate additional mechanisms like masks, or even assign different networks to each task. If you are not familiar with continual learning backbones, feel free to learn more in my beginners’ guide: backbone network and architecture-based approaches.

The backbone is a sub-config under the experiment index config (CL Main). To configure a custom backbone, create a YAML file in the backbone/ folder. Below is an example of the backbone config.

Example

configs
├── __init__.py
├── entrance.yaml
├── experiment
│   ├── example_clmain_train.yaml
│   └── ...
├── backbone
│   └── clmlp.yaml
...

configs/experiment/example_clmain_train.yaml

defaults:
  ...
  - /backbone: clmlp.yaml
  ...

configs/backbone/clmlp.yaml

_target_: clarena.backbones.CLMLP
input_dim: 784
hidden_dims: [256, 100]
output_dim: 64
batch_normalisation: true

Supported Backbones & Required Config Fields

In CLArena, we have implemented many CL backbone networks as Python classes in the clarena.backbones module that you can use for your experiments.

To choose a backbone, assign the _target_ field to the class name of the backbone. For example, to use CLMLP, set the _target_ field to clarena.backbones.CLMLP. Each backbone has its own hyperparameters and configurations, which means it has its own required fields. The required fields are the same as the arguments of the class specified by _target_. The arguments for each backbone class can be found in the API documentation.

API Reference (Backbones) Source Code (Backbone Network)

Below is the full list of supported CL backbone networks. Note that the names in the “Backbone” column are the exact class names that you should assign to _target_.

General

These backbones can be generally used in the continual learning main experiments, unless noted otherwise.

Backbone	Description	Required Config Fields
CLMLP	Multi-layer Perceptron (MLP). a fully-connected network with all linear layers. The hidden dimension and number of layers are customizable.	Same as CLMLP class arguments
CLResNet18	A smaller version of ResNet. ResNet is a convolutional network with residual connections.	Same as CLResNet18 class arguments
CLResNet34	A smaller version of ResNet. ResNet is a convolutional network with residual connections.	Same as CLResNet34 class arguments
CLResNet52	A smaller version of ResNet. ResNet is a convolutional network with residual connections.	Same as CLResNet52 class arguments
CLResNet101	A larger version of ResNet. ResNet is a convolutional network with residual connections.	Same as CLResNet101 class arguments
CLResNet152	A larger version of ResNet. ResNet is a convolutional network with residual connections.	Same as CLResNet152 class arguments

For architecture-based continual learning, the backbone is specialized for the algorithm by introducing new mechanisms like masks. We provide specialized backbones for several architecture-based CL algorithms.

HAT

These backbones should be used with CL algorithm HAT and its extensions AdaHAT, FGAdaHAT. Please refer to Configure CL Algorithm (CL Main) section.

Specialized Backbone	Description	Required Config Fields
HATMaskMLP	HAT-masked version of MLP.	Same as HATMaskMLP class arguments
HATMaskResNet18	HAT-masked version of ResNet18.	Same as HATMaskResNet18 class arguments
HATMaskResNet34	HAT-masked version of ResNet34.	Same as HATMaskResNet34 class arguments
HATMaskResNet52	HAT-masked version of ResNet52.	Same as HATMaskResNet52 class arguments
HATMaskResNet101	HAT-masked version of ResNet101.	Same as HATMaskResNet101 class arguments
HATMaskResNet152	HAT-masked version of ResNet152.	Same as HATMaskResNet152 class arguments

WSN

These backbones should be used with CL algorithm WSN. Please refer to Configure CL Algorithm (CL Main) section.

Specialized Backbone	Description	Required Config Fields
WSNMaskMLP	WSN masked version of MLP.	Same as WSNMaskMLP class arguments

The backbone network refers to the feature extractor before the output heads. The backbone is shared, which is a neural network that truncates the head and output a feature.

Backbone is a sub-config under the experiment index config (MTL). To configure a custom backbone, you need to create a YAML file in backbone/ folder. Below shows an example of the backbone config.

Example

configs
├── __init__.py
├── entrance.yaml
├── experiment
│   ├── example_mtl_train.yaml
│   └── ...
├── backbone
│   └── mlp.yaml
...

configs/experiment/example_mtl_train.yaml

defaults:
  ...
  - /backbone: mlp.yaml
  ...

configs/backbone/mlp.yaml

_target_: clarena.backbones.MLP
input_dim: 784
hidden_dims: [256, 100]
output_dim: 64
batch_normalisation: True

Supported Backbone & Required Config Fields

In CLArena, we implemented many MTL backbone networks as Python classes in clarena.backbones module that you can use for your experiment.

To choose a MTL backbone network, assign the _target_ field to the class name of the MTL backbone network. For example, to use MLP, set _target_ field to clarena.backbones.MLP. Each backbone network has its own hyperparameters and configurations, which means it has its own required fields. The required fields are the same as the arguments of the class specified by _target_. The arguments of each backbone class can be found in API documentation.

API Reference (Backbones) Source Code (Backbone Network)

Below is the full list of supported MTL backbone networks. Note that the “Backbone” is exactly the class name that the _target_ field is assigned.

General

These backbones can be generally used unless noted otherwise.

Backbone	Description	Required Config Fields
MLP	Multi-layer Perceptron (MLP). a fully-connected network with all linear layers. The hidden dimension and number of layers are customizable.	Same as MLP class arguments
ResNet18	A smaller version of ResNet. ResNet is a convolutional network with residual connections.	Same as ResNet18 class arguments
ResNet34	A smaller version of ResNet. ResNet is a convolutional network with residual connections.	Same as ResNet34 class arguments
ResNet52	A smaller version of ResNet. ResNet is a convolutional network with residual connections.	Same as ResNet52 class arguments
ResNet101	A larger version of ResNet. ResNet is a convolutional network with residual connections.	Same as ResNet101 class arguments
ResNet152	A larger version of ResNet. ResNet is a convolutional network with residual connections.	Same as ResNet152 class arguments

The backbone network refers to the feature extractor before the output head. The backbone is shared, which is a neural network that truncates the head and output a feature.

Backbone is a sub-config under the experiment index config (STL). To configure a custom backbone, you need to create a YAML file in backbone/ folder. Below shows an example of the backbone config.

Example

configs
├── __init__.py
├── entrance.yaml
├── experiment
│   ├── example_stl_train.yaml
│   └── ...
├── backbone
│   └── mlp.yaml
...

configs/experiment/example_stl_train.yaml

defaults:
  ...
  - /backbone: mlp.yaml
  ...

configs/backbone/mlp.yaml

_target_: clarena.backbones.MLP
input_dim: 784
hidden_dims: [256, 100]
output_dim: 64
batch_normalisation: true

Supported Backbone & Required Config Fields

In CLArena, we implemented many STL backbone networks as Python classes in clarena.backbones module that you can use for your experiment.

To choose a STL backbone network, assign the _target_ field to the class name of the STL backbone network. For example, to use MLP, set _target_ field to clarena.backbones.MLP. Each backbone network has its own hyperparameters and configurations, which means it has its own required fields. The required fields are the same as the arguments of the class specified by _target_. The arguments of each backbone class can be found in API documentation.

API Reference (Backbones) Source Code (Backbone Network)

Below is the full list of supported STL backbone networks. Note that the “Backbone” is exactly the class name that the _target_ field is assigned.

General

These backbones can be generally used unless noted otherwise.

Backbone	Description	Required Config Fields
MLP	Multi-layer Perceptron (MLP). a fully-connected network with all linear layers. The hidden dimension and number of layers are customizable.	Same as MLP class arguments
ResNet18	A smaller version of ResNet. ResNet is a convolutional network with residual connections.	Same as ResNet18 class arguments
ResNet34	A smaller version of ResNet. ResNet is a convolutional network with residual connections.	Same as ResNet34 class arguments
ResNet52	A smaller version of ResNet. ResNet is a convolutional network with residual connections.	Same as ResNet52 class arguments
ResNet101	A larger version of ResNet. ResNet is a convolutional network with residual connections.	Same as ResNet101 class arguments
ResNet152	A larger version of ResNet. ResNet is a convolutional network with residual connections.	Same as ResNet152 class arguments