Configure Your Backbone Network

The backbone network acts the feature extractor before the output heads. Typically, the backbone is shared and fixed, which is a neural network that truncates the head and output a feature. Hoewever, in some CL approaches (particularly in architecture-based approaches), the backbone is dynamic which can expand, incorporate additional mechanism like masks, or even assign lots of individual networks uniquely to each task. If you are not familiar with the network architecture in continual learning, feel free to get some knowledge from my CL beginner’s guide about the architecture and architecture-based approaches.

To configure the backbone neural network for your experiment, link the backbone field in the main YAML file to sub-config file in the sub-directory /backbone/ , and specify the class and its arguments in the sub-config. In this package we pre-defined many backbone classes in clarena.backbones, see the full list below.

Here is an example:

./clarena/example_configs
├── __init__.py
├── entrance.yaml
├── experiment
│   ├── example.yaml
│   └── ...
├── backbone
│   └── mlp.yaml
...

example_configs/experiment/example.yaml

defaults:
  ...
  - /backbone: mlp.yaml
  ...

example_configs/backbone/mlp.yaml

_target_: clarena.backbones.MLP
input_dim: 784
hidden_dims: [256, 100]
output_dim: 64

Supported Backbone List

Here is the full list of supported backbone networks so far. This includes fixed (inherited from built-in torch.nn.Module networks in PyTorch) and dynamic networks changing with the task. To select a backbone, replace with the class name. Please refer to the documentation of each class to know what fields (arguments) are required for the class.

Backbone	Class Name	Description
MLP	MLP	Fully connected network with all linear layers. The hidden dimension and number of layers are customizable.