Skip to main content

Meta-Continual Learning

Meta-learning approaches aim to learn initialization, update rules, or representations that are inherently amenable to continual learning. The key idea is to "learn to learn continually": rather than designing anti-forgetting mechanisms by hand, meta-learning discovers them automatically by training on sequences of tasks and optimizing for the ability to learn new tasks without forgetting old ones. This family bridges continual learning with the broader meta-learning literature (Hospedales et al., 2021).

Online Meta-Learning (OML)​

Javed and White (2019) (Javed & White, 2019) proposed OML (originally introduced under the name Meta-learned Representations for Continual Learning, MRCL), which meta-learns a representation that is robust to catastrophic forgetting. The architecture separates the network into a representation learning network (RLN) and a prediction learning network (PLN). The RLN is meta-trained using a MAML-like outer loop (Finn et al., 2017) where the meta-objective explicitly penalizes forgetting: after the inner loop updates the PLN on a sequence of tasks, the outer loop adjusts the RLN so that the learned representation supports learning new tasks without degrading performance on earlier ones. OML demonstrated that representations can be meta-trained to be inherently continual-learning-friendly, generalizing well to long task sequences (up to 200 tasks) not seen during meta-training.

A Neuromodulated Meta-Learning Algorithm (ANML)​

Beaulieu et al. (2020) (Beaulieu et al., 2020) introduced ANML, which draws inspiration from neuromodulation in biological brains (the process by which neuromodulatory signals such as dopamine and acetylcholine selectively gate plasticity in neural circuits). ANML adds a neuromodulatory network that produces activation gates for the prediction network during learning. These gates determine which neurons are allowed to update, selectively protecting important features while allowing plasticity for new learning.

The neuromodulatory network is meta-learned end-to-end, so it automatically discovers task-dependent gating patterns that minimize forgetting. On the Omniglot sequential-class benchmark used by both methods, Beaulieu et al. report that ANML retains higher accuracy than OML as the number of sequentially learned classes grows, indicating better scalability to long task sequences. The biological plausibility of the neuromodulatory mechanism is an appealing feature, connecting algorithmic continual learning research to neuroscience.

La-MAML​

Gupta et al. (2020) (Gupta et al., 2020) proposed La-MAML (Look-Ahead Meta Learning for Continual Learning), which takes a different approach to meta-continual learning. Rather than meta-learning representations, La-MAML meta-learns per-parameter learning rates. The key idea is that parameters important for previous tasks should have low learning rates (to prevent forgetting), while parameters less important for previous tasks can have higher learning rates (to enable plasticity). La-MAML sets these learning rates by optimizing a "look-ahead" loss that evaluates the effect of a gradient update on both the current task and a buffer of samples from previous tasks.

La-MAML achieves strong performance on both task-incremental and class-incremental settings while being more computationally efficient than standard MAML-based approaches (it avoids the expensive second-order gradient computation of MAML through a first-order approximation). Because it learns per-parameter learning rates rather than a frozen representation, La-MAML applies directly to standard image-classification continual learning benchmarks (CIFAR and TinyImageNet splits) where the representation-centric OML and ANML were not originally evaluated.

CLS-ER: Experience Replay with Complementary Learning Systems​

Arani et al. (2022) (Arani et al., 2022) proposed CLS-ER, which implements a dual-memory system inspired by the complementary learning systems (CLS) theory from neuroscience (Kumaran et al., 2016). CLS-ER maintains two model copies: a "working model" that learns quickly (high plasticity) and a "stable model" that consolidates knowledge slowly (high stability). The stable model is updated as an exponential moving average of the working model, and both models provide complementary supervision during replay. This architecture mirrors the hippocampal-cortical interaction in biological memory systems and achieves strong performance, particularly on challenging class-incremental benchmarks.

Comparing the Approaches​

The four methods above differ along three axes: what is meta-learned, whether replay is required, and the deployment regimes they target. OML and ANML meta-learn a representation (with ANML adding a neuromodulatory gating network) and are evaluated mainly in the few-shot, many-class regime of sequential Omniglot, where ANML retains accuracy better than OML as the task sequence lengthens. La-MAML instead meta-learns per-parameter learning rates over a small replay buffer, which lets it operate on standard CIFAR and TinyImageNet continual learning splits at lower computational cost than full second-order MAML. CLS-ER drops meta-learning altogether in favor of a dual-model exponential-moving-average scheme over experience replay, trading the up-front meta-training cost of the other three for a heavier memory and dual-model inference footprint. The methods are therefore complementary rather than strictly ordered: representation-centric meta-learning (OML, ANML) suits few-shot sequential settings, learning-rate meta-learning (La-MAML) suits buffer-based class-incremental settings, and CLS-ER offers a meta-learning-free baseline that is competitive on the same class-incremental benchmarks.

MethodWhat is meta-learnedReplay requiredPrimary evaluation regime
OMLRepresentation (RLN/PLN split)NoFew-shot sequential Omniglot
ANMLRepresentation + neuromodulatory gatesNoFew-shot sequential Omniglot
La-MAMLPer-parameter learning ratesYes (buffer)Class-incremental CIFAR / TinyImageNet
CLS-ERNothing (EMA stable model)Yes (buffer)Class-incremental benchmarks

Limitations and Open Questions​

Meta-continual learning methods face several challenges:

  1. Computational cost: The meta-training phase is computationally expensive, requiring training on many simulated task sequences. OML and ANML both require thousands of meta-training episodes, each involving an inner loop (sequential task learning) and outer loop (meta-gradient computation). For large models, this cost can exceed that of standard continual learning by an order of magnitude.

  2. Generalization gap: The generalization of meta-learned strategies to task distributions different from those seen during meta-training is not guaranteed. If meta-training uses 5-way classification tasks from Omniglot but deployment involves 100-way classification on ImageNet classes, the learned update rules may not transfer. Bridging this domain gap between meta-training and meta-testing distributions is an open challenge (Hospedales et al., 2021).

  3. Task boundary requirement: Most methods require explicit task boundaries during meta-training, even if the goal is task-free continual learning at deployment. Developing meta-learning approaches for task-free settings, where the meta-learner must also discover when tasks change, remains largely unexplored.

  4. Scaling: Scaling these methods to large models and long task sequences remains challenging. The memory requirements of second-order meta-gradients (needed by MAML-based approaches) grow with model size, though first-order approximations like La-MAML (Gupta et al., 2020) and Reptile (Nichol et al., 2018) partially address this.

  5. Relationship to pre-training: With the success of large pre-trained models, the role of meta-continual learning is evolving. Pre-trained models already provide a representation that is broadly useful across tasks, arguably achieving the same goal as OML's meta-learned representation but through a different mechanism (massive supervised pre-training rather than meta-learning). Whether meta-learning can add value on top of pre-training, for instance by learning task-adaptive prompting strategies, is an active area of investigation.

Nevertheless, the principle of learning the learning algorithm itself, rather than designing it by hand, is a promising direction for fundamentally solving the continual learning problem. The success of meta-learning in few-shot learning (Finn et al., 2017) suggests that analogous breakthroughs may be possible for continual learning when the right meta-objective and training procedure are identified.

References