Catastrophic Learning Attacks

Arxiv pdf 2026-08-01T00:00:00
arXiv Paper — PDF not available. Only the Executive Summary is available here. To read or download the full paper, visit the arXiv abstract page.

Abstract

Continual Learning (CL) enables deep learning models to iteratively learn from a stream of data without forgetting prior knowledge. Existing adversarial research on CL primarily aims to re-enable catastrophic forgetting, thus attacking stability and reducing availability. In this work, we identify a novel security flaw: data manipulated by an attacker can reduce the learnability of the current or of upcoming iterations. We term such manipulations learning blockers, as they specifically attack the plasticity of CL algorithms. Learning blockers are particularly harmful because they are difficult to detect during training and evaluation of the current iteration, since they can target iterations whose data the model has not yet encountered. When learning blockers additionally induce catastrophic forgetting, the resulting overall degradation is what we call catastrophic learning. We formalize this scenario, define an associated threat model, and propose six attack strategies: LabelExchange, Tensor-Exchange, Attraction-Coincident, AttractionPreceding, Repulsion-Coincident, and Repulsion-Preceding. The Attraction variants minimize the loss between the poisoned and the victim iteration label, pulling their representations together in feature space; the Repulsion variants maximize this loss, pushing them apart so the stability mechanism resists the required parameter shift. In the Coincident variants, the poisoned and the victim iteration coincide, using a clean reference iteration only as a label source; in the Preceding variants, the poisoned iteration precedes the victim, leaving it clean but not learnable due to distorted representations learned beforehand. We evaluate the attacks on MNIST and Split-CIFAR10 against three CL strategiesDER, ER-ACE, and iCaRLacross more than 4,480 simulations. Our results demonstrate a strong vulnerability of CL algorithms: an adversary can selectively impede plasticity to hinder the acquisition of new knowledge, while simultaneously promoting the loss of previously learned information, thereby inducing a catastrophic learning scenario.

Loading executive summary...

LINK COPIED TO CLIPBOARD