Steering Vectors Mediate Subliminal Learning in Language Models
This work elucidates subliminal learning in language models, demonstrating it is governed by steering vectors that modify model activations. The teacher's system prompt can be distilled into such a vector, which the student model learns during fine-tuning, even when the training data is semantically unrelated to the prompt's traits. This vector-mediated transfer explains the model-specific nature of subliminal learning and necessitates adaptive optimizers to capture the subtle gradient signals involved.