When Safety Routing Breaks: Understanding Alignment Fragility under Benign Fine-Tuning
Researchers demonstrate that even benign fine-tuning can severely degrade LLM safety alignment by selectively sharpening pathways in model internals that bypass refusal mechanisms, revealing a previously underexamined geometric explanation for alignment fragility.
Why this matters
This work identifies why standard fine-tuning on benign tasks causes large language models to lose their safety training even when no adversarial data is involved. The authors reject the common explanation of gradient conflict and instead propose that safety mechanisms rely on a low-rank geometric structure in the model's parameter space. During fine-tuning, this structure flattens in most of the model, but crucially, a specific pathway re-sharpens in the output layers—the components responsible for choosing what to generate. This selective re-sharpening allows the model to route around its safety mechanisms despite retaining the underlying representations that support refusal. After just a hundred benign fine-tuning steps, attack success rates can climb dramatically while normal model capabilities degrade only slightly. This asymmetry is important because it shows safety is surprisingly easy to break compared to general performance. The finding that safety recovers quickly when shown just a few safety examples during fine-tuning suggests the safety knowledge was never truly lost, only temporarily inaccessible. The analysis also examines why some training techniques like LoRA and ASAM provide temporary protection by keeping output layers from sharpening, though this protection erodes as fine-tuning scales up. For builders, this suggests that the fragility of alignment goes deeper than surface-level training dynamics, and that understanding the geometric properties of safety mechanisms may be needed to make them more robust to everyday model updates.
Check the original work
This explanation is Korpalis’s guide to the material, not a replacement for it. Read the publisher’s page for the full method, evidence and limitations.