Many organizations are more and more deploying giant language fashions (LLMs) equivalent to OpenAI’s GPT sequence, Anthropic’s Claude, Meta’s LLaMA, and varied fashions from DeepSeek, with minimal customization. This widespread reuse results in mannequin homogeneity throughout functions – from chatbots to productiveness instruments – and creates a safety vulnerability: jailbreak prompts that bypass refusal mechanisms will be precomputed as soon as and reused throughout many deployments. This mirrors the traditional rainbow desk assault in password safety, the place attackers exploit shared cryptographic targets to reuse precomputed inputs.
These generalized jailbreaks are an issue as a result of many firms have customer-facing LLMs constructed on prime of mannequin lessons – that means that one jailbreak might work in opposition to all of the situations constructed on prime of a given mannequin. And, in fact, these jailbreaks might have a number of undesirable impacts – from exposing delicate inside knowledge, to producing incorrect, inappropriate, and even dangerous responses.
Taking inspiration from password salting – the idea of introducing small per-user variations to interrupt reuse of precomputed inputs – we developed a method we name ‘LLM salting’: introducing focused variations in mannequin conduct to invalidate jailbreaks. We unveiled this method not too long ago, on the 2025 Convention on Utilized Machine Studying in Data Safety (CAMLIS), and this text explores our analysis in-depth.
Refusing to go the salt
Constructing on latest work figuring out a subspace in mannequin activations accountable for refusal conduct by Arditi et al, we developed a light-weight fine-tuning process that rotates this subspace. This easy change ensures that jailbreaks crafted in opposition to an unsalted mannequin not succeed on salted ones.
Evaluation of inside representations reveals that the refusal route stays largely steady below commonplace fine-tuning. As proven in Determine 1, the cosine similarity between the mannequin’s residual activations and a precomputed refusal route at Layer 16 stays persistently excessive all through coaching except explicitly modified. This means that alignment procedures that don’t straight goal refusal mechanisms are unlikely to disrupt the latent options exploited by jailbreak assaults.
Determine 1: Cosine similarity between the mannequin’s inside activations and the precomputed refusal route at Layer 16 throughout coaching. Beneath commonplace finetuning (white), the refusal route stays largely unchanged. In distinction, salted fine-tuning (orange) explicitly rotates the illustration away from the refusal axis. This means that commonplace alignment strategies don’t alter refusal-relevant instructions except explicitly incentivized.
In distinction, LLM salting introduces a focused perturbation that rotates this route, thereby decreasing the efficacy of beforehand profitable assaults with out adversely affecting the mannequin’s basic conduct.
We evaluated LLM salting in opposition to the Grasping Coordinate Gradient (GCG) jailbreak assault. Experiments on LLaMA2-7B-Chat and Vicuna-7B confirmed that salting persistently breaks intra-model transferability, whereas preserving the mannequin’s efficiency on benign prompts.
Importantly, LLM salting can be utilized along side present guardrail strategies equivalent to immediate filtering and classifier-based rejections. In keeping with commonplace finest safety practices, we advocate a layered protection technique, combining salting with different safeguards to enhance robustness in opposition to jailbreak assaults.
Our experiments
Coaching knowledge
We constructed the coaching dataset for finetuning by mixing examples from two sources. 90% of the information is drawn from the trl-internal-testing/hh-rlhf-helpful-base-trl-style dataset on Hugging Face, which comprises useful and innocent directions. The remaining 10% comes from AdvBench, a benchmark of dangerous prompts designed to elicit refusals in aligned fashions. This combination ensures that, throughout fine-tuning, the mannequin is uncovered to each prompts requiring useful responses and prompts requiring refusal, reinforcing the specified conduct in every case.
Analysis knowledge
To judge jailbreak transferability, we use dangerous directions and adversarial prompts from AdvBench, specializing in GCG – a suffix-based assault that appends adversarial tokens to consumer prompts. We consider on 300 GCG jailbreaks per mannequin, concentrating on two broadly adopted open-source chat fashions: LLaMA-2-7B-Chat and Vicuna-7B.
Extracting the refusal route
Following Arditi et al, we extracted a route r in activation area that mediates mannequin refusals. We undertake their difference-in-means method, evaluating residual activations following dangerous and innocent directions. Let t ∈ D be a coaching token with label yt and residual activation x(l)(t) at layer l. We partition the dataset into Ddangerous and Dinnocent relying on whether or not the immediate is meant to set off a refusal. For every transformer layer l and post-instruction token place i, we compute, as per Arditi et al:
Every candidate r(l)i represents the distinction in common activations between dangerous and innocent prompts. We consider all candidates on a held-out validation set utilizing the causal probing process from Arditi et al and choose the simplest place for r∗.
Salting through loss modification
We implement LLM salting by modifying the coaching loss to cut back alignment with the refusal route r∗ on dangerous prompts.
The overall loss is outlined as:
The loss operate contains two elements. The primary is the usual cross-entropy time period, which inspires the mannequin to generate coherent and contextually acceptable outputs. It additionally reinforces refusal conduct the place warranted—for instance, if the mannequin beforehand refused to reply a dangerous immediate, it ought to proceed to take action.
The second time period introduces the salting goal. It penalizes alignment between the mannequin’s inside activations and the precomputed refusal route r∗ on dangerous prompts, thereby encouraging the mannequin to ‘refuse in a different way’ and disrupting the activation patterns exploited by jailbreaks.
To focus this intervention the place it’s best, we apply the salting loss solely at layers with the very best cosine similarity to r∗ throughout refusals, following the method of Arditi et al. In our experiments on LLaMA-2-7B-Chat and Vicuna-7B, we use L = {16, 17, 18, 19, 20}.
Outcomes
We seeded our analysis with 300 GCG jailbreak prompts that obtain a 100% assault success charge (ASR) on the unmodified baseline fashions. We then assessed whether or not these assaults stay efficient below a spread of defenses, and whether or not our proposed salting technique can remove the subset of jailbreaks that persist.
Figures 2 and three present ASR (left axis) and Large Multitask Language Understanding (MMLU) accuracy (proper axis) for 4 mannequin variants:
- The unique mannequin with out fine-tuning (No FT)
- An ordinary fine-tuned mannequin educated on our alignment dataset (Customary FT)
- A mannequin with a (varied) modified system immediate (System Immediate Change)
- A mannequin fine-tuned with our cosine-based salting loss (Salting)
Determine 2: LLaMA2-7B: ASR of GCG jailbreaks and MMLU accuracy throughout completely different defenses. Salting reduces ASR to three% whereas preserving efficiency
Determine 3: Vicuna-7B: ASR of GCG jailbreaks and MMLU accuracy throughout completely different defenses. Salting reduces ASR to 1% whereas preserving efficiency
Jailbreak robustness
For LLaMA-2-7B (Determine 2), we observe that commonplace finetuning and system immediate adjustments scale back ASR solely partially, bringing it right down to roughly 40–60%. In distinction, salting reduces ASR from 100% to only 2.75%.
The same pattern holds for Vicuna-7B (Determine 3), the place the ASR drops from 100% to 1.35% below salting. These outcomes show that our method successfully eliminates the subset of jailbreaks that stay strong below conventional defenses, outperforming each parameter-based and prompt-based methods.
Functionality preservation
To make sure that this robustness doesn’t come at the price of mannequin utility, we consider basic capabilities with the MMLU benchmark utilizing lm-evaluation-harness. For each LLaMA-2-7B (46.8 %) and Vicuna-7B (49.2%), the salted fashions obtain MMLU accuracies which are statistically indistinguishable from their unsalted counterparts—variations are properly below typical run-to-run noise and present no systematic drift. This means that the refusal beneficial properties delivered by salting don’t compromise helpfulness or basic process efficiency.
Mannequin introspection
To know how salting disrupts jailbreak transferability, we look at the cosine similarity between residual activations and the precomputed refusal route throughout layers, simply as Arditi et al. Within the authentic mannequin, dangerous and innocent prompts exhibit a transparent separation of their alignment with the refusal route: dangerous inputs preserve excessive constructive cosine similarity, whereas innocent prompts are negatively aligned.
When GCG is utilized to a dangerous immediate, the ensuing activation similarity shifts downward, more and more resembling these of innocent inputs.
Determine 4: Cosine similarity between enter activations and the precomputed refusal route throughout layers within the authentic mannequin. Innocent and dangerous inputs are initially properly separated, however GCG-perturbed adversarial prompts (blue) more and more align with dangerous trajectories (orange) in deeper layers, revealing convergence towards refusal-triggering representations
Within the salted mannequin (Determine 5), this convergence not happens. GCG prompts stay distant from the dangerous trajectory and not shift activations into benign areas. We hypothesize that, since salting successfully inverts the refusal route, GCG’s authentic optimization now will increase alignment with the rotated vector, unintentionally reinforcing refusal conduct.
Determine 5: Cosine similarity between enter activations and the refusal route within the salted mannequin. Salting disrupts adversarial impact by rotating the activation area: GCG-modified prompts (blue) not align with dangerous representations, preserving separation from the refusal subspace
Conclusion and future work
We current LLM salting, a light-weight fine-tuning method that disrupts jailbreak reuse by rotating inside refusal representations. This method virtually solely neutralizes the success of precomputed GCG jailbreaks on each LLaMA-2 and Vicuna, whereas preserving the mannequin’s efficiency on benign inputs.
Future work might discover making use of salting to bigger fashions and evaluating its robustness in opposition to a broader vary of jailbreak methods, equivalent to AutoDAN and TAP.
