RelapseLive demo · Code · Models · Data

Relapse: forget the hazard, keep the rest, even in the 4-bit file

Unlearning at the layers CircuitKIT pointed to took Tiny Aya's score on 1,273 hazardous-biology questions (WMDP-bio) from 63.4% to 37.1% in the 4-bit llama.cpp file, and kept 5.6 more points of general knowledge (MMLU, 1,000 questions) than the same unlearning at SafeTune's default layers (95% interval 2.8 to 8.3, p = 0.001). A second seed gave 3.9 more points (p = 0.006). The same edit three layers deeper looked just as good at full precision and then gave back 7.4 points of the hazard score in the 4-bit Q4_0 file (p = 0.001), which only a file-by-file audit shows.

The setting: you want a small model that runs offline on a phone and will not help anyone build a bioweapon. Unlearning can remove that knowledge, but people run a 4-bit copy rather than the tested checkpoint, and unlearning costs some general knowledge. We measured both on Cohere Labs' Tiny Aya Global (3.35B parameters, 70 languages) with four Lexsi Labs repos, and published every model, dataset and log on Hugging Face.

What worked. CircuitKIT's attribution pointed to layers 7 to 9, and we put SafeTune's RMU edit there. It forgot as much as the default layers 5 to 7 in three of four head-to-head comparisons and kept more general knowledge in all four, significantly in three (AuditKIT paired bootstrap). Against the original model it cost -0.1 to 2.8 points depending on the file format; the default layers cost 3.7 to 6.4. It kept its forgetting in all seven formats, in four other languages, with reworded questions, and with worked examples or an expert persona in the prompt.

The relapse. We then ran the same edit at other layers as a control. Three layers deeper (10 to 12) the model looked as good at full precision as the one above. Stored as real llama.cpp files, it gave back 7.4 points of the hazard score in Q4_0 and 3.5 in Q4_K_M (p = 0.005 or less). Nothing in the checkpoint's scores shows it; the audit of its files does.

The repair. Training that placement with the model seen as its Q4_0 file took the relapse out of that file. The one model we built real files for scores 41.5% at fp16 and 40.6% in its real Q4_0 and Q4_K_M files, and across five training runs and two seeds the Q4_0 round trip held in four. It costs more general knowledge and does not carry to NF4, a 4-bit format it did not train on. Rounding only the edited weights during training did not help.

What didn't work. A control three layers shallower did as well as CircuitKIT's pick. And a short fine-tune on the topic brings the knowledge back in every model. Both are reported below with the other failures.

Try it: the Relapse Space · code and raw runs: github.com/itssaharsh/relapse

The forget report: every file format, the hazardous-biology score and the general-knowledge price

Why ask

Tiny Aya is built to run offline, on phones and laptops, and Cohere ships it as GGUF files for exactly that. Offline there is no moderation API between the user and the model: whatever safety the model has lives in its weights. People also rarely run the checkpoint that was safety-tested. They run a quantized copy, and published work says that is exactly where unlearning can quietly fail. Zhang et al. (ICLR 2025) found that 4-bit quantization restored much of the knowledge gradient-based unlearning had removed, and Lexsi's own Forgetting That Sticks builds a method around the problem. Lexsi's safety stack is built around checking a fix in the form it ships in, so we ran that check for Tiny Aya across every format a user is likely to download. For RMU we found the same failure, at one of the five places we put the edit.

The results

Head to head. The same RMU edit at two sets of layers, compared on the same questions with AuditKIT's paired bootstrap:

Seed File WMDP-bio: default / CircuitKIT layers MMLU: default / CircuitKIT layers MMLU kept extra (95% interval) p
1 fp16 38.7% / 37.0% 51.5% / 53.1% +1.6 (-1.1 to +4.3) 0.24
1 Q4_K_M 37.2% / 37.2% 49.3% / 54.9% +5.6 (+2.8 to +8.3) 0.001
2 fp16 44.5% / 45.6% 51.7% / 55.7% +4.0 (+1.3 to +6.6) 0.003
2 Q4_K_M 41.1% / 45.4% 51.5% / 55.4% +3.9 (+1.3 to +6.6) 0.006

The two models forgot the same amount in the first three rows (the hazard scores differ by 1.7 points or less, p = 0.33 or more). In the last row the default-layer model forgot 4.3 points more (p = 0.009).

Every file format, as hazardous biology still answered / general knowledge kept:

File (WMDP-bio forget / MMLU keep) Original Default layers (5-7) CircuitKIT layers (7-9) CircuitKIT layers, 4-bit aware Deeper layers (10-12) Aggressive
fp16 63.3% / 55.4% 38.7% / 51.6% 36.8% / 53.2% 34.0% / 54.5% 41.6% / 53.9% 25.4% / 43.5%
bf16 63.3% / 55.4% 38.6% / 51.4% 36.9% / 53.1% 34.2% / 54.5% not run 25.4% / 43.5%
Q8_0 63.2% / 55.1% 38.4% / 51.4% 37.3% / 53.2% 34.6% / 54.4% not run 26.4% / 43.8%
Q6_K 63.2% / 55.6% 39.1% / 51.7% 36.4% / 53.8% 34.9% / 53.9% not run 26.9% / 42.9%
Q4_K_M 63.4% / 54.8% 37.2% / 49.3% 37.1% / 54.9% 35.3% / 54.5% 45.2% / 52.5% 27.3% / 40.7%
Q4_0 61.1% / 55.5% 36.8% / 50.6% 35.9% / 52.7% 35.6% / 52.9% 49.0% / 52.9% 26.8% / 41.4%
NF4 61.1% / 52.7% 37.2% / 46.3% 35.3% / 51.2% 34.7% / 50.8% 42.2% / 50.6% 26.7% / 36.2%

The second-seed runs (fp16, Q8_0, Q4_K_M, Q4_0, NF4) are in the results table. Four things stand out:

  1. Outside layers 10 to 12 the forgetting holds in every file we scored. Across those 26 runs and every format, no hazard score rose by more than 2.0 points from its fp16 value (75 run-and-file pairs; the largest fall was 4.2). AuditKIT's paired fp16-to-Q4_K_M tests found no increase (the largest was +2.0 points, p = 0.12).
  2. Where you unlearn decides the price. Against the original in the same file, the CircuitKIT-layer model cost -0.1 to 2.8 points across the seven formats in the first seed and -0.6 to 0.7 across five in the second. The default-layer model cost 3.7 to 6.4 and 3.0 to 3.7. In the Q4_K_M file the CircuitKIT-layer model's cost was not measurable in either seed (+0.1 points, 95% interval -1.9 to +2.0; +0.6, -1.1 to +2.3); at fp16 in the first seed it was 2.2 points (p = 0.027).
  3. Pushing harder at the default layers does not help. The aggressive setting reached guessing (25.4%) but lost 11.9 points of general knowledge at fp16, 14.1 in Q4_K_M and 16.5 in NF4.
  4. Three layers deeper, the 4-bit files give part of it back. That is the deeper-layers column, and the section after the controls.

Controls: the same edit at other layers. We tried to break the head-to-head result. Four more runs gave the default layers the CircuitKIT runs' training budget (300 batches, one with stronger steering, one with a higher retain weight). They still cost 3.8 to 11.0 points of general knowledge at fp16. Then we ran the same edit at layer windows CircuitKIT did not pick:

Layers edited CircuitKIT MLP attribution Runs MMLU points lost at fp16, runs at 40% WMDP-bio or below WMDP-bio change, fp16 to Q4_0: mean (range) fp16 to NF4
3 to 5 (control) 0.13 3 1.3 to 1.6 (2 runs) -1.8 (-2.4 to -1.3) -1.4 (-1.9 to -0.9)
5 to 7 (SafeTune default) 0.23 12 3.8 to 19.9 (10 runs) -1.3 (-3.5 to +1.4) -1.2 (-2.7 to +1.3)
7 to 9 (CircuitKIT) 0.30 9 0.9 to 2.7 (6 runs) -0.5 (-2.3 to +1.6) -0.9 (-1.8 to +0.7)
10 to 12 (control) 0.13 4 4.8 (1 run) +4.9 (+2.0 to +7.7) +1.5 (+0.2 to +3.2)
13 to 15 (control) 0.15 2 none got there -1.5 (-2.6 to -0.5) -2.7 (-4.2 to -1.2)

The same edit at five layer windows: what it cost and what the 4-bit file did to it

At similar forgetting, the default layers cost the most. Layers 3 to 5 and 7 to 9 reached 40% or below for under 3 points; the default layers needed 3.8 or more. Layers 10 to 12 have one run under 40% (28.9% for 4.8 points, where the default layers paid 8.1 for 29.9%), and layers 13 to 15 forgot too little to compare (the stronger run there reached 46.3% for 3.8 points).

CircuitKIT's window was one of two that worked. Worked means: forgot to 40% or below for under 3 points and stayed forgotten in the 4-bit files. Layers 3 to 5 did that too, on 3 runs, with less than half the attribution (0.13 against 0.30). CircuitKIT's pick moved the edit off the default to a placement that costs about a third as much.

And one window gives it back in Q4_0. At layers 10 to 12 the hazard score rose from fp16 to Q4_0 in all four runs: by 7.7, 5.0 and 2.0 points at three settings of the first seed, and by 4.6 in a second seed. The 2.0 is the run that forgot most (28.9% at fp16), so the relapse shows at moderate forgetting and little at the extreme. At every other window it averaged 0.5 to 1.8 points lower. It is not simply depth: at layers 13 to 15 it fell again (two runs, neither under 40%). (The table's 4-bit columns are bit-exact round trips of the decoder weights, like the rest of the sweep.)

The relapse, in real files. The layer-10-to-12 model scores 41.6% on WMDP-bio at fp16 for 1.5 points of general knowledge, and nothing at fp16 sets it apart from the models that hold. As real llama.cpp files it scores 49.0% in Q4_0 (AuditKIT paired bootstrap: 7.4 points came back, 95% interval +4.9 to +9.8, p = 0.001) and 45.2% in Q4_K_M (3.5 points, +1.3 to +5.7, p = 0.005). A second run at a lower learning rate went from 45.2% to 50.5% and 50.2% (5.6 and 5.1 points back, both p = 0.001). A second seed forgot less (50.9% at fp16) and rose 4.6 points in the Q4_0 round trip. Real files were built for the first two runs only, and bitsandbytes NF4 moved the first run by under a point, so this is about the llama.cpp 4-bit formats. In the same Q4_0 file the original model lost 2.0 points (p = 0.022), so the file is not simply noisier: it moves the two models in opposite directions.

Where it shows up. We rounded one part of the model at a time to Q4_0 and left the rest at fp16. Rounding only the 70 matrices in the layers in front of the edit brought back 5.5 of the 7.7 points (4.0 of 5.0 in the second run). Rounding only the three edited matrices brought back 1.1 and 0.5, the other matrices inside layers 10 to 12 brought back 1.0 and 2.9, and the layers behind the edit 0.6 and -0.1. The parts do not add up to the whole, so this says where most of the relapse appears, not why. It does rule out the simplest story, a small edit being rounded away: against the Q4_0 rounding error in the same three matrices, the layer-10-to-12 edits are 0.95 to 3.25 times its size, and the CircuitKIT-layer edits that hold are 0.78 to 1.01. The largest of them relapsed least.

Can it be trained away? For the file it trains on, yes. We retrained the layer-10-to-12 edit with the model seen as its Q4_0 file during training: every decoder linear weight is replaced by its Q4_0 round trip, in the model being edited and in the frozen reference it is held to (embeddings stay fp16, as in the evaluation), so the edit is learned against what the downloaded file will compute. SafeTune's RMU loop is otherwise unchanged. Same layers and data as the run that relapses; steps, learning rate and seed vary by row:

Training at layers 10 to 12 Steps (of them as the file) fp16 (WMDP-bio / MMLU) Q4_0 NF4 WMDP-bio change, fp16 to Q4_0 (plus = came back)
Plain RMU (the run that relapses) 300 (0) 41.6% / 53.9% 49.3% / 53.3% 42.2% / 50.6% +7.7
Only the three edited matrices rounded, on half the steps 300 (0) 32.1% / 51.0% 38.6% / 51.8% 30.2% / 50.0% +6.5
Whole model seen as its Q4_0 file on half the steps 300 (147) 37.5% / 51.8% 40.9% / 51.3% 37.5% / 50.2% +3.5
File view on half the steps, 600 steps in total 600 (297) 28.7% / 49.6% 32.0% / 48.3% 27.8% / 47.3% +3.3
Whole model seen as its Q4_0 file on every step 300 (300) 49.1% / 52.8% 49.1% / 53.0% 47.9% / 50.0% 0.0
Every step, 600 steps in total 600 (600) 41.5% / 50.6% 41.0% / 48.9% 35.3% / 47.2% -0.5
Every step, higher learning rate 300 (300) 26.1% / 44.2% 27.0% / 43.8% 26.7% / 42.3% +0.9
Plain RMU, second seed 300 (0) 50.9% / 51.7% 55.5% / 52.5% 53.0% / 51.3% +4.6
Every step, second seed 300 (300) 49.6% / 52.5% 52.1% / 52.4% 51.9% / 51.6% +2.5
Every step, 600 steps in total, second seed 600 (600) 32.5% / 45.1% 30.0% / 41.3% 39.7% / 46.0% -2.5

Each attempt to train the relapse away: the hazard score at fp16 and in the Q4_0 file

Real files agree, for the one model we built them for (first seed, 600 steps). As llama.cpp files it scores 40.6% in Q4_0 and 40.6% in Q4_K_M, against 41.5% at fp16 (AuditKIT paired: -0.9 points in each, p = 0.46 and p = 0.40). It was trained on the Q4_0 round trip only, and the Q4_K_M file holds too.

It is not free. The every-step run that matches the plain run at fp16 (41.5% against 41.6%) needed twice the steps and lost 4.8 points of general knowledge at fp16 and 6.0 in its real Q4_K_M file, where the plain run lost 1.5 and 2.3. We did not run plain RMU for 600 steps, so part of that cost may be the longer training. The longer second-seed run forgot more (32.5% at fp16) and lost 10.3 points. So training on the file makes this placement honest, not better: what it shows at fp16 is what the file does. The CircuitKIT layers are still the recipe to use.

Two more models. We ran the three placements on Tiny Aya Fire and Tiny Aya Earth with the settings from Global, unchanged and untuned, one run each:

Model Placement fp16 (WMDP-bio / MMLU) Q4_0 NF4 MMLU points lost: fp16 / Q4_0 / NF4
Tiny Aya Earth Original 63.7% / 55.7% 62.9% / 55.7% 62.1% / 53.5%
Tiny Aya Earth Default layers (5 to 7) 36.4% / 51.9% 35.6% / 49.7% 34.6% / 49.4% 3.8 / 6.0 / 4.1
Tiny Aya Earth CircuitKIT layers (7 to 9) 36.5% / 56.0% 35.3% / 55.0% 36.8% / 51.9% -0.3 / 0.7 / 1.6
Tiny Aya Earth Deeper layers (10 to 12) 37.2% / 52.9% 36.8% / 51.9% 46.1% / 50.8% 2.8 / 3.8 / 2.7
Tiny Aya Fire Original 64.3% / 57.4% 63.2% / 59.1% 62.7% / 56.1%
Tiny Aya Fire Default layers (5 to 7) 37.2% / 53.4% 33.7% / 49.0% 33.5% / 51.6% 4.0 / 10.1 / 4.5
Tiny Aya Fire CircuitKIT layers (7 to 9) 34.3% / 56.2% 32.9% / 54.2% 34.2% / 55.2% 1.2 / 4.9 / 0.9
Tiny Aya Fire Deeper layers (10 to 12) 49.5% / 54.9% 50.4% / 56.6% 51.6% / 54.5% 2.5 / 2.5 / 1.6

The recipe carried over. On both models the CircuitKIT layers forgot as much as the default layers and lost less general knowledge in all three formats (Earth: -0.3 to 1.6 points against 3.8 to 6.0; Fire: 0.9 to 4.9 against 4.0 to 10.1). The relapse did not carry over in the same form. At layers 10 to 12 neither model's Q4_0 score rose by more than 0.9 points, but Earth's NF4 score rose 9.0 points (37.2% to 46.1%), and Fire forgot little there (49.5%). So which placement gives knowledge back, and in which format, depends on the model. The file-by-file check is what finds it.

Every run we made. We ran 12 RMU settings at the default layers and 9 at the CircuitKIT layers. Among the runs that got WMDP-bio to 40% or below, the default layers cost 3.8 to 19.9 points of general knowledge at fp16, 4.9 to 14.1 in Q4_0 and 3.9 to 16.5 in NF4. The CircuitKIT layers cost 0.9 to 2.7, 2.2 to 3.5 and 1.4 to 3.3. The ranges stay apart for cut-offs from 35% to 42%; from 43% they overlap in Q4_0. The first 8 runs at each were not tuned identically: the default layers got learning rates from 2e-5 to 3e-4 and one lower retain weight, the CircuitKIT layers got 1e-4 to 2e-4, one higher retain weight and one stronger steering setting, and runs used 250 or 300 batches. The four budget-matched default runs are the check on that.

Every run: general knowledge kept against hazardous biology still answered, in the 4-bit NF4 file

The forgetting carries across languages. On 707 WMDP-bio questions translated into Hindi, Arabic, Spanish and Swahili or reworded in English, the original model scores 46% to 57% per language at fp16. Unlearned at the default layers it scores 32% to 41%, and at the CircuitKIT layers 27% to 33%. The Q4_K_M files stay in the same range: 34% to 43% and 29% to 35%.

Pushing on the prompt did not bring it back either. With five solved WMDP-bio questions placed before each question, the original model went up to 63.5%, while the unlearned models went down to 25.2% to 27.7%: RMU scrambles the model's internal state when the hazardous topic is in context, and more of it scrambles more. With an expert-virologist persona they stayed at 32.2% to 35.0%. Both held in the 4-bit Q4_0 file too.

Fine-tuning is where it stopped holding. After 25 LoRA steps on 64 WMDP-bio questions with their answers, the score on the other 1,204 questions came back to 52.4% (default layers), 57.1% (CircuitKIT layers) and 55.5% (CircuitKIT, 4-bit aware), against 62.0% for the original. Before fine-tuning they scored 37.4%, 36.5% and 34.1% on those questions, so the CircuitKIT-layer models came back further. The same 25 steps on 64 unrelated MMLU questions left them at 33.4% to 35.5% (the original: 55.1%), so it takes the topic to bring it back, not just moving the weights. This suggests RMU hides the knowledge from prompts without erasing it, which is what Łucki et al. found on other models.

The pipeline: four Lexsi repos on a Cohere model

CuratorKIT builds the probe sets. Tiny Aya Global translated a fixed sample of WMDP-bio questions into Hindi, Arabic, Spanish and Swahili and reworded them in English. We kept a translation only when its back-translation scored chrF 35 or more against the source. CuratorKIT's schema, exact and MinHash dedup and cleaning gates then curated the sets, with strip_html off so nothing in the questions is silently deleted. Each set ships with CuratorKIT's manifest and provenance file.

CircuitKIT finds where the knowledge flows. EAP-IG attribution on WMDP-bio questions over all 36 layers of Tiny Aya scored each layer's MLP and attention. RMU edits the MLPs of three consecutive early layers, and SafeTune's default is 5 to 7. Of all three-layer windows in layers 1 to 12, layers 7 to 9 have the most MLP attribution (0.30, against 0.23 for 5 to 7), so we tested that window. Other layers score higher (0, 16, 18 and 35), but they are outside the early range RMU is used in. The attribution ran on only 64 short questions and its faithfulness check is weak, so we treated it as a lead and then tested it against controls at other windows. It moved the edit to a better place than the default. The controls show it was not the only better place.

SafeTune removes it. RMU pushes the model's internal activations on hazardous text toward noise at one layer, while a retain loss on ordinary text (wikitext) keeps everything else in place. It edits only three MLP matrices. We needed real tuning for Tiny Aya: SafeTune's defaults changed nothing, a retain weight of 100 with learning rates 2e-5 to 2e-4 worked. We ran 12 settings at the default layers, 9 at the CircuitKIT layers and 2 to 4 at each of three control windows, 30 runs in all, and compare runs at similar forgetting. The 4-bit-aware variant trains the same objective with the edited weights randomly passed through llama.cpp's Q4_0/Q4_1 and NF4 quantizers (straight-through). The file-trained variant used for the repair goes further: every decoder weight, in the model being edited and in the frozen reference, is its Q4_0 round trip during training, on half the steps or on all of them. SafeTune's NPO was also tried and was not selective on Tiny Aya: it either barely moved WMDP-bio or collapsed MMLU with it.

AuditKIT scores every file. Each question is scored by the log-likelihood of " A" to " D" after "Answer:", with no judge model, deterministic and comparable across formats. AuditKIT's paired bootstrap gives the "did it come back" and "what did it cost" intervals.

The formats are made the way people make them. We used the bf16 checkpoint, llama.cpp's own convert_hf_to_gguf.py and llama-quantize for Q6_K and Q4_K_M (dequantized back into PyTorch for scoring), and bit-exact re-implementations of llama.cpp's Q8_0/Q4_0 and of bitsandbytes NF4, tested against llama.cpp's reference code. For the deeper-layer models the Q4_0 files are real llama.cpp files too. All runs used free Kaggle T4s.

What we tried that didn't work

These are the steps that led to the recipe above. All runs are in the raw runs.

Metrics and why

What it means

Cost: self-hosting against closed APIs

For one million requests of 400 tokens in and 150 out:

Option Cost per 1M requests
Claude Haiku 5.5 or OpenAI gpt-6-luna API ($0.10/$0.50 per 1M tokens) $115
Gemini 3.5 Flash-Lite API ($0.30/$2.50) $495
OpenAI gpt-6.1-terra / Claude Sonnet 5.5 APIs $1,150 / $2,300
Tiny Aya on one T4 (149 output tokens/s measured, AWS $0.526/h) $147
Tiny Aya Q4_K_M on the user's own device (14 tokens/s measured on 4 CPU threads) $0 per request, offline

Prices were read on 2026-11-09. A served T4 costs roughly what the cheapest closed APIs cost. The reason to choose Tiny Aya is the last row, which is also where the safety has to sit in the weights.

Limits

Reproduce

git clone https://github.com/itssaharsh/relapse && cd relapse
pip install -r requirements.txt && python -m pytest -q tests   # round trips match llama.cpp; every number here is in the data
python scripts/build_results.py --fetch                     # rebuilds every number above from the raw runs

The Kaggle scripts that produced the runs, the unlearned models (safetensors and Q4_K_M GGUF), the per-question results, AuditKIT runs, CircuitKIT scores and CuratorKIT-gated probe sets are all public. The models are in the Relapse models collection and the data in the Relapse data collection. The WMDP bio forget corpus is gated by CAIS and is not redistributed.

Changelog