Circuit Breakers and Representation Rerouting Explained

Circuit breakers and representation rerouting explained in one sentence: instead of teaching a model to refuse harmful requests, you retrain its hidden states so that when it starts down a harmful path, those states are pushed somewhere that cannot support a fluent answer, and the output derails. Zou and colleagues proposed the method in 2024, and a later critique tested its limits. This post covers how it works, a toy calculation, what the paper measured and what the critic found.

If you want the neighbouring ideas as short, hands-on lessons, Learn AI Alignment Theory has courses on evaluations and red teaming, robustness and interpretability, and it sets out each side of the open debates with no verdict.

Why circuit breakers exist: the trouble with refusal training

Refusal training teaches a model to say no to harmful prompts. The paper's own framing is that this "induces refusal states" inside the model, but "harmful states remain accessible once these initial refusal states are bypassed." A prefilling attack shows the problem. The attacker starts the assistant's reply with "Sure, here's how to...", and the model finds it hard to reverse course once it has begun. Our post on AI jailbreaks covers more ways this happens.

The usual patch is adversarial training: find attacks, train against them. The authors say such methods "often fail to generalize to new attacks that were unseen during training" and can cost capability. The circuit breaker bet is different. If many different attacks all end in the same kind of internal state, you can target that state and ignore which attack got there.

What a circuit breaker is, and what it is not

The name is a metaphor. A household breaker cuts the current when it gets dangerous, whatever caused it. Here the current is the sequence of hidden states a language model builds as it writes. The paper describes remapping the representations that lead to harmful outputs "towards incoherent or refusal representations."

The word "circuit" is loose. It is not a circuit in the mechanistic interpretability sense, a traced subgraph of the network. The method comes from representation engineering, which reads and edits representations directly. It is related in spirit to activation steering and to the refusal direction, but it changes the model's weights through training rather than adding a vector at run time.

Circuit breakers and representation rerouting explained: the training recipe

Representation rerouting (RR) is the paper's main version of the method. It needs four pieces:

• A frozen copy of the original model, which gives the reference hidden states and never changes.
• A tuned copy, trained with small low-rank adapters (LoRRA) instead of editing every weight.
• A circuit breaker set of text that makes the model produce harmful content.
• A retain set of ordinary text, so everyday behaviour stays the same. The paper uses UltraChat conversations and XSTest, a set of requests that only look harmful.

One data detail matters for models that already refuse. The authors keep the harmful assistant replies in the circuit breaker set but remove the harmful user requests, so the model's refusal behaviour is left alone and the breaker only fires once a refusal has been bypassed.

The two loss terms, with a toy calculation

Training adds two losses. On circuit breaker examples, the rerouting loss is the cosine similarity between the tuned model's hidden state and the frozen model's hidden state, clipped at zero with a ReLU. On retain examples, the retain loss is the L2 distance between the two. The mix shifts over training: the paper's example schedule weights rerouting by alpha(1 - t/2T) and retention by alpha(t/2T), so the early steps break the harmful pathway and the later steps pull everything else back.

Here is a toy version. The numbers are invented for illustration, not taken from the paper, and real hidden states have thousands of dimensions.

• Harmful example. The frozen state is (1, 0). The tuned state starts at (0.6, 0.8). Cosine similarity is 0.6, so the rerouting loss is 0.6. After training the tuned state is (0, 1). Cosine similarity is 0 and the loss is 0. If it ended at (-1, 0), the cosine would be -1 and the ReLU would still give 0. The loss asks for "no longer like the original", not for one fixed target.
• Retain example. The frozen state is (3, 4) and the tuned state drifts to (3, 4.5). The L2 distance is 0.5, and training pushes it back toward 0.
• Schedule. With alpha = 1 and T = 100 steps, step 0 weights rerouting at 1.0 and retention at 0. Step 50 gives 0.75 and 0.25. Step 100 gives 0.5 and 0.5.

The result is a model whose harmful-content states point away from where they started, while its states on benign text stay close to the original. In the paper's Figure 1, the refusal-trained model under attack still starts listing materials, while the circuit-broken model writes a few tokens and then repeats end-of-sequence tokens.

What the paper measured

The authors tested Mistral-7B-Instruct-v2 and Llama-3-8B-Instruct against HarmBench, using attacks such as GCG, PAIR, AutoDAN, a multilingual attack, prefilling, input-embedding optimization and a representation-engineering attack. For Llama-3-8B-Instruct the average attack success rate fell from 38.1% to 3.8%, while MT-Bench moved from 8.05 to 8.00. The chart shows five of the attacks and the average.

Beyond text, the paper reports an 84% cut in compliance with harmful prompts for a multimodal model under an image attack, and cuts of 84% and 83% in harmful function calls for an agent setup (the second figure under forced function-calling). It also tested how far the effect travels: robustness on a category of harm depended largely on how close the training data was to it.

What a critic found: embedding attacks and the limits of one evaluation

Schwinn and Geisler, in a short technical report, re-ran the paper's embedding-space attack with three small changes: a different optimizer, a more meaningful starting string, and checking several generations during the attack. They report that attack success on the released circuit breaker models rose from 15.7% and 9.6% to 100%, with no extra tuning. They also say the weakness is not unique to circuit breakers, because the same attack reaches 100% on other open models, and that embedding-space attacks are "an unrealistically strong threat model" mostly relevant to open-weight models. They did not retest the paper's other attacks, but they say ongoing outside evaluations see vulnerabilities to manual jailbreaks, such as spacing out the toxic content.

Read both together. The first paper shows a large drop against a wide set of attacks. The second shows that a defence's numbers depend on how hard the evaluator tries. Neither says circuit breakers are useless or solved. For the broader habit of testing safeguards with a motivated attacker, see AI red teaming.

Where it fits among other hidden-state defences

Latent adversarial training perturbs hidden states during training to make a model robust to internal disturbance. Circuit breakers instead aim to remove the usable internal route to a defined set of harmful outputs. A linear probe only reads a state and leaves the model as it is. The paper itself says its method targets attacks on the model's ability to produce harmful content, tested on single-turn conversations. It would not protect a vision model used as a plain image classifier, and it does nothing about a model whose goals are wrong but whose outputs stay inside what the breaker set covers.

Frequently asked questions

What is a circuit breaker in AI safety?

It is a training method that changes a model's hidden states so that, when it starts to produce harmful content, the states are pushed away from the original and the output breaks down.

What is representation rerouting?

It is the main training recipe in the paper. It pushes hidden states on harmful examples away from the frozen model's states while keeping states on ordinary examples close to them.

Is it the same as refusal training?

No. Refusal training works at the output level, teaching the model to say no. Circuit breaking acts on internal representations, so it does not depend on how a prompt is worded.

Has it been broken?

One third-party report got 100% attack success against the released models using an embedding-space attack, which it calls an unrealistically strong threat model. The paper's results on other attacks were not retested there.

Get started

To build the background that makes papers like this easier to read, start with the short lessons and work up from the basics. You sign in with Google or an emailed code, and it is a web app. Start Learn AI Alignment Theory: short hands-on lessons from first principles to open research, with every side of every debate and the sources to read next.