The Unaligned Intermediary.

The dangerous capability is not that one model can say what another refuses to say. It is that unrestricted generation can be placed upstream of a more privileged system and used to search for language that the second system mistakes for authority.

A safe answer and a safe system are different things.

A refusal is an observable model behavior: given a request it associates with harm, the model declines, redirects, or limits its answer. This behavior matters, but it is only one layer. It does not by itself determine which files an agent can read, which recipients it can contact, which commands it can run, or which secrets it can place into an output.

When a language model is connected to tools, the security question changes. The system must distinguish the user’s authorized objective from instructions found in emails, webpages, documents, code comments, retrieved memories, and other model outputs. All of those sources use the same medium—language—even though they carry very different authority.

Removing a brake does not improve the map.

“Abliteration” is a community term for modifying model weights or activations to weaken learned refusal behavior. The underlying line of research showed that, across a set of open chat models, refusal could be strongly influenced by a comparatively narrow direction in the model’s internal representation. Intervening on that direction suppressed refusals while leaving several general capability scores close to their baselines.

The result is easy to misread. An abliterated model is not made more truthful, more precise, or more competent. It is made less likely to stop. It may answer harmful requests more readily, but it can still hallucinate, misunderstand tools, produce brittle plans, and express confidence without evidence. Safety tuning and capability are not a single switch.

That distinction is central to the threat model. The modified local model does not need to be the most capable system in the chain. It only needs to be capable enough to generate alternatives, remove obvious warning language, imitate familiar institutional forms, and continue after rejection.

Social engineering, aimed at a machine.

Human social engineering exploits trust cues: urgency, authority, familiarity, reciprocity, and procedural language. A model-targeted version does something similar. It surrounds an unauthorized objective with context that makes the request look like a routine continuation of the agent’s work.

A locally controlled model can automate this rhetorical labor. It can translate one rejected objective into many superficially different requests, divide an operation into benign-looking substeps, produce documents that resemble trusted artifacts, and adapt its framing after every refusal. None of this requires the local model to know the target’s hidden instructions. The target’s visible behavior supplies enough feedback to continue searching.

This is best understood as an adaptive input-generation problem, not as a duel between two personalities. The local model is an untrusted intermediary: a cheap system for proposing language to a second system that possesses better capabilities, richer context, or real-world permissions.

Two outcomes define the boundary.

Extraction and action often reinforce one another. Retrieved context can improve a later attempt; a tool call can move information to a location the attacker controls. The risk is highest when the same model can observe untrusted content, access sensitive context, decide what to do, and execute the decision without a separate control.

Prompt injection research has demonstrated this structural weakness for years: LLM-integrated applications blur the boundary between data and instructions. Agent benchmarks extend the finding to systems that operate tools over untrusted environments. A persuasive string is not executable code, but when an agent interprets that string and possesses an execution path, the effect can resemble code injection at the application layer.

Local models change persistence more than possibility.

Many individual techniques in this threat model do not require a local model. What local inference changes is the cost and observability of iteration. An operator can generate large numbers of candidates without per-request provider moderation, account enforcement, or remote logging. Model weights can be changed, prompts can be retained privately, and the entire loop can run beside other automation.

This does not make every attempt effective. Smaller local models may be less coherent, less knowledgeable, and worse at sustained planning than the system they target. Abliteration can also degrade behavior in ways that do not appear in broad capability benchmarks. The advantage is operational: abundant retries, local control, and a willingness to explore requests that a hosted assistant would decline.

Assume the model can be persuaded.

Training models to follow an instruction hierarchy and recognize injections is useful. Input classifiers, output monitors, and red teams raise the cost of attack. But a probabilistic model-layer defense cannot be the final authorization mechanism for a deterministic real-world action.

The stronger design treats successful persuasion as an expected failure mode. It limits what the model can see, separates planning from execution, checks each sensitive action against structured policy, and gives the user a clear decision at the moment their authority is required.

1. Separate data from authority

   Treat webpages, files, messages, tool output, and model-generated text as untrusted evidence—not as permission to change the task.

2. Enforce policy in code

   Bind every sensitive operation to structured authorization, scoped identities, and deterministic checks outside the language model.

3. Minimize reachable privilege

   Give the agent only the data, tools, network destinations, and write access required for the current task.

4. Interrupt consequential actions

   Require a fresh, intelligible confirmation for disclosure, external communication, destructive changes, purchases, or privilege escalation.

5. Constrain information flow

   Keep secrets out of general context, filter tool results, restrict egress, and prevent one untrusted source from naming its own destination.

6. Test adaptive sequences

   Evaluate multi-turn, best-of-many attacks and record whether defenses preserve legitimate task completion while stopping unauthorized outcomes.

These controls also reduce damage from ordinary model error. A system that cannot send secrets to an unapproved destination remains safer whether the bad instruction came from an adversary, a poisoned document, an abliterated model, or the agent’s own hallucination.

Measure outcomes under adaptation.

A useful evaluation should pair realistic user tasks with attacker-controlled content and let the attacker adapt over multiple attempts. The test should score both security and utility: a defense that blocks every tool call may stop the attack, but it has also stopped the product.

Testing should happen in an isolated environment with synthetic secrets, inert tools, controlled destinations, and explicit authorization. Publishing exact successful payloads can transfer capability faster than it transfers understanding; defensive reports should prioritize the violated boundary, the preconditions, and the control that failed.

Research behind the threat model.

1. Refusal in Language Models Is Mediated by a Single Direction

   Arditi et al., 2024. Mechanistic evidence that refusal behavior can be suppressed through directional ablation and related weight modification.

2. Not What You’ve Signed Up For

   Greshake et al., 2023. A threat model and practical demonstrations of indirect prompt injection in LLM-integrated applications.

3. AgentDojo

   Debenedetti et al., 2024. A dynamic environment for evaluating tool-using agents under prompt injection attacks and defenses.

4. The Instruction Hierarchy

   Wallace et al., 2024. Training language models to prioritize privileged instructions over lower-trust inputs.

5. Mitigating the Risk of Prompt Injections in Browser Use

   Anthropic, 2025. A layered defense account emphasizing that browser-agent prompt injection remains unsolved.