Pretraining poisoning can introduce backdoors into an Agent's planning and tool-calling stages. Security teams need to go beyond reviewing prompts and track training data, model versions, and execution traces. The following checklist provides a practical way to investigate.
Pretraining Poisoning: Where Agent Backdoors Hide
The New Attack Surface Behind 63 Points
The paper Agentic backdoors via pretraining poisoning reached the Hacker News front page, earning 63 points and 14 comments. Paper link: https://arxiv.org/abs/2610.37741, with arXiv ID 2610.37741.
The work does more than demonstrate prompt injection once again. It moves the Agent backdoor threat from runtime inputs back to the model training stage: poisoning the pretraining data embeds malicious behavior into the model as its parameters are loaded.
The attack surface changes with it.
Traditional applications inspect user inputs, system prompts, and tool return values. Pretraining poisoning bypasses those runtime checks. The malicious pattern is already encoded in the weights and activates when an Agent receives a specific task, triggering through planning, tool selection, or parameter generation.
The available material does not specify the experimental model, attack success rate, test benchmark, or code. The paper's technical conclusions still need to be verified against the full text. Its 63 points and 14 comments show community interest, not a replicated result.
The research direction, however, is clear enough. Security teams should not only ask whether an Agent can be fooled by prompt injection. They also need to ask: Where did the downloaded weights come from? Can the training data be traced? Does the model behave abnormally under a narrow set of conditions?
Agent Backdoors Do More Than Change Answers
A conventional backdoored model often produces a specific answer, refuses to respond, or generates fixed text after encountering a trigger.
Agentic backdoors are more difficult to detect.
An Agent's execution path is usually a sequence of state transitions rather than a single block of text:
understand task → build plan → select tool → generate arguments → read result → continue planning → return answer
Malicious behavior can hide at any intermediate step.
For example, a normal workflow might retrieve information from a ticketing system and write the results to a CRM. A poisoned model might invoke one additional tool when a rare condition is met, select the wrong tool, or pass plausible-looking but altered arguments to a downstream service.
The final answer may show no obvious signs of compromise.
The task is complete. The logs show that every step used an allowed tool. The problem is that the model treated the wrong step as part of the correct plan from the start.
This is harder to detect than overtly unauthorized output. A security review that checks only the final response is like inspecting the cabin after a flight has landed, with no view of the route that was altered in flight.
Tool calling, planning, and execution in Agentic AI expand the potential impact of backdoors. Malicious behavior may not appear as a direct refusal or unauthorized text. It could be a tool call that selects the right tool but uses the wrong order or adds unnecessary arguments.
Why Poisoning Is Hard to Detect
The first layer of concealment in pretraining poisoning comes from potentially narrow trigger conditions.
An attacker could hide a condition inside an uncommon combination, such as:
- A specific customer name appearing together with a particular ticket type;
- A state that occurs only during overnight batch processing;
- A combination of standard business terminology and an unusual value;
- A condition that becomes true only after a multi-step task reaches its third stage.
Conventional evaluations usually cover common tasks and do not systematically enumerate these combinations.
The second layer is the appearance of normal behavior. An Agent plans, calls tools, and completes most tests successfully. The model may alter its actions only along an extremely rare path, making the behavior easy to dismiss as noise during a single end-to-end test.
The third layer is the supply chain.
Model developers typically receive a curated dataset rather than a complete training corpus. It may consist of cleaned JSONL files, web snapshots, synthetic data, or third-party data shards. A malicious sample might contain no conspicuous malicious string. It could simply differ in syntax, tone, or factual pattern.
In many cases, developers do not even know whether a particular shard participated in training. Confusing data versions, opaque cleaning rules, and overlap between training and evaluation sets make problems harder to locate.
An investigation cannot rely only on asking the model some unusual questions. It should begin with data provenance and verify sample hashes, collection times, cleaning versions, deduplication records, and the training manifest. Teams should also confirm that evaluation data did not appear in the pretraining corpus.
This is the most practical reminder in the article: model security cannot be achieved by adding a system prompt after downloading the weights. Prompts can constrain behavior, but they cannot prove that the weights are clean.
Do Not Test Only the Agent's Final Output
When evaluating an Agent, recording only the final answer is usually insufficient.
A minimal executable logging structure can preserve each tool call's name, arguments, result summary, and surrounding state:
from dataclasses import asdict, dataclass
from datetime import datetime, timezone
@dataclass
class ToolCall:
step: int
tool: str
arguments: dict
result_summary: str
timestamp: str
class AgentTrace:
def __init__(self, task_id: str):
self.task_id = task_id
self.calls: list[ToolCall] = []
def record(
self,
step: int,
tool: str,
arguments: dict,
result,
) -> None:
self.calls.append(
ToolCall(
step=step,
tool=tool,
arguments=arguments,
result_summary=str(result)[:200],
timestamp=datetime.now(timezone.utc).isoformat(),
)
)
def to_dict(self) -> dict:
return {
"task_id": self.task_id,
"tool_calls": [asdict(call) for call in self.calls],
}
This code does not detect backdoors, but it addresses a common prerequisite: without execution traces, anomalous paths cannot be compared.
Once traces are available, teams can add four categories of tests.
First: Trigger-Condition Tests
Break suspicious conditions into separate dimensions, including customers, dates, file types, task stages, permission roles, and argument boundaries. Test whether every component must be present rather than trying only one complete sentence.
Second: Anomalous-Path Tests
Run the same task several times and compare sequences of tool calls. If the model's output appears stable while tool selection or arguments vary slightly, determine whether this reflects ordinary model uncertainty or a distributional shift before and after a backdoor trigger.
Third: Least-Privilege Tests
Expose each tool with only the minimum capabilities required to complete the task. Separate reading files from writing them. Separate sending email, deleting resources, and accessing credentials.
Inducing an Agent to make one additional read-only call should not carry the same risk assessment as inducing it to send a file to an external address.
Fourth: Result-Validation Tests
Do not merely check whether the task was completed. Verify intermediate states as well. For example, confirm that CRM fields came directly from the ticket, that monetary amounts passed range validation, and that file paths remain within approved directories. Malicious arguments may still produce a successful response.
For Agents built with Claude, LangGraph, or other frameworks, place tests at the state-graph and tool-execution boundaries rather than only at the final-answer layer. Logs should capture an input and output summary for every step. Redact sensitive data from arguments.
Model Supply Chains Need Audit Evidence
Investigating pretraining poisoning ultimately depends on evidence retained during training. At minimum, preserve the following records:
- Training data provenance, licenses, and collection times;
- The version, hash, and processing history of each data shard;
- Cleaning, deduplication, synthesis, and annotation rules;
- Mappings between datasets and training configurations;
- The exact commit associated with the model weights;
- Backdoor test sets and regression-test results.
For open-weight models, retaining only the final weights is not enough. A model's risk profile can change after fine-tuning, quantization, merging, or Adapter loading. Audit records must evolve with each model version.
For closed-source APIs, vendors need stronger change-control and incident-disclosure mechanisms. Developers should at least retain request versions, tool configurations, permission policies, and call traces. Without them, reproducing an anomalous interaction after a model update becomes difficult.
Here is a minimal model manifest:
model: internal-agent-model
weights_commit: "sha256:example-model-commit"
base_model: "vendor/model-name@2026-01"
training_data_manifest: "data/manifest-v3.json"
data_cleaning_version: "cleaner-2.4.1"
tool_permissions: "policy/tools-minimal-v2.yaml"
evaluation_suite: "eval/backdoor-regression-v1"
last_verified_at: "2026-01-15T10:00:00Z"
These example values must not be used in production. A real manifest should point to verifiable commits, data versions, and test results while avoiding credentials or internal data paths in the repository.
More importantly, teams need regression tests for abnormal behavior. A trigger that does not fire today may fire after a model upgrade, tool rename, or data-cleaning change.
Security teams should maintain a dedicated regression suite for Agent models. It does not need to disclose the attack method, but it must include:
- Call boundaries for sensitive tools;
- Combination tests for rare conditions;
- Validation of abnormal arguments and paths;
- Privilege-escalation and indirect prompt-injection scenarios;
- Stability baselines for normal tasks.
Rerun the suite whenever the model, tools, or data version changes.
The implementation details of Agentic backdoors via pretraining poisoning still require a full reading of the arXiv paper. The available material contains no experimental model, attack success rate, benchmark, or code. The research direction should not be presented as a set of already replicated attack figures.
Editorial Perspective
This research matters because it extends the boundary of Agent security into the model supply chain, not merely because it shows that pretraining can be poisoned. Prompt injection occurs while a task is running, while training poisoning happens before the model is delivered. The latter is harder to trace and less amenable to a quick fix through an additional system prompt. The next generation of Agent security reviews will likely evaluate weight provenance, data versions, and execution traces alongside tool permissions.
Unlike previous prompt-injection cases, malicious behavior can hide in planning and argument generation while the final answer remains correct. Conventional success-rate metrics may also fail to expose the problem. However, the available material does not include the paper's experimental model, attack success rate, or benchmark. The attack surface and research direction can therefore be confirmed, but the real-world stability of the attack cannot yet be established.