Signal
A safety score can look precise while leaving its meaning blurry.
A July 30 preprint tested that problem across four benchmarks for AI agents. The authors reran R-Judge, InjecAgent, AgentHarm, and AgentDojo with official implementations and author-provided scorers. Their audit covered up to 22 models, depending on the benchmark.
The result gives readers a practical rule: keep the coordinates attached to the score. Which behavior was tested? Which metric counted success? Which models were compared? What harness and tools were in the loop? Which outside result was used to check whether the score meant anything beyond the benchmark?
Remove those details and a narrow measurement can grow into a broad safety claim it never earned.
What changed
The paper starts by separating the four targets.
R-Judge tests whether a model can identify unsafe and benign interaction traces. InjecAgent measures resistance to prompt injection. AgentHarm checks compliance with harmful agentic tasks. AgentDojo also examines prompt injection, inside a tool-use environment.
Those behaviors can matter at the same time. They still answer different questions. A model that refuses harmful tasks may perform differently when asked to spot risk in a trace or ignore a malicious instruction hidden in tool output.
Then the audit looks at the ruler itself.
That metric problem becomes concrete in R-Judge. Its dataset labels 52.7 percent of items unsafe and uses F1 as the headline measure. An evaluator that calls every trace unsafe would receive an F1 of 0.690. It would make no real distinction between safe and unsafe traces, yet that score sits above five of the 21 evaluated models with valid outputs.
That happens because F1 rewards recall on the positive class and gives no credit for true negatives. Balanced accuracy tells a different story because it counts correct decisions on both classes.
Meanwhile, panel size moves the story. The reported correlation between R-Judge specificity and AgentHarm safety was -0.64 on a seven-model panel. On the larger common panel of 18 models, it became +0.02. The clean-looking trade-off disappeared.
AgentDojo had results for five models. The other benchmarks covered more, but their overlap remained uneven. A leaderboard can change when the field of runners changes.
Why it matters
So benchmark numbers travel farther than their footnotes.
A score moves from a paper into a model card, a procurement deck, a risk register, or a headline. Along the way, "resisted prompt injection in this harness" can quietly become "safe agent." The second claim carries much more weight.
And capability adds another wrinkle. In the authors' original panel, their capability composite correlated +0.60 with task success. More capable models tended to complete more of the selected retail tasks. Relationships with the safety criteria moved with the outcome and the panel.
The strongest held-out association connected AgentHarm safety with a three-template jailbreak measure after controlling for capability. Both tests examine harmful compliance, so the authors call this convergent validity. That supports a bounded claim about one behavior. It does not certify a model for deployment.
A useful safety claim needs a small evidence contract:
- Target behavior: harmful-task refusal, prompt-injection resistance, trace-risk awareness, or another named behavior.
- Benchmark and metric: including score direction, class prevalence, and a simple baseline.
- Evaluation configuration: model version, harness, tool representation, scorer, and execution policy.
- Comparison panel: which models were included, which were missing, and how much overlap existed.
- Outside check: the held-out criterion and why it is relevant.
- Uncertainty: confidence intervals, panel limits, proxy limits, and results that weakened under expansion.
That contract will not settle every safety question. It gives the next reviewer enough context to ask the right one.
What this does not prove
The study is a preprint. Its code and artifacts were not reproduced for this card.
It audits four benchmarks rather than the entire agent-evaluation field. Benchmark disagreement can reflect different target behaviors, which is expected, instead of a defective instrument. The held-out criteria are stand-ins for deployment and do not directly measure real-world harm.
The panels are small or uneven. Some estimates weaken or change as coverage expands. Null findings do not establish that no relationship exists, and the reported associations do not show that capability causes safe or unsafe behavior.
This card is research orientation. It is not a production-safety certification, legal or compliance conclusion, procurement verdict, or recommendation about a named model or provider.
What to watch
- Two-sided metrics: whether trace-judgment leaderboards report specificity and balanced accuracy beside F1.
- Simple baselines: whether every headline score is compared with constant or low-information policies.
- Panel refreshes: whether relationships survive as new model families enter the comparison.
- Exact configurations: whether reports preserve the model, harness, tool, scorer, and execution-policy combination.
- Held-out behavior: whether independent audits connect benchmark scores to outcomes that matter in deployment.
A leaderboard tells you who won the race it staged. Before carrying that result into the real world, check the track, the rules, the runners, and what winning was supposed to mean.