A bad shortcut can outlive the job that taught it.
A new preprint describes how this can happen inside a self-improving AI agent. The agent completes a task, converts the successful path into a reusable skill, and carries that skill into later work. If the path was unsafe, the system may preserve the hazard as operating procedure even after the original prompt and session are gone.
The authors of Practice Makes Unsafe call this skill misevolution. Their benchmark tracks the full sequence from a task episode to skill writing, later retrieval, and fresh-session behavior. That matters because a final answer can look correct while the process that produced it leaves behind a dangerous lesson.
Across 25 agent-method configurations, each covering 525 tasks in 25 episodes, all 21 configurations that evolved skills authored unsafe artifacts. Fifteen later produced harm in a fresh session. In that carryover test, the conversation, filesystem, and native memory were reset. The agent-authored skill file was the durable state crossing the boundary.
The gap between 21 unsafe artifacts and 15 harmful carryovers is important. A flawed procedure does not cause damage every time it exists. Later harm also depends on whether the procedure is retrieved and how it is used. That makes this a lifecycle problem rather than a simple bad-file problem.
The paper also tested repeated exposure. Three malicious tasks raised the carryover attack success rate from 16.0 percent to 35.3 percent. The finding is bounded to the authors’ benchmark, but the direction is worth taking seriously: a small amount of compromised experience may change what an adaptive system learns to repeat.
There is a related risk at the other end of the lifecycle. Another August preprint, Convergent Detour Hijacking, studied an attacker-controlled third-party skill that drew otherwise valid tasks into unnecessary work while preserving completion. On one tested model, the coordinator skill was selected in 80.02 percent of tasks. Among affected runs that still completed, token use rose 66.91 percent and execution time rose 92.45 percent.
Together, the studies suggest two routes into the same durable layer. A risky procedure may arrive from an outside publisher, or an agent may write one after learning from compromised experience. Reviewing only installed skills leaves the agent-authored half of the lifecycle largely untouched.
Verification bottleneck
Outcome checks are too late to explain what became reusable procedure.
An operator needs a record linking each skill change to the task that produced it, the evidence available at the time, and the reason the agent judged the procedure worth preserving. Retrieval needs its own record: which later task called the skill, what matched, and what the skill changed in the route.
The Practice Makes Unsafe authors tested a mitigation called SafeEvolve at the writing and reuse stages. In their benchmark, it reduced unsafe retrieval by 26.7 percentage points and fresh-session harm by 17.3 points, while mean benign utility changed by 0.4 points. Those are promising benchmark results, not production validation. The study is an August 2026 arXiv preprint using synthetic tasks, selected agent frameworks, one shared model backbone, and its own judges and definitions. This review did not run the authors’ code or independently reproduce the experiments.
The bottleneck is therefore practical and evidentiary: can an organization show what the agent learned, where that lesson came from, when it was reused, and whether anyone had authority to approve the change?
Opportunities
Teams can begin with a skill-change ledger. Every durable update should point back to its originating task and preserve the relevant trace, review status, scope, and rollback path. Skills learned from adversarial, ambiguous, or unusually privileged episodes can be quarantined from automatic reuse.
Retrieval policy deserves equal attention. A stored procedure can remain available for inspection without being eligible for every task. Narrow permissions, expiry rules, and fresh-session replay tests would make reuse a governed decision rather than an invisible side effect of past success.
There is also room for tools that compare a skill before and after adaptation, flag new permissions or procedural detours, and test whether the updated version changes unrelated work. The useful product would make the learning residue visible to a reviewer without requiring that person to reconstruct an entire agent session.
Adaptive systems will keep turning experience into procedure because the efficiency gains are real. The governance question is which experiences deserve to become rules that travel.
Sources
- Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents
- Convergent Detour Hijacking: Task-Preserving Resource Amplification in Skill-Based LLM Agents