ContextDrift
Watch an AI agent violate its own constraints in real time — the rules are visible in the context window, but attention weight decay means the agent just stops caring.
ContextDrift demonstrates attention weight decay — the failure mode where an agent's constraint compliance degrades as context fills, even though the constraint text never leaves the window. A 40-step simulation shows an agent refactoring authentication while three hard constraints decay from 94% attention weight to 3%. At Step 28 the agent modifies billing.py with the 'DO NOT touch /billing' rule physically visible in the context panel. Three escalating violations play out while every constraint remains in-window. Agent Failure Series #13.
The build log
Follow the work.
New essays and prototypes, in your feed reader. Email subscriptions are paused while I connect a working delivery service. No email is collected here.
Follow via RSS