Write the Acceptance Test Into Your AI Agent's Spec
A green build proves the code compiled. It says nothing about whether the work is correct. Write the acceptance test into the spec, then measure how well it survives an agent gaming its own grade.
24 posts tagged #rules.
A green build proves the code compiled. It says nothing about whether the work is correct. Write the acceptance test into the spec, then measure how well it survives an agent gaming its own grade.
A vibe-coded Stripe integration buries nine decisions in one handler. A teaching clause plus a plan-mode gate surfaces all nine - only three are worth your own trip to the docs.
Your daily review-minute budget is fixed, but a missed bug's dollar cost isn't. Spend the same minutes unevenly by blast radius and cut expected losses by roughly half, without reviewing more.
Agent-driven file discovery burns 12,000 tokens rediscovering three files you already knew. A parameterized priming command drops that to 6,200, and the saving holds from session one.
A single root rules file guarantees every convention is always present for every edit - which is exactly what floods a backend edit with frontend rules it will never use. The fix is directory-scoped files, and the tax has a number: 62.5%.
Raw frequency tells you which failures are real. It still ranks them wrong. Weight each promoted category by what one instance costs, and the rule that looked #2 by count drops to last.
Against a 1-in-12 hallucination rate on a checkable failure, a deterministic hook with one capped retry drives the shipped-wrong rate to exactly zero and moves the remaining risk onto one computable number: the halt tax.
The MCP spec hands a server three levers: resources for the application, tools for the model, prompts for the user. Skip that sort and your reads ride the tool list every turn, at a price you can compute.
A soft AGENTS.md rule is a per-commit probability, and probabilities compound: 95% compliance becomes a 92.31% chance of a lapse somewhere in a 50-commit sprint. The hook most teams write only cuts that to 22.37%. Here is the arithmetic, layer by layer, and the one layer that zeroes it.
Ask an agent to grade the work you just described and it grades the paragraph, not the code, because agreeing with a confident claim is the model's strongest reflex. Hand the same model the live diff and a rubric you author, and it finds the four misses your summary hid.
A confidently wrong answer is two bugs in one costume: a fact that never reached the context window, and a fact that arrived and got ignored. Log the window, score two binaries, and each bug routes to a different layer of the stack.
Every inline authorization check an agent writes is an independent guess at the same policy. The guesses disagree, quietly. Here is the arithmetic of how fast that compounds, and the one call that stops it.
The correction you keep re-typing into the chat is a workflow you haven't written down. Priced with a small invented schedule, it costs about 43 silently skipped build gates a quarter. Encoded as a slash command, it costs zero.
A 20-issue Sentry triage sweep run inline leaves 21,000 tokens of tool exhaust in the main thread. Dispatched to a subagent under a three-line return contract, it leaves 60. The ratio holds at every cadence.
An event can fire a headless agent run as reliably as a hook fires inside a session. But wiring the event only removes the human who had to remember to ask. The one who caught bad output has to be rebuilt as code too: rules for what a careful review would do, guards for what it would never let happen unattended.
A rules file is right the day you write it and silently wrong forever after, because nothing marks a reversed line. Auto-memory keeps both truths and cannot pick one. A trigger, a dated demotion, and a human gate end it.
Every failed agent run is a labeled example. Capture them with a hook, judge them, count them. The count tells you which failure to fix first. The shape of the failure tells you where the fix belongs, and in one two-week log, half the rows are not fixable in the rules file at all.
Your agent spends about 38 tokens working out where to make a change for every 1 token it spends making it. Then it pays that bill again on the next run. Here's the math, and the throwaway file that ends it.
Every ambiguous noun in a prompt forces the agent into a blind search over candidate files. A committed glossary turns that search into a lookup - and nets 4,410 tokens back on the very first prompt, no waiting required.
A requirements doc fixes the destination, a phased plan fixes the journey - and every technique on this blog for keeping a long build sharp only protects the journey. Here's what the destination costs when nobody checks it first.
A rule in your rules file only gets read if it survives to the top of a shrinking window. Move it into a hook that fires on the event instead, and expected compliance across one session's edits goes from 42.8% to 95.0%.
An unattended loop that reads git log -5 each pass has a five-pass memory. In a 12-item plan, the decision made at pass 3 ages out at pass 9, two passes before pass 11 needs it. One line in PLAN.md pointing at the commit hash recovers it, for zero extra commits.
Auto-compact looks like memory because the session keeps running. Each pass is a lossy summary, and the loss compounds: two passes leave a Phase 1 decision a 25% chance of still being known. A plan-file relay holds 100% at any phase count.
Copilot, Cursor, Claude Code, Codex, and OpenCode all converged on the same primitive - a rules file the agent reads on every session. Use it.