Plan Mode's Real Value: the Agent Interrogates You First

You type: “Build SignalDash - a signup dashboard with an email-capture gate for returning visitors.” Before it writes a line, the agent asks: “Should someone who already gave you their email hit the gate again, or does a returning visitor skip it?”

You hadn’t thought about that yet. Now you have to. That one question is the entire point of a planning step. The plan that comes after, headings and all, is downstream of it.

A plan produced from three sentences looks like alignment. A document exists, it has phases and a file list, and skimming it feels like agreement. But a document and a decision look identical on the page. The value was never in the document. It was in the interrogation that has to happen somewhere - before the build, where a wrong guess costs one sentence, or after it, where the identical wrong guess costs the whole build.

Open your last real feature prompt to an agent - the one that shipped, not a toy. Count how many of these it answered before the agent started writing code: what happens for a returning visitor versus a first-time one, what the empty state looks like before anyone has submitted anything, what happens when a submit fails, whether an admin or demo account gets a bypass, what “active” means under your billing plan, and what happens to a returning visitor who cleared their cookies.

If your prompt, or the agent’s own planning step, already surfaced answers to most of those before any code existed, you already have the interrogation this post is about. Close the tab.

If it didn’t - if you wrote three sentences, the agent produced something that looked like a plan, and you found out about the missing branch when a returning visitor complained - keep reading. The rest of this post puts a number on what that gap costs, and a procedure that closes it.

Put a number on the cost before naming the fix, because “ask more questions” is easy to nod along to and easy to skip under a deadline. So here is the claim up front, before any of the math: leave the ambiguity in and you should expect to attempt the same build about 11 times before one passes. Ask one clarifying question first and it’s about 1. Two numbers, one benchmark behind them, and the rest of this section shows exactly where they come from.

A March 2026 benchmark called ClarEval tested GPT-4o on coding tasks under two conditions: the same task with an ambiguity resolved by a clarifying question first, and the same task left ambiguous, guessed at cold. Clarified, GPT-4o’s Pass@1 score - the fraction of tasks it solves correctly on the first try, no retries counted - was 89.02%. Ambiguous, Pass@1 fell to 8.94%, on the identical underlying task, just without the one question that would have removed the guess.

Turn a pass rate into something you can actually plan around: how many tries before one succeeds. That’s the expected attempts to first success, and it’s just 1 divided by the pass rate - the standard way to turn a per-try success chance into how many tries you’d expect before one lands. Skip the clarifying question and, at an 8.94% first-try pass rate, expect to attempt the build about 11 times before one passes. Ask the question first, at 89.02%, and expect about 1 attempt to land. Those are the two numbers from the claim up front, now with their arithmetic visible, and the gap between them is entirely the interrogation.

(That 11-versus-1 treats each attempt as an independent retry - the agent throws the whole attempt away and starts fresh with no memory of why the last one failed. It’s an idealization, not a measured count of real rebuilds - more on that when the toy below runs the same math in tokens.)

One clarifying question moves the pass rate from 8.94% to 89.02%, and the expected rebuild count divides by 9.96.

Fix one running example and carry it through the rest of this post: SignalDash, a signup dashboard, and specifically its email-capture gate for returning visitors - the exact feature from the opening question. A prompt like “build SignalDash’s email gate” hides at least six real decisions inside it, each one a branch an agent has to guess at if nobody answers it out loud:

1. Returning vs. first-time visitor. Does someone who already gave an email skip the gate, or get it again? Skip it, and you dodge annoying a converted visitor. Gate it again, and you protect a reactivation funnel that depends on re-capturing attention. Real tradeoff, no default answer.
2. The empty state. What does the dashboard show before anyone has submitted anything?
3. The error state. What happens when a submit fails?
4. Admin bypass. Does a demo or admin account see the gate at all?
5. What “active” means. The billing model already defines an active user somewhere - does the gate use that definition, or invent its own?
6. The 30-day cookie case. A returning visitor clears cookies and comes back a month later. New visitor, or returning?

Six decisions, all sitting inside one sentence of a prompt, none of them answered by that sentence. An agent that starts writing code from “build the email gate” has to invent an answer to all six, silently, and ship whichever guess it landed on. Every guess is a coin flip you didn’t know you were making.

Six branches in one sentence of prompt is a normal number, and normal is exactly what makes it easy to skip.

Here’s the failure most people carrying a vague-prompt habit have already lived through, just with SignalDash’s numbers attached. You write the one sentence. The agent produces a plan - phases, a file list, maybe a sequence of pull requests. It looks like alignment. It’s the agent’s best guess across all six branches, formatted to look like a shared decision.

You skim it, it reads plausibly - the branches you didn’t think to check don’t announce themselves as missing - and you say go. The dashboard ships. Then a returning visitor who cleared cookies a month ago hits the gate a second time, or your reactivation funnel tanks because the gate started skipping anyone with a stored email, and you find out which of the six guesses was wrong by watching it fail in production.

The wrongness compounds the further downstream it surfaces - a directional point about software requirements that predates agents by decades and didn’t get less true when the one writing the code stopped being human. A misunderstanding caught in the interview costs one sentence, the correction. Caught after the build, it costs the whole build, and the toy below puts an exact token figure on that difference.

The document was always a guess in a nicer font; the only question is whether anyone checked the guess before it shipped.

The fix treats planning as an extraction task rather than a document-generation task: a repeatable procedure that walks every one of SignalDash’s six branches before it writes a line of spec.

This is two foundations primitives working together. A skill is an invokable procedure - saved instructions the agent loads on demand. A slash command is how you trigger it by name. Wrap the interview as a skill, bind it to /spec, and “be thorough this time” - which you will forget under a deadline - becomes a button, which you won’t.

# Skill: spec

You produce a build-ready spec by interrogating the author, not by guessing.

## Procedure

1. Ask the author for a long, messy problem description. Tell them
   to over-explain. You will do the structuring.

2. Explore the repository before asking anything you could answer
   yourself. Read the relevant modules, configs, existing patterns,
   and data models. Report what you found in 3-5 lines.

3. Interrogate. Walk every branch of the design tree. Resolve one
   decision at a time - never open a new branch while a prior one
   is still unresolved.

   RULES for the interview:
   - Ask exactly ONE question at a time. Wait for the answer.
   - With every question, propose a recommended answer and say why.
     The author corrects faster than they specify from scratch.
   - If a question can be answered by reading the codebase, READ IT
     instead of asking.
   - Cover at minimum: empty states, error states, returning vs.
     first-time users, permissions, and the one tradeoff you'd
     most regret getting wrong.
   - Do not stop until no branch has an open question.

4. Only now, write the spec to ./SPEC.md.

The six branches map onto that procedure directly: explore SignalDash’s repo first (does an is_returning_visitor check already exist somewhere?), then walk the six one at a time, then write. Nothing about the six branches required a human to remember to ask - the procedure asks by default, every time it runs.

What SPEC.md becomes once it exists is a separate problem this post doesn’t cover: splitting it into a destination and a phased build plan (see destination and journey), and keeping it alive across a long build’s context resets (see treat the plan file as external memory and the relay, not the window). Both of those assume the interrogation already happened and the spec’s content is already correct - this post is what has to happen before either one starts.

All the skill changes is the default: the six questions now run every time, deadline or no deadline.

The one-question-at-a-time rule inside that procedure is doing more work than it looks. Hand someone a list of all six branches at once and they answer in a batch, shallowly - skimming for the ones that look important and defaulting the rest. Ask one, with a proposed answer attached, and you get a real decision: they either accept the recommendation in one second, or they push back, and the pushback is where the actual judgment surfaces.

“Should a returning visitor with a stored email skip the gate? Recommend yes, since re-gating burns your reactivation funnel.” That’s branch one, proposed. The real answer might be “no - skip it for anyone who converted through the referral flow, but keep gating organic returns, because those numbers are still bad.” The model could not have generated that sentence on its own. It required someone who actually knows the referral flow’s numbers, and it only came out because a specific proposal was sitting there to correct.

Name the move: Guess Out Loud. The agent was already going to guess - that’s what a vague prompt makes it do by default, silently, and ship. Guess Out Loud doesn’t stop the guessing. It forces the guess into the open, one branch at a time, with a proposed answer attached, so a human corrects it instead of composing an answer from nothing.

A silent guess ships as a fact. A guess said out loud gets corrected before it costs anything.

Put exact numbers on SignalDash’s six branches, the way the spoiler promised. Two toy inputs, both invented for traceability and stated as such: interrogating one branch - question, proposed answer, correction - costs about 250 tokens, so all six cost 6 x 250 = 1,500 tokens. One full build-and-verify attempt - the agent writes the gate, you check it, it either matches what you meant or it doesn’t - costs about 18,000 tokens.

Compute the obvious way first, the way the spoiler was derived: expected attempts to first success is 1 divided by the pass rate. Ambiguous, at 8.94%: 1 / 0.0894 = 11.19 attempts. Clarified, at 89.02%: 1 / 0.8902 = 1.12 attempts. That’s the same 11-versus-1 from the spoiler, now carried to two decimal places.

This is an idealization worth naming exactly here, not softened later: it assumes each attempt is an independent retry, where the agent throws the whole build away and starts fresh, learning nothing from why the last guess was wrong. Real agents partially correct between attempts instead of restarting blind, so 11.19 is an upper bound on what asking first saves, not a measured rebuild count. The honesty section below says this again, because it’s load-bearing enough to say twice.

Now attach the toy’s currency, tokens, and the spoiler stops being a rate and becomes a real cost for shipping SignalDash’s gate once:

• Ambiguous path: 1 / 0.0894 x 18,000 tokens per attempt = 201,342 tokens, 11.19 attempts’ worth, and no interview ever happened - every one of those tokens paid for a full rebuild, guessing at all six branches again each time.
• Clarified path: 1,500 tokens of interrogation, plus 1 / 0.8902 x 18,000 tokens per attempt = 20,220 tokens of building, for 21,720 tokens total.
• Saved, on this one spec: 179,622 tokens.

Check the exactness stamp: run the numbers from the two toy inputs and the two Pass@1 figures with nothing else plugged in, and you land on 201,342 and 21,720 exactly - the same figures the spoiler’s “11 attempts against 1” promised, now counted in the currency an agent actually spends. The interrogation costs 1,500 tokens up front. Skipping it costs 201,342 tokens in re-rolled attempts - the guess was expensive all along.

One spec is the toy. Real work ships more than one spec, so replay the same arithmetic at 1, 2, 3, 5, and 10 specs interrogated - this is the master table the rest of this post keeps returning to:

Notice what does not move: the multiple. It sits at 9.27x on every row, from one spec to ten. That’s a different shape from a cache you build once and amortize across many uses, where the multiple grows as a fixed setup cost gets spread thinner. Here there’s no fixed cost to amortize - the interrogation is per spec, 1,500 tokens every time, scaling exactly as fast as the specs do. The saving compounds in absolute tokens (179,622 at one spec, 1,796,221 at ten) without the ratio ever improving, because both sides of it grow at the same rate.

Ten specs interrogated instead of guessed saves 1.8 million tokens at a flat 9.27x - the same multiple you’d get from doing it once, just paid out ten times over.

One refinement worth keeping: factor the interrogation out so it can run without forcing a document. Sometimes you don’t want SPEC.md yet - you want to think, with something poking holes in a half-formed idea one branch at a time.

Keep a standalone /grill command that runs the explore-then-interrogate steps and stops there. No file, just the six-branches-deep conversation. When the idea firms up, /spec runs the identical interview and writes it down.

This overlaps with plan mode - the built-in state where an agent investigates and proposes before touching anything. The two solve different problems, and it’s worth being precise about which. Approve the plan, not the diff uses plan mode to surface an irreversible action for a safety veto before it runs - the failure it guards against is a destructive command. Guess Out Loud uses plan mode’s read-only window to surface a requirement for a correctness check before code exists - the failure it guards against is building the wrong feature entirely, safely. Same primitive, two different jobs: plan mode is the guardrail that says don’t touch anything yet, and Guess Out Loud is what runs inside it.

Plan mode says wait. Guess Out Loud is the reason waiting produces something worth building.

Three honest limits, stated the way this register requires: out loud, not buried.

The 11-versus-1 gap is a ceiling on what asking saves. It comes from treating every attempt as an independent, unlearning retry, already flagged above, worth repeating here because it’s the number doing the most work in this post. A real agent that fails a build often keeps some of what it learned into the next try; it doesn’t restart from zero the way the geometric-distribution math assumes. That means 11.19 attempts, and the 201,342 tokens riding on it, is the most the interrogation could plausibly save, never an average observed across real rebuilds. Treat the 9.27x as an upper-bound shape rather than a guaranteed rate.

A one-file bugfix has no branches to walk. If there’s no real decision buried in the request - fix the typo, bump the dependency, rename the function - running six interrogation rounds over it is theater, the same way baking an acceptance test into a spec you can’t write yet is theater; see bake the acceptance test into the spec for the matching case on the test side. If there’s nothing to disagree about, there’s nothing to interrogate.

This isn’t the first place the idea showed up, and it’s worth naming who else builds it in. GitHub’s Spec Kit ships a seven-phase pipeline, Constitution, Specify, Clarify, Plan, Analyze, Tasks, Implement, with a dedicated /clarify step that hunts ambiguity before the plan gets written; it’s MIT-licensed and past 120,000 stars in 2026. Cursor goes further than a skill you have to add on: its built-in Plan Mode already asks clarifying questions and produces an editable plan before it touches a file, natively, for anyone using Cursor out of the box. Amazon’s Kiro leans hardest into the instinct - a requirements document written in EARS notation, then a system design, then a task list, all committed before a single line of implementation code, marketed explicitly around writing zero lines first. BMAD-METHOD scales the same idea past one person: twelve-plus specialist agents handing off files across an entire software development lifecycle, a team-scale version of the interview this post scopes to one person and one spec.

None of that makes Guess Out Loud redundant with any of them. What it adds is that the protocol is documented and portable - the propose-then-correct discipline travels to any tool with a skill mechanism, instead of living as one product’s undocumented internal heuristic for when to ask. Cursor already asks. This post is what to do about it everywhere else Cursor doesn’t reach - and a name for what Cursor is quietly already doing.

Put the result back on the master table: interrogating a spec instead of guessing at it costs a flat 9.27x less, whether you do it once or ten times, because the interrogation’s cost scales with the specs instead of amortizing against some fixed setup the way a cache does. At one spec that’s 179,622 tokens saved. At ten, 1,796,221. Neither number moves the ratio - it was 9.27x at row one and it’s still 9.27x at row ten.

That reconciles the opening claim exactly: skip the interview and expect about 11 attempts before SignalDash’s gate passes; ask first and expect about 1. The whole gap traces to six branches sitting in one sentence of a prompt, invisible until something asked about them one at a time, with a proposed answer to correct instead of a blank to fill.

What this post doesn’t answer is the cost the interrogation itself doesn’t erase: someone still sits through six questions per spec, and nothing here bounds how many branches a relentless interview should actually walk before it turns into its own time sink. A ten-branch feature and a two-branch one get the identical instruction, don’t stop until no branch has an open question, with no cap on either. Hierarchy beats the group chat put a hard turn cap on a runaway orchestrator for exactly this reason. Guess Out Loud doesn’t have one yet.

The interview that catches every wrong guess is only free until someone has to answer the twentieth question - and this post hands off exactly where that limit should sit.

About the numbers. The two Pass@1 figures, 89.02% clarified and 8.94% ambiguous, are quoted from ClarEval (arXiv 2603.00187), checked 2026-08-15; nothing about them is estimated. Every attempt-count and token figure that follows is arithmetic derived from those two figures plus two declared toy inputs (250 tokens per interrogated branch, 18,000 tokens per build-and-verify attempt), independently re-run against a standalone script before publishing. The expected-attempts math (1 / Pass@1) assumes independent, unlearning retries, a stated idealization rather than a measurement of any real agent’s actual rebuild count, and the honesty section above says so at the point it matters. Nothing in this post is a transcript, a session log, or an observed dollar figure; there wasn’t one to report.

For the per-tool mechanics, see Skills for encoding the interview as a reusable procedure, Slash commands for triggering it by name, and Plan mode for the read-only guardrail the interview runs inside.