Writing specs and plans an AI agent can execute
An agent does what the spec says, not what you meant. Scope, named files and an executable check turn a long unattended session into a reviewable diff.
"Add rate limiting to our public API." Hand that sentence to a coding agent and it will come back with something that works, is tested, and rests on a dozen decisions you never made. Is a client an IP address or an API key? Does the limiter fail open or closed when Redis is down? Is the limit per endpoint or global? Each question has a reasonable default, the defaults differ between engineers, and the agent picks one silently. The result is not broken, which is exactly why it is expensive to review: you have to reverse-engineer the assumptions from the diff.
This walkthrough takes that one request from a vague sentence to a merged feature, and shows what each stage has to produce for an agent to carry it out without you steering. The examples use Claude Code, but nothing depends on one tool. On my own project, the times it went wrong were almost always the times I skipped the planning step and gave a vague prompt, which produced confident, plausible, wrong code. With a good plan, the same assistant produces in an hour what used to take me a day.
What the agent has, and what it lacks#
A colleague fills gaps with shared history: the last incident, a remark in standup. An agent starts each session with the repository, your instructions and whatever it reads while working. Anything outside those does not exist for it.
Two properties follow. The context window is the scarce resource: Anthropic's best-practices guide is organised around the way a long session that mixes exploration, planning and implementation carries all of it into every later step. And an agent stops when the work looks done. Without something it can run, "looks done" is the only signal available, and you become the verification loop. A spec therefore has to do two jobs: remove the guesses, and define an end state the agent can test.
Not every change needs this. If you could describe the diff in one sentence, a rename or a log line, prompt directly. Plan mode in the session suits a change confined to one area. A written spec and plan are for work with open questions or several files, or work that will outlast one session, and that is the case from here on.
Let the agent interview you#
Writing a spec from a blank page is a slow way to discover what you forgot. The best-practices guide suggests the faster route: give a minimal prompt and ask Claude to interview you, then write the result to a file. Claude asks through its AskUserQuestion tool, which suits this because the questions come in focused groups. Start the session in plan mode, where the agent reads files and answers questions but changes nothing: claude --permission-mode plan, or press Shift+Tab until the status bar shows it.
I want to add rate limiting to our public API. Interview me about the hard
parts: what counts as a client, what happens at the limit, how limits are
stored, how we roll it out and how we turn it off. Ask one group of
questions at a time and skip anything you can read from the code. When we
are done, write the spec to docs/specs/rate-limiting.md.When the spec is written, start a new session to execute it. The new session begins with clean context and a document to read, instead of a long conversation in which the spec is one message among hundreds.
What goes in the spec#
The most useful specs are self-contained: they name the files and interfaces involved, state what is out of scope, and end with an end-to-end verification step.
# Rate limiting for the public API
## Goal
Protect the public API from a single client exhausting capacity, without
rejecting normal traffic. Success means: a client over its limit receives
429 with a Retry-After header; every other client is unaffected.
## Decisions already made
- A client is an API key, not an IP address.
- Limits live in Redis, keyed by `ratelimit:{apiKey}:{window}`.
- Limit: 100 requests per minute per key, fixed window.
- The limiter fails open: if Redis is unreachable, requests pass and an
error is logged.
- A flag `rate-limit-enforce` switches between log-only and enforcing.
## Files and interfaces
- New: `src/middleware/rate-limit.ts` exporting `rateLimit(options)`.
- Changed: `src/server.ts` registers it after authentication.
- Do not change: anything under `src/billing/`.
## Out of scope
- Per-endpoint limits, burst allowances, a dashboard, IP-based limits.
## Verification
1. `pnpm test src/middleware/rate-limit.test.ts` passes.
2. `pnpm test:e2e` passes.
3. A script sends 101 requests with one key in a minute: the 101st gets 429
and a request with another key still gets 200.Three sections carry most of the value. Decisions already made is where the agent would otherwise guess, and writing the answers down turns each guess into a constraint. Out of scope does the opposite job: agents are eager, and without a boundary a rate limiter acquires a dashboard and a configuration system. Verification is the part the agent can run, and item 3 is behaviour rather than "the tests pass". Tests written by the same agent that wrote the code can pass while the feature is wrong, so include at least one check that comes from outside the implementation. If you cannot write such a check, the goal is still fuzzy and the spec is not ready.
Two failures keep recurring for me, and both now have a line in my specs. Ask for a feature and you get the feature plus settings, abstractions and generalisations nobody asked for, so scope is stated explicitly. And the assistant optimises for code that works, not code that is cheap at scale: left alone it will reach for a query that reads a thousand records to answer a question about one. I audit every data-access pattern myself, and my design docs cover what the change must not do, how it can fail and what it costs at scale.
Decide what is a constraint (fail open, key by API key) and leave implementation detail to the agent. A spec that dictates every function name leaves nothing for the agent to do well and breaks on the first surprise.
Turn the spec into small tasks#
Ask for the plan in the same session or a new one, with the spec as the only input:
Read docs/specs/rate-limiting.md. Write docs/plans/rate-limiting.md: an
ordered list of tasks. Each task must change at most three files, state
which files, state the check that proves it is done, and be finishable
without reading the other tasks. Put the riskiest task first.Each constraint in that prompt earns its place. At most three files fits a task into a session without crowding the context. A named check per task means "done" is never a judgement. A task that stands alone can run in a fresh session, or go to a second agent, without a long briefing. And the riskiest task first means a wrong assumption fails early, while little work depends on it.
### Task 2: Fixed-window counter in Redis
Files: `src/middleware/rate-limit.ts`, `src/middleware/rate-limit.test.ts`
Do:
- Implement `rateLimit({ limit, windowSeconds })` using INCR and EXPIRE.
- Set the expiry only when INCR returns 1.
- On a Redis error, log and call next (fail open).
Done when: `pnpm test src/middleware/rate-limit.test.ts` passes, including
tests for the limit boundary, a new window, and Redis being unavailable.
Not in this task: registering the middleware (task 3).In plan mode, Ctrl+G opens the proposed plan in your editor so you can change it before the agent proceeds. Reading a plan for a wrong assumption takes minutes; finding the same assumption in a diff takes much longer.
Run one task per session behind a cheap gate#
Each task runs in a fresh session with a short prompt:
Read CLAUDE.md, docs/specs/rate-limiting.md and task 2 in
docs/plans/rate-limiting.md. Implement task 2 only. Run the check and show me
the output. Do not start task 3.Asking for the check output turns "done" into evidence you can read, and the explicit stop keeps the agent from carrying on into the next task with a context that is already half full.
The per-task check should be a script, not a model. Put typecheck, lint and the tests for the touched package behind one command, so the agent, a hook and you all run the same thing:
{
"scripts": {
"typecheck": "tsc --noEmit",
"lint": "eslint .",
"test": "vitest run",
"gate": "pnpm typecheck && pnpm lint && pnpm test"
}
}State in CLAUDE.md that a task is not complete until pnpm gate passes. When you cannot supervise, make that enforceable instead of requested. The best-practices guide describes a Stop hook that runs your check as a script and keeps the turn from ending until it passes, and a /goal condition, a completion condition that a separate model checks after every turn so the agent keeps going until it holds. A hook is deterministic where a prompt instruction is advice, a difference covered in Claude Code hooks, skills and memory. The gate is only as good as your tests; a thin suite means a thin gate and more for the review to find.
Review once per feature, in a fresh context#
It is tempting to put a review agent after every task. Resist it. A reviewer that sees one task cannot notice that task five calls the function from task two with its arguments the wrong way round, and the defects that survive to production live at those seams: a type widened in one task and a consumer elsewhere that still assumes the narrow one, a retry added twice around the same call, a renamed variable read under its old name, two migrations whose order matters. A first end-to-end test of the whole feature also tends to cause changes, which invalidates reviews of code that no longer exists. A subagent starts with an empty context and must rebuild its understanding every time, so each extra review is also an extra bill.
So: cheap deterministic gates after every task, and one model review over the whole feature, run once the gates are green and you have tested it. The guide's own example sets the shape: a reviewer that compares the changed files against the spec, checks that every requirement is implemented and the listed edge cases have tests, and reports only gaps that affect correctness. That last instruction matters, because a reviewer asked to find problems will find some even in sound work, and fixing all of them is how you end up with extra layers and tests for cases that cannot happen.
---
name: feature-reviewer
description: Reviews a finished feature branch against its spec. Use once, after all tasks are done and the gate passes. Not for single-task changes.
tools: Read, Grep, Glob, Bash
model: sonnet
---
You review a finished feature, not a single change. You are given the base
branch and the path to the spec.
1. Run git diff --stat against the base branch.
2. Read the spec and list its acceptance criteria.
3. Read the full diff, then the files around each change when the diff is not
enough.
4. Mark each criterion met, partly met or not met, with file and line.
5. Look for defects between files: mismatched types or arguments, duplicated
retries or validation, identifiers renamed in one place only, migrations
that depend on order, config written in one place and read in another.
6. Report only gaps that affect correctness or the stated requirements.
Skip style. If you find nothing blocking, say so; do not invent findings.
You cannot edit files. Report only.Pin the model. By default a subagent you define runs on the same model as the conversation that starts it, so an Opus session quietly spawns an Opus reviewer every time. Setting model: sonnet in the definition decouples the reviewer's cost from whatever the main session is using, and you can pin something stronger for a feature that touches money or authentication. Leaving out the edit tools is a guard rail rather than a sandbox, since Bash can still modify files; the last line of the prompt and your permission settings do the real work. Invoke it by name, once, at a moment you choose:
Use the feature-reviewer subagent. Base branch: main. Spec:
docs/specs/rate-limiting.md. The gate passes on the current commit.Findings go back to the implementing agent, and each fix is checked by the gate. Re-run the full reviewer only if the fixes were substantial; for a ten-line fix, read the diff yourself. Then read the whole diff as the final reviewer. The agent's review narrows what you have to look at; it does not replace you. My rule is that nothing merges unless I could defend it line by line in a code review, so diffs stay small: a change too big to review properly was too big to ask for. I also poke the feature by hand even when the tests pass, because tests written by the author of the code share its blind spots, and when something looks off I ask the assistant why it chose that approach, which tends to expose a hidden assumption.
When the code disagrees with the spec#
Sometimes the agent reads the code and finds that a decision does not work: the framework does not allow middleware at that point, or the key you chose is not available. Treat that as a result, not a failure. Stop, change the spec, then continue. If you let the agent route around the conflict, you end up with an implementation that no longer matches the document, and every later session reads a spec that is wrong.
Plans rot for the same reason. A plan whose first three tasks are done and whose fourth is now wrong is worse than none. Make updating it the last step of every task: mark the task done and note anything that changes the ones after it. And keep the standing instructions file short. For each line in CLAUDE.md, ask whether removing it would cause mistakes; if not, cut it, because a bloated file gets parts of itself ignored. Put the gate command in it. Mine holds the conventions for naming, error handling and logging, and the assistant follows them more reliably than I do at 11 p.m. The written docs also became the project's memory: when I return to a service after two months, they tell both of us what the constraints were, so the session picks up where the thinking stopped.
Exploration is the exception to all of this. While you do not yet know what you want, a loose conversation is a good way to find out, and a spec written too early just records your first guess. Write the spec once the exploration has told you what the decisions are, and keep the judgement for yourself: it narrows what the agent can get wrong, and the review is still yours.
Related articles
All articles →Most incident reviews end as documents nobody opens again. A one-page format, actions filed as ordinary issues and a thirty-day check make change likely.
9 min
Denormalising trades write complexity for read speed, and the bill arrives later. Four questions decide when a copy is worth it, and how to keep it honest.
11 min
Hook exit codes that do not block, rules that never load, skills that abort and memory that stays on one machine: the failure modes and the fix for each.
11 min