Course · Pi · Subagents

Chain agents in sequence, then fan a job out in parallel

One spawn call handles one URL. The import waiting behind it is not one job - call it what it is: 347 saved URLs, a declared toy count for this lesson (see the note below), none of them in stash (the course’s running example, a small FastAPI feed reader) yet, every one needing the same three questions from the module opener answered - does it have a feed, does the first sync land, do the rows come out clean.

This lesson takes the same primitive from the last lesson - spawn a child, pay the handoff tax: a report you didn’t watch get produced, standing in for work you didn’t watch happen - and runs it enough times that the shape of the control flow around it stops being obvious.

You’re not going to write that spawn call by hand 347 times - but before you reach for any shape of multi-agent work, ask the one question that actually decides which shape fits: if two pieces finished in a different order, would the answer change?

When the answer is yes: chain them

Some jobs are only expressible as a sequence. Before any URL gets synced, you want to know which ones can be - and that’s naturally three steps where each one needs what the last one found: audit which import URLs have discoverable feeds, first-sync exactly that list, verify the saved rows came out clean. Step two can’t run until step one hands it a list; step three can’t run until step two hands it a result.

Trace it on three URLs instead of an abstract 347, so you can check every step by hand:

https://notes.example.org/ (the one you spawned against in the last lesson)
https://slow-blog.example/post-089
https://old-forum.example/thread/201

Predict before you run the audit step. Guess: all three become feeds. The audit child comes back with two, not three - notes.example.org publishes its feed at https://notes.example.org/atom.xml, and slow-blog.example turns out to be a saved article whose site feed lives at https://slow-blog.example/feed.xml, so both approve with a feed URL different from the saved URL. old-forum.example/thread/201 has no feed anywhere - the site is read-only threads, no syndication at all.

The audit didn’t just save a dead sync; it caught a bad assumption about the import itself. An export of saved pages is not a list of feeds. Step two first-syncs only the two approved sites. Step three checks their rows against the real database. Three steps, one $INPUT handoff between each, and the chain never touches old-forum.example at all - because step one told step two not to.

A chain exists because step two’s prompt needs step one’s real answer - not because the job is naturally slower. Notice also what the chain deliberately does not do: fix discover(), the feed-discovery function that is currently a no-op. Every step works around it instead - the audit greps pages for feed links rather than trusting the function - because a code fix is judgment work and this module’s whole argument is that judgment work stays in your window while the mechanical parts fan out. The fix goes on your chore list; the import doesn’t wait for it.

Pi has no built-in “pipeline” primitive - no chaining flag, no --max-turns cap, no concurrency governor (checked against Pi’s CLI documentation, September 2026) - so you get there with the same spawn call as before, just looped sequentially instead of fired once, with each step’s prompt built from the previous step’s raw output. The YAML below is a design sketch for a driver you write; Pi does not load this file automatically:

steps:
  - prompt: "Audit the import URLs: which ones have discoverable feeds"
  - prompt: "Add and first-sync the feeds listed in $INPUT"
  - prompt: "Verify the rows saved in $INPUT against the goal in $ORIGINAL"

$INPUT is a plain string substitution - whatever the previous step’s child agent wrote as its final answer, dropped into the next step’s prompt verbatim. $ORIGINAL carries the very first task through the whole chain, so a step three levels deep still knows what it’s ultimately in service of.

A driver script walks the steps with a plain for...await loop - spawn, wait for agent_end, capture the output, build the next prompt, spawn again. Nothing about this is more sophisticated than the single spawn call from the last lesson; it’s that same call, three times, each one seeded by the last.

Field	Type	Description
`steps`	array	Ordered list of prompts to run sequentially
`steps[].prompt`	string	The prompt for this step. `$INPUT` = previous step’s output, `$ORIGINAL` = the initial input
`model`	string	Optional model override for the chain
`max_turns`	number	Optional cap enforced by your driver; Pi itself has no `--max-turns` flag

When the answer is no: fan it out

In the real driver these two shapes compose rather than compete: the chain’s second step is this fan-out - audit once, fan the approved list out, then verify - and the rest of this lesson is about that middle step. The first syncs themselves don’t have that shape at all. Whether notes.example.org lands before or after slow-blog.example changes nothing - each feed’s sync depends only on that feed, never on another one’s outcome. Running all of them sequentially, one after another in a chain, would be correct and slow for no reason. This is the other shape: independent slices, run at once.

Do the arithmetic on toy figures, so you can check it yourself. The audit approved 312 of the 347 import URLs (35 had no feed to find - the toy numbers divide evenly, so nothing here is approximate). Batched at 12 per child, that’s exactly 26 batches (312 / 12 = 26 - pick a batch size that divides your real import evenly and this stops being approximate).

Give each child roughly 15 seconds per URL - a toy figure, invented for this trace, not a measured Pi speed - to add the feed, run the sync, and check its own rows.

Run all 312 sequentially, one after another in your own window: 312 × 15s = 4,680 seconds. Seventy-eight minutes, sitting in the one window you also need for the feature you actually came here to think about.

Fan the 26 batches out five at a time - the concurrency cap this lesson settles on below - and each wave takes as long as its one slowest batch: 12 URLs × 15s = 180 seconds, three minutes, per wave. Twenty-six batches at five per wave is six waves (⌈26 / 5⌉ = 6).

Six waves × three minutes = eighteen minutes, wall-clock, for the same 312 URLs - about four times faster on this toy arithmetic, and every one of those seconds spent in windows you never had to sit inside.

Split the import into batches - by site, by save date, by whatever keeps each batch a reasonable size - and spawn one child per batch, all at once. Name the pattern before you build it: fan-out is running the same child-agent prompt over N inputs concurrently, batched to respect your rate limit, and collected with Promise.allSettled rather than Promise.all - deliberately, so one batch crashing or timing out doesn’t cost you the other batches that finished fine.

Partial failure in a fan-out is data you triage, not a crash that discards everything.

This is exactly the kind of glue worth having Pi write instead of typing it by hand. You already have the one primitive from the last lesson - the spawn("pi", [...]) child runner, with its LF-only JSONL parsing loop wired to agent_end.

Hand Pi a prompt roughly like this: “Take the child-runner function from the spawn lesson (the spawn-and-parse code you saved as your own driver file) and wrap it in a fan-out helper: split an array of URLs into batches of a given size, spawn one child per batch concurrently, and return each batch’s result - success or failure - without letting one crashed batch cancel the others. Use Promise.allSettled, not Promise.all. Cap concurrency so we’re not spawning more than a handful of pi processes at once.”

Read the generated .ts before you /reload it (Pi’s command that reloads extension code without restarting the session) - check that it’s actually built on the child runner you already have rather than reinventing spawn logic, that it batches instead of firing every URL as its own process, and that it really does use allSettled. See Spawn a child Pi and read its JSONL for the child-runner primitive this builds on, and the extension anatomy lesson for the shape any generated .ts file should take.

Before you toggle anything, predict what happens to the tokens when you flip from running the job inline to fanning it out. The honest answer has two parts: the work itself costs the same either way - the same files get read and the same edits get made wherever they happen - and the handoff adds a little on top, because each child’s summary lands in your window too. Fan-out changes where the tokens are spent, never how many the work needs; it does not reduce the total, and the summaries make it slightly larger. One bridge before you flip the switch: the toggle below runs a smaller, different job than this import - a three-part migration delegated to three children, not 312 URLs in twenty-six batches - because the shape is the point, not the count. Watch two things as you toggle: your own window’s share drops hard, and the children burn the same work in windows you never sit inside:

That’s the reconciliation: eighteen toy minutes instead of seventy-eight, and zero tokens saved anywhere - the children burned the same work either way, and your window kept only the summaries. Delegating doesn’t make the work cheaper - it makes it happen off to the side, in windows you never have to sit inside, while your own context stays free for the feature you actually came here to think about. Fan-out buys you wall-clock time and window space. It never buys you fewer tokens.

The one control knob that matters here

Concurrency isn’t free, and the wave arithmetic above shows exactly what it costs. Cap it at five concurrent children, the way this lesson’s trace does, and you pay six waves of three minutes each to clear the toy import. Push it to twenty-six - one child per batch, all at once - and every batch finishes in the same three minutes, but you’ve traded five simultaneous model calls for twenty-six, and you’ll hit your provider’s rate limit long before you hit anything Pi enforces itself.

Past that ceiling, children don’t run slower - they start failing outright, one 429 at a time, and the naked spawn has no backoff built in for that. Your driver either retries with a delay, or the batch is just gone.

The other risk narrowing concurrency doesn’t fix is a child that hangs: a bash command that never returns, a site that trickles bytes forever. Raw spawn hands you a process, not a supervisor, and nothing kills it for you. A stuck child holds up everything behind it - in a wave-by-wave driver, every wave after it; in a slot-style pool, one slot forever - because your driver is still waiting on an agent_end event that’s never coming.

Start narrow - a handful of batches, five like this lesson’s trace - and put a timeout in your own driver before you add more concurrency, not after.

The same instinct applies to which shape you reach for at all. A chain when a later step’s prompt has to be built from an earlier step’s real output; a fan-out when the work is genuinely embarrassingly parallel. Reach for fan-out on a chain-shaped job and you’re guessing at what a later step needs before an earlier step has told you. Reach for a chain on fan-out-shaped work and you’re waiting in line for no reason.

Both patterns are the exact same spawn call from the last lesson - the only thing that changed is the control flow wrapped around it.

The import’s processed - eighteen toy minutes instead of seventy-eight, the same SubagentFanout toggle you just drove showing the children burning the same work either way while your window kept only the summaries. But look at what you actually did: each child checked its own rows and reported clean, and you accepted every one of those self-reports at face value. Nothing outside the child checked whether they were true - and even the chain’s verify step is itself a report you accept on trust. That’s the handoff tax landing exactly where you’d expect - on the one thing isolation can never verify for itself: the report. Next: build a second agent whose entire job is to not believe the first one.