Course · Pi · Context
How Pi compacts a long session
One decision from the dedup work is already committed in stash/db.py (stash is the course’s running example, a small FastAPI feed reader); one debt from it still stands. The decision: save_article inserts with ON CONFLICT(link) DO NOTHING, under a unique index on articles.link. The debt is that neither piece can run yet. SQLite refuses to create a unique index while duplicate rows violate it, and an ON CONFLICT(link) clause with no matching index is an error before anything executes - so until stash.db is cleaned up, the fix is code on disk, not behavior in the service.
The cleanup runs at two speeds. The migration is seconds: delete the duplicates, keeping the first row per link, then create the index. The proof is the long part. To show the constraint holds under real use, you re-sync every stored feed - twice each - and after each pass run the duplicate-count query and watch it come back empty.
Pi drives the syncs, reads the responses, runs the checks, and every feed adds its fetch output and its row counts to the window. The shape of that run, illustrative - the URLs and counts are invented, the pattern is the job:
feed 1 https://example.com/feed.xml sync -> {"saved": 24} sync again -> {"saved": 24} duplicate links: nonefeed 2 https://notes.example.org/atom.xml sync -> {"saved": 19} sync again -> {"saved": 19} duplicate links: nonefeed 3 https://journal.example.net/feed sync -> {"saved": 31} sync again -> {"saved": 31} duplicate links: none...One property of the check first - why it can’t be the endpoint’s saved number: it counts articles fetched, so it reads the same whether the second sync inserted rows or not. The query is the evidence. {"saved": 24} twice with a flat row count is exactly what winning looks like here.
Before you watch the window fill with that, sit with two numbers Pi spends without asking, before a single feed gets fetched. 16,384 tokens - reserved for the model’s own reply. Off the table before it writes a word. 20,000 tokens - the slice of conversation Pi refuses to summarize away, no matter how full the window gets. The floor nothing touches.
Add them and, in a full window, at least 36,384 tokens are beyond summarizing: 16,384 reserved for the reply, plus the 20,000 newest tokens kept verbatim. On a round 128K-token window (131,072 tokens - a common tier size, not a measured figure for stash’s actual model, which we haven’t measured here), that’s roughly 28% the cleanup can never reclaim. The reserve is unavailable from the first turn; the kept slice only fills as the session grows, and it holds exactly the recent turns you’d want kept.
Partway into the proof, the window is dense with feed XML and check output, and - without you doing anything about it - Pi condenses the thread on its own. This is compaction, and unlike a black-box “hope it does the right thing,” Pi’s version is documented well enough to reason about.
When it fires
Section titled “When it fires”Compaction isn’t a vague “getting full” feeling - it’s a formula. Pi compacts once:
contextTokens > model.contextWindow - reserveTokensreserveTokens defaults to the 16,384 tokens from above, configurable in ~/.pi/agent/settings.json or .pi/settings.json. Because the formula is relative to model.contextWindow, the actual trigger point moves with whatever model you’re running; it isn’t a fixed number baked into Pi itself. During a multi-turn run, Pi checks the threshold after tools finish and their results are appended, before the next assistant response starts - remember that ordering; it earns its keep near the end of this lesson.
There are three reasons compaction can fire, and it’s worth telling them apart:
- Threshold - proactive, checked between assistant responses, after tool results land, as you approach the limit.
- Overflow - a turn actually exceeded the available space; Pi aborts that turn, compacts, then retries it with room to spare.
- Manual - you asked for it directly with
/compact [instructions], optionally steering what the summary should emphasize.
What survives, and what gets summarized
Section titled “What survives, and what gets summarized”Compaction doesn’t summarize everything - it protects the most recent slice of the conversation first. That’s the second number from the top: keepRecentTokens, defaulting to 20,000. Pi walks backward from the newest message, keeping whole messages and accumulating their token estimates until the 20,000-token budget is reached - the message that reaches the line is still kept; the walk stops there. Everything older than that cut point gets condensed.
That older portion is flattened into a plain-text form - lines prefixed [User]:, [Assistant]:, [Assistant tool calls]:, [Tool result]: (thinking blocks are left out of the serialization as of 1.2, since replayed thinking may not be in the live context and could make the summary request larger than the context it summarizes) - deliberately not natural dialogue. The reason is specific: a summarizer model handed something that reads like a real conversation tends to keep talking instead of describing it. Flattening it into labeled, clipped lines keeps the summarizer in “describe this” mode instead of “continue this” mode.
Your per-feed output lands in that serialization as [Tool result]: lines, truncated to 2,000 characters with an explicit truncation marker.
What comes back is a structured summary, not a paragraph - fixed sections every time: ## Goal, ## Constraints & Preferences, ## Progress (broken into Done / In Progress / Blocked), ## Key Decisions, ## Next Steps, ## Critical Context. Separately, which files got read and which got modified is tracked in its own block, alphabetically sorted - so even after several rounds of compaction, Pi still knows it already read stash/db.py and stash/feeds.py and doesn’t re-read them out of amnesia. What survives is the record that they were read, not their contents - re-reading one on purpose, when current contents are needed, is normal and still spends the tokens.
On this job the Progress section does real work: feeds that re-synced clean are Done items, and the feed currently mid-check is In Progress.
That last point matters on a second compaction. Pi doesn’t re-summarize from scratch and risk losing what the first summary captured - it runs an update pass over the existing summary, with explicit instructions to preserve what’s there and to refresh both the Progress and the Next Steps sections based on what was accomplished. Your cleanup could get compacted twice in one long session without the Goal or Key Decisions sections drifting - the two sections that should move, move; the rest holds.
Trace it by hand first
Section titled “Trace it by hand first”Here’s a small session, invented for this trace and not a real stash run: eight turns, each with a made-up token cost, small enough to add up yourself. Predict before you read the table: which turns survive compaction untouched, and which get flattened into a few lines of summary?
Claim to check: turns 1 through 3 get summarized; turns 4 through 8 survive verbatim.
The toy session runs on a 40,000-token window - also declared, not real. Reserve 16,384 tokens and the trigger line sits at 23,616. Total cost across all eight turns is 30,000, which crosses that line, so compaction is due. Now walk backward from the newest turn, accumulating toward the real 20,000-token keepRecentTokens budget:
| Turn | Tokens (toy) | Running total from newest | Kept or summarized |
|---|---|---|---|
| 8 (newest) | 4,500 | 4,500 | kept |
| 7 | 3,500 | 8,000 | kept |
| 6 | 6,000 | 14,000 | kept |
| 5 | 3,000 | 17,000 | kept |
| 4 | 4,000 | 21,000 | kept - the turn that tips the budget is still the last one in |
| 3 | 2,500 | - | summarized |
| 2 | 3,500 | - | summarized |
| 1 (oldest) | 3,000 | - | summarized |
Turns 4 through 8 are kept the moment turn 4 is counted - the running total now reads 21,000, and that is how the rule works: the message that reaches the budget stays, so the kept slice can end slightly over 20,000. Turns 1 through 3 fall on the far side of the cut and get flattened into the six-section summary above. The claim holds, exactly as stated - not because compaction is smart about which three turns mattered least, but because of where they happened to sit relative to the end.
Watch the same shape play out at a bigger, moving scale below - a long session filling up, the fixed overhead that never goes away, and what gets cut versus kept once the threshold hits. Before you click through: predict which category of content survives untouched and which gets cut, using the rule you just traced by hand - fixed overhead (system prompt, tool schemas, the rules file) never counts against keepRecentTokens because it isn’t conversation; the most recent turns do; everything older than the cut point doesn’t.
Confirm it: step to the /compact segment and check what lands in “Structured summary” against what’s gone. The window size (200k) and the compact line (around 80% full) drawn here are picked for a readable bar, not Pi’s real formula - the table above used the real reserveTokens and keepRecentTokens; this one trades exactness for a shape you can watch move in real time. One more stand-in to know about: the simulator is tool-generic. Its rules segment (whose label names other tools’ file conventions) stands for whatever rules file a tool loads - on Pi, that story is the next lesson’s subject - and its summary block lists fewer sections than Pi’s six. Same mechanism, two different scales.
If you’d rather shape the summary yourself than trust the default cut, you can:
/compact focus on which feeds re-synced clean and their duplicate counts, drop the raw feed XMLWhat compaction doesn’t solve
Section titled “What compaction doesn’t solve”Be honest about what a “structured summary” actually is: a model’s best guess at what mattered, sorted into six fixed sections. A best guess can be wrong. If the real signal in your cleanup was one feed whose second sync left a row count you didn’t expect - a detail sitting in tool output older than the cut point, fitting neatly under no section heading - a summarizer optimizing for those six sections can drop exactly that detail, with no error message telling you it happened.
Compaction is lossy by design, not by accident - the simulator’s own summary block flags this on its own. That’s not a reason to avoid it; every alternative (letting the window overflow, or retyping your own notes from memory) loses more. It’s a reason to write down anything you can’t afford to lose - a per-feed verdict, a diff, a test result - somewhere that isn’t the thread, before you let a long stretch get summarized.
This used to have a sharper edge, worth knowing as history rather than a live hazard: auto-compaction could fire right after an assistant message that itself contained tool calls, aborting before those tool results came back - the results existed, but the threshold check ran too early to see them. It was flagged in Pi’s own issue tracker twice (8464 is the earlier-numbered of the pair), and fixed in August 2026. Current behavior is what this chapter described above: the check runs after tools finish and their results are appended, before the next assistant response starts.
The full, uncompacted history - every turn, every tool call, before any summarization touched it - still stays on disk in the session’s JSONL file, permanently. Compaction only changes what gets sent to the model on the next turn; /tree (Pi’s session-tree navigator) still gets you the rest of it, three compactions later, if you need to check exactly what one feed’s row count actually said.
The gap compaction can’t close
Section titled “The gap compaction can’t close”Compaction is genuinely good at what it does - goal, decisions, file list, recent turns, all protected on purpose. It does not stop you from re-explaining, in every fresh session or every fork (a session branched from an earlier point), the things about stash that never change: httpx and never requests or urllib, tests run with pytest, pydantic models stay in stash/models.py, feed dates normalized on the way in.
Compaction manages a session’s transient memory. The durable rules - true on session 1, still true on session 40 - need a different home, one that’s loaded before compaction, before forking, before any of it. And here’s the twist that hands off to the next lesson: those rules already exist, written down in a SYSTEM.md at the repo root, left there by the project’s original author - at a path Pi never reads.
One closing account, in the terms this chapter has been keeping: 16,384 and 20,000 tokens - the reserve keeps room for the reply from the first turn; the kept slice protects the recent turns as the session grows (and when even that room runs out mid-turn, the overflow path above is the backstop, not these numbers). Pi decided both numbers; you only get a say if you go into settings.json and change them. Next: write the durable rules down once, in the file Pi reads before Pi decides anything.