Course · Pi · Models & config

Route the cheap work to the cheap model

Here’s the backlog sitting in front of you on stash (the course’s running example, a small FastAPI feed reader) right now, and it’s deliberately lumpy:

Re-sync every entry saved during the broken-parse window. A few hundred rows with a missing published, an empty link, or a blank summary: fetch the feed, parse it with the fixed code, upsert the row. Mechanical. Every row is the same shape of work.
Harden fetch_feed against the feeds it still chokes on. One judgment call with real branches, and it lives in stash/feeds.py: what to do when a feed isn’t Atom-shaped at all, or dies halfway through as malformed XML, or mixes namespaces the parse doesn’t expect. Get the error paths wrong and the next bad feed quietly re-creates the backlog you’re sweeping.

Run both through the same pinned model at the same reasoning depth and you’re either overpaying on the sweep or underthinking the hardening. Pi actually gives you two separate levers to avoid that, and they’re worth naming separately because they compound.

Here’s the claim this lesson derives by hand, on the sweep, before it touches a single JSON file: pinned to the expensive corner (strongest model, highest thinking), four rows of fetch, parse, upsert cost about fifteen times what the same four rows cost at the corner matched to the work - for output that’s identical either way. Not fifteen times more thinking. Fifteen times more bill.

(This chapter’s dial is Pi’s own settings.json and models.json; the Foundations model-selection chapter has the spec-level view of how all six tools in this site’s scope handle the same choice.)

The two dials

Which model answers - set by defaultModel (and defaultProvider) in settings.json, or picked mid-session:

/model opens the model picker - and Ctrl+S there saves the row you’re on as the startup default.
Ctrl+L opens the same model selector from the keyboard.
Ctrl+P cycles to the next model among the ones enabled for cycling - the shortlist your enabledModels setting defines, which is also the enabling step: a model whose id is listed there is cycle-able (the config lesson walks that field), not every model every provider exposes.

How hard it thinks - Pi calls this the thinking level, and it’s a first-class setting, not a hidden knob: off | minimal | low | medium | high | xhigh | max, set globally via defaultThinkingLevel, cycled mid-session with Shift+Tab, and - if a provider’s defaults don’t fit - retuned per level with thinkingBudgets (a custom token allocation for each rung of the scale).

Neither dial cares what the other is set to. That’s the point: a cheap model at high thinking and a frontier model at off are both real, useful, and completely different tools. The re-sync sweep wants a cheap model at low or off - there’s nothing to reason about, only rows to process. The fetch_feed hardening wants your strongest model at high - a wrong call here is a rewrite, same as it would be anywhere else in this course.

Trace the sweep by hand

Take four rows off the re-extraction backlog - small enough to add up in your head, and standing in for the “few hundred” that make up the real chore. Give each row a toy spend unit of 1 at the cheapest matched corner: a light model, thinking off. Nothing about the row changes what it costs to process; four mechanical rows cost four units, at that corner, by definition.

Guess before the next paragraph: if you ran those same four rows through your strongest model at its highest thinking level instead, would you guess the bill doubles? Triples? Something bigger?

Now trace it. A capable model bills roughly five times what a light model does, per token. High thinking generates roughly three times the tokens a low-thinking pass does, on the same request - reasoning tokens the sweep has no use for, since there’s nothing in fetch, parse, upsert to reason about. Both multipliers are declared toys, chosen so the arithmetic stays hand-checkable.

Multiply them: 5 x 3 = 15. Four rows at the expensive corner cost 4 x 15 = 60 toy units, against 4 units at the matched corner. Fifteen times the spend. Zero extra output. That’s the number this lesson opened with, rederived from two multipliers you can multiply yourself.

Route the other task - the fetch_feed hardening - through the cheap corner and the trap flips shape instead of disappearing. That job has a real branch in it: whether a malformed feed should crash the sweep, skip the feed, or fetch it raw for later. A light model at low thinking doesn’t do a shallower version of that job; it’s more likely to get the branch wrong.

And a wrong call here isn’t a bad paragraph, it’s error handling you’re now shipping. Catch it in review and you pay for the light model’s attempt, then pay again for a capable model to redo it properly - which comes out more expensive than routing it correctly the first time. Verdict: wrong-cheap costs more than right-expensive.

The 5x and 3x multipliers above are the same ones the widget below uses, on a small set of task types rather than one toy sweep - deliberately coarse, chosen so the ratios are the point, not exact pricing. Set every task’s dials to the corner you’d have picked out of habit, then to the corner each task’s own verdict recommends, and watch which one racks up the redo tax:

A lumpy day: four kinds of task, and two dials on each - which model answers, and how much effort it spends thinking. Everything starts where most people leave it: pinned to the expensive corner. Re-dial each task and watch what the day costs.

presets

first runs495 unitsredo tax0 unitsvs the matched day1.9×

Everything ships - no failures, no redo tax - and the day still costs 1.9× what it should. That’s the quiet leak of a pinned dial: the mechanical work bills like hard work, five and three times over. Dial it down to the cheapest corner that ships it; the hard problems keep their budget.

Numbers are illustrative - the ratios are the point (capable ≈ 5× per token, high effort ≈ 3× the tokens). The redo tax counts an underpowered task’s failed attempts plus the escalation you’d run anyway, not the hour lost to confident wrong answers.

Actually adding the cheap model

enabledModels and defaultModel only help once a cheap option exists for Pi to route to - and if a cheap model from a provider you already use shows up in /model, that already counts. models.json is for adding one that is not listed, and it’s usually easier to have Pi do the editing than to hand-write the provider block yourself - it can read its own docs and figure out the shape a new provider needs. Say you want to add a low-cost coding model behind an OpenRouter-style API key, for the sweep work specifically:

> add a custom provider to models.json for a cheap coding model I can
  route the stash re-extraction backlog to, then tell me where to paste
  the API key

Pi identifies that this isn’t a login-subscription provider the way OpenAI or GitHub Copilot are - it needs an entry in ~/.pi/agent/models.json with its own baseUrl, api shape, and an apiKey field, plus (depending on how the key resolves) a matching entry in auth.json:

{
  "providers": {
    "cheap-coder": {
      "baseUrl": "https://openrouter.ai/api/v1",
      "api": "openai-completions",
      "apiKey": "$OPENROUTER_API_KEY",
      "models": [
        { "id": "some-cheap-coding-model", "name": "Cheap coder", "reasoning": true, "contextWindow": 128000 }
      ]
    }
  }
}

apiKey resolves three ways - a literal string, a $ENV_VAR interpolation, or a !command whose stdout it uses - so you’re never forced to put the key in the file at all. One convenience worth knowing while you’re in here: models.json reloads every time you open /model, so you can edit it mid-session without a restart.

Paste the key yourself; don’t ask Pi to read the file back and confirm it took. Agents that can read files can also print them, verbatim, to whatever terminal you’re sitting in - including one you might be sharing or recording. If a key is ever echoed back to you in plaintext, treat it as burned and rotate it, on principle, whether or not you think anyone else saw it.

Once the provider’s live, /model shows it in the picker, and Ctrl+P reaches it once you add its id to the enabledModels cycling list (the config lesson covers the field). Point the re-sync sweep at it, low thinking, and let it run through the backlog for a fraction of what the frontier model would have billed for the same mechanical rows. Keep the fetch_feed hardening on your strongest model at high - that one still earns every token.

That reconciles the opening claim: the toy trace put the ratio at 15x for the sweep specifically, because the sweep has no real use for the extra reasoning it would otherwise be billed for. A different task mix will land on a different multiple, but the mechanism is the same one you just multiplied by hand - model price times thinking-generated tokens - and it’s checkable on any task the same way: pick the toy size, pick the corner, multiply.

When routing doesn’t help: a one-off task you’ll type once and never repeat isn’t worth a /model switch - whatever the switch would save costs you more attention than it’s worth, and attention is the thing this whole exercise is supposed to protect.

Routing also doesn’t help when you can’t actually tell, before you start, whether a task is mechanical or has a branch hiding in it - misjudging that costs more either direction than not routing at all. Route the backlog chores you’ve done before and recognize on sight; leave anything genuinely new on your default model until you’ve seen its shape once.

One thing routing doesn’t give you: a ceiling. Nothing in settings.json or models.json caps total spend or warns you before a runaway loop bills the frontier model all night - Pi’s core doesn’t manage token spend for you. The community has filled that gap with third-party extensions rather than Pi shipping one; if a hard spend cap matters to you, check your provider’s own dashboard for a spend alert, because Pi’s config surface won’t stop you before the fact.

Routing between hosted models is one lever. There’s a more extreme version of “cheap” available too: a model that costs nothing per token because it’s running on the machine in front of you. See what that actually buys you - and where it runs out of room.