Cost-Aware Model Selection: When Local Beats Frontier

Every task I hand an agent used to go to the same place: the most capable model I could reach. It felt like the responsible default. It is also how you end up paying frontier prices and shipping every line of a task’s context off-box for work a local model would have handled identically, and, as it turned out when I actually extracted and tested my own routing logic, how a widget on this very site quietly disagreed with its own code comment for weeks without anyone noticing.

The actual problem

“Always use the best model” is not a policy. It’s the absence of one, wearing a policy’s clothes. It treats every task as if it needed the same ceiling, which means the common case, the one with a clear spec and no real ambiguity, pays the same cost and privacy tax as the genuinely hard task that needs it.

The fix isn’t “use local more.” It’s routing on the two things that predict whether local is sufficient: how sensitive the data is, and how complex the task genuinely is, not how complex it feels.

“Feels risky” and “is risky” are two different measurements. Only one of them should set your budget.

What a real routing policy looks like

engine-gateway-lab’s production policy runs two independent checks per engine, not one. A privacy gate (local-only, local-preferred, vendor-allowed) controls whether an engine is even permitted to see the task at all, keyed per engine to how much vendor exposure and network egress it involves. Underneath that, a set of task-type rules picks what confirmation, if any, a specific kind of task needs within whatever the privacy gate already allows. One real rule, verbatim:

- task_type: fast-capture
  privacy_min: local-only
  preferred: local-harness
  fallback: human-only
  reason: "Low blast radius. Local eval evidence adequate for summarisation and triage."
  evidence: "local-llm-lab routing-strategy.md - fast capture is likely local"

Every rule in the file carries that same shape: a privacy floor, a preferred engine, a fallback, and a stated reason with a citation, not just a tier number.

Sensitivity and complexity as two clean, independent axes is a simplification of that same idea, not a copy of the production policy itself. It’s the shape I built into this site’s own /agentic-sdlc routing widget, the interactive one that lets a reader pick sensitivity and complexity and see a recommendation, as an illustration a reader can turn a dial on. The underlying principle both share: don’t ask “which model is best for this,” ask “can the cheapest tier that’s allowed do this,” and only escalate when the answer comes back no.

The artefact

I extracted the widget’s simplified two-axis table into a small, standalone tool: github.com/dermdunc/model-router. Feed it a sensitivity level and a complexity level, get back a recommended tier, the reasoning, and the controls that tier requires.

$ node bin/model-router.mjs --sensitivity high --complexity high
Recommended tier: Local SLM or low-cost hosted model
Why: Retrieval, classification, summarisation, templating, and narrow transforms do not need a frontier model.
Required controls:
  - Prefer local or approved-only models
  - Data classification and PII controls
  - Retention rules
  - Legal and data review

(Sensitivity capped this below what complexity alone would justify - the whole point of asking both questions.)

Two bugs it found

Porting the widget’s logic into a standalone tool with real tests meant writing down, precisely, what the widget’s code actually did, not what its comment claimed it did.

Those turned out to be two different things. The comment said high sensitivity pulls the recommendation back toward local. The code never applied sensitivity to the tier at all, only to which controls were displayed. A second, related bug: the controls shown were computed from a separate complexity threshold that didn’t match the one the tier calculation actually used, so a low-sensitivity, medium-complexity task could recommend the mid-tier while only showing the local tier’s controls next to it.

Nobody had run this widget against its own stated comment until it got extracted and put under test. That’s the finding underneath the finding: “measure it, don’t assume it” applies one level up too. A routing rule that nobody checks against its own code is exactly as trustworthy as the frontier-by-default habit it exists to replace.

The decision: both bugs got fixed the same day, in the CLI and ported back into the live widget, not just in one place, since a decision aid that disagrees with the widget it was extracted from is worse than either alone. Recorded in this site’s own docs/decisions.md, 2026-07-22.

The reusable pattern

If you’re building a routing policy of your own, the shape is worth stealing even outside coding tasks: sensitivity and complexity as two genuinely independent inputs, not folded into one vague “how careful should I be” score. This CLI only proves the shape holds for coding work, it’s a 3x3 lookup table extracted from one widget, not evidence it generalises to, say, customer-support triage. But the two-axis idea, plus routing to the cheapest tier that’s allowed and sufficient, escalating on decline rather than by default, and logging which tier actually handled each request so the policy stays a checked claim, the way engine-gateway-lab’s own run ledger does, doesn’t depend on the domain being code.

And test the policy the same way you’d test anything else that makes a decision on your behalf. A routing rule with a comment nobody verified is not a safer default than no policy at all. It’s a policy-shaped guess.

A comment describing what the code does is a claim. Only the code, run, is evidence.

What next

Next time: the boring, load-bearing agents that make all of this possible, and why the interesting ones are never the flashy ones.