Model-cost controls: routing, caching and context budgets

Model cost is a product constraint. My historical engineering work used task routing, prompt caching and context budgets to make that constraint visible in the system. This note describes those methods. The original quantitative headline has been retired because a reconciled measurement receipt is not available for publication.

01 · Route by task and risk

Classifying a short reply and reasoning through a billing edge case need different checks. A routing policy should consider criticality before it selects a model by task type. Tasks touching money or compliance need the appropriate review and validation path.

The following is an illustrative router from the original note. The precedence cases are tested locally; that verifies the example's branches, not a production cost result.

function pickModel(task) {
if (task.criticality === 'high') {
  return 'claude-sonnet-4.6';
}
if (task.type === 'classify' || task.type === 'extract') {
  return 'gpt-5.4-nano';
}
if (task.type === 'generate' || task.type === 'summarize') {
  return 'gpt-5.4-mini';
}
if (task.type === 'reason') {
  return 'claude-sonnet-4.6';
}
return 'gpt-5.4-mini'; // default
}

02 · Keep reusable context stable

Keep instructions and stable reference material separate from the changing request. This makes caching easier to reason about and lets the application track which context was actually needed. Cache eligibility and costs depend on the provider and model in use.

03 · Budget context explicitly

Loading every available document into every request adds cost and makes failure harder to diagnose. Retrieve relevant material, cap the context, and record what the request used. Evaluate the effect on task quality before changing the production path.

04 · Measure outcomes alongside spend

A lower model bill is useful only when the workflow still does its job. Use representative fixtures and human review for the decisions that matter. Track spend against accepted outputs, retries and escalation, with comparable cohorts and dates.

05 · Publish the method with its limits

Routing, caching and context controls are engineering mechanisms I can describe. A numerical outcome needs its own dated measurement receipt. This public note makes no percentage-saving or unchanged-quality claim.