Commit 250c97c
feat(providers): dev-only hand-pricing reconciler vs models.dev (Tier-2 T6)
The generator's --check guards only the *generated* fallback catalog. The
small hand-maintained PRICING table Chimera actually bills had no drift guard
and is the one that silently goes stale. scripts/audit_model_pricing.py
reconciles PRICING against the public models.dev catalog and reports where a
billed rate diverges from the first-party figure.
Report-only by design: hand corrections always win over upstream, so it never
rewrites a price. Exits non-zero on drift (CI-able) but is intentionally NOT
wired into ci.yml. Default run is offline (committed model_catalog.py snapshot,
no network) and high-signal (first-party comparison only; reseller markups are
surfaced separately, --include-resellers opts in); --live fetches models.dev
via the generator's stdlib urllib path; --json for machine consumption.
Override convention: a new PRICING_OVERRIDES frozenset in cost.py marks the
prefixes whose divergence from upstream is deliberate (GLM / DeepSeek-SKU
placeholders, cross-endpoint billing nuances, local / open-weight $0 families).
The audit skips them; membership does NOT affect runtime resolution (the hand
table still always wins).
Caught a real in-repo drift on the first run: deepseek-chat ($0.27/$1.10) and
deepseek-reasoner ($0.55/$2.19) disagree with first-party models.dev
($0.14/$0.28).
Dev-only: zero new runtime dependency; nothing under chimera/ imports the
script. Tests reconcile over fixtures (no network) plus a network-free canary
on the committed snapshot. Guide: model-catalog.md (Auditing the hand table).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>1 parent a2c5de9 commit 250c97c
5 files changed
Lines changed: 771 additions & 0 deletions
File tree
- chimera/providers
- scripts
- site/src/content/docs/guides
- tests/providers
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
12 | 39 | | |
13 | 40 | | |
14 | 41 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
109 | 139 | | |
110 | 140 | | |
111 | 141 | | |
| |||
0 commit comments