I built a price table for 2,499 models so I could stop guessing what a run cost
I wanted a dollar figure on every AI run that lands in my system, computed once at ingest and never quietly rewritten afterwards. The naive version of that is a dictionary of rates in the source code. I wrote the naive version first. It holds up until the first price change, and then every historical run in the database starts claiming a cost it never had.
So the rates became a table with a date on it, and it took 2,499
priced models from the 2026-08-08 sync of LiteLLM's MIT-licensed
model_prices_and_context_window.json. The parts of that build worth writing
down turned out to be the refusals: what the sync throws away, and what it
declines to do at all.
Why a constant can't hold a price
Cost is computed at ingest and never recomputed. That is the rule the rest of this follows from, and it exists because I want a figure I showed a client in March to still say the same thing in June.
If rates live in code, then a deploy in April rewrites what March cost. Nobody
does that on purpose. It happens because a constant has no date attached and
therefore no way to say "this was the price then". So every computed cost gets
stamped with the version of the price table in force at the moment it was
computed, and the version is just the dataset's own fetch date:
litellm:2026-08-08. A run priced last month carries last month's stamp forever.
When a price moves, the new rows apply to new runs.
That stamp is the difference between a ledger and a dashboard. A dashboard shows you today's arithmetic over yesterday's events. A ledger says what was booked and when.
What the sync keeps
The upstream dataset carries a lot of fields. Mine keeps three per model, plus the model id:
| Field | Why it survives |
|---|---|
input_cost_per_token |
required; a row missing it is dropped |
output_cost_per_token |
required; a row missing it is dropped |
cache_read_input_token_cost |
optional; falls back to the input rate |
Everything else goes: context windows, capability flags, provider metadata, the
sample_spec entry that isn't a model. If an entry lacks either the input or
the output per-token price, the sync skips it entirely. A model that can be
half-priced is worse than a model that can't be priced, because half a price
produces a number rather than a gap, and a number gets believed.
The cache-read fallback is the one judgement call in that table. When a provider publishes no separate cache rate, cached tokens bill at the model's input rate, which overstates the cost for any provider that discounts caching without saying so. I'd rather my figure be too high than too low, and for Claude Code work the cached tokens are the bulk of the traffic, so this default is doing real work.
The floor under the sync
The sync refuses to run if it sees fewer than 500 priced models.
It's there for the case where the upstream URL returns something valid but wrong: a rate-limit page that parses as JSON, a partial fetch, a schema change that moves prices under a new key. The sync would then write a table with a handful of models in it, every other model in the system would stop resolving, and the ingest path would start producing runs with no price at all.
Refusing to run means somebody reads an error and goes and looks, which costs a morning. The alternative is a stretch of ledger you can't defend afterwards, because the runs are already booked and cost is never recomputed.
There is a second refusal in the same file, and I only added it after it bit me. The vendored snapshot ships with the code and re-runs on every deploy, while the weekly cron pulls the newer upstream dataset. Left alone, a deploy re-applies the snapshot over rows the cron installed hours earlier, and the table walks backwards in time. Now a dataset older than the row it would overwrite leaves that row alone, and the sync prints how many rows it skipped. The log line matters as much as the rule: the reason the reverting went unnoticed for a while was that nothing said out loud what it was doing.
Money is Decimal, and the JSON is float
A small thing that cost more time than it should have. The upstream file is
JSON, so every rate arrives as a float, and floats carry artifacts: one came in
as
2.9999900000000002e-06, which is 22 decimal places against a column that holds
12.
Each rate is converted through str exactly once at sync, then quantized to the
column's own precision before anything compares it to what is already stored.
Otherwise the in-memory Decimal never equals the stored one, every sync sees a
change that isn't there, and rows get rewritten on every deploy. The pointless
writes were the small problem. The real problem was that the table's whole job
is to record when a price moved, and it was reporting movement daily on prices
that had been static for months.
The row that said "I don't know"
Claude Code reports its model as claude-opus-5[1m], with the context window as
a bracketed suffix. No upstream LiteLLM key carries a bracket. I checked all
2,986 of them.
So the lookup missed, and here is the fork in the road that I think separates honest cost tooling from the other kind. The run booked unpriced and flagged, and no price was invented for it. The tempting alternative is to default a missing rate to zero, because zero is a number, the row renders, the total still adds up and nothing looks broken. It also makes your costs lower and your savings higher, and the error is invisible from the output.
The flag is what sent me looking. The fix was small: resolution tries the
literal model id first, then the bracket-stripped id, so claude-opus-5[1m]
finds claude-opus-5. An explicit long-context price still wins if one exists,
and so does a per-org override, so nobody's existing numbers moved under them.
Stripping a bracket is safe precisely because 0 of 2,986 upstream keys contain
one, which means the fallback can never shadow a real published model id.
The unpriced row is still in my ledger. Cost is computed at ingest and I don't rewrite history. A fossil of a bug is more useful to me than a clean screen.
What a per-token table still can't tell you
- Per-request charges, image and audio pricing, and anything billed by the call sit outside a per-token schema entirely.
- Negotiated enterprise rates and committed-use discounts are private, so the public table is a list price. Per-org overrides exist for this, and using them is a manual act by someone who knows the contract.
- Batch and off-peak discounts vary by provider and don't appear in a flat input/output/cache triplet.
- The table is only as current as its last sync, and the fetch date is on every row so you can see how stale it is.
- LiteLLM is maintained by people who don't work for me. It's the best public source I know of, and it can be wrong, and when it is wrong my numbers are wrong in the same direction.
- A price table is sourced, and being sourced is weaker than being audited. Nothing above proves any individual rate is correct. The check that settles it is your own provider invoice at the end of the month.
If you want the same thing for your own runs
A free workspace prices runs against this table and shows the version stamp on each one.
If you build your own, the piece worth copying is the refusal. Decide now what your pricing code does when it doesn't know a rate, and write that decision down where the next person will find it. The answer shouldn't be zero.