We run a small OpenAI-compatible API relay. To price anything sanely we had to answer a boring question for every model we carry: what does one request actually cost us?
Official price lists did not answer it. So we measured. Over the past two weeks we sent fixed probes to every model on the catalogue and read back the delta on the upstream billing endpoint. A few things came out of that which surprised us, and which apply to anyone building on LLM APIs - even if you never touch a reseller.
1. The sticker price is the least interesting number
Every provider quotes two numbers: price per million input tokens and per million output tokens. Your bill is decided by neither of them directly.
What decided our actual cost, in order:
Output length. Output tokens cost roughly 4-8x input tokens. A model advertised at $0.50 / $3.00 is not "a $0.50 model" - the moment answers get long, you are paying the second number.
Retries. A 5% failure rate on 100k requests is 5,000 calls yo
Discussion
Break the silence
Take the opportunity to kick things off.