Email 1 of 5
Your inference bill is a design decision
Nobody chose the number on the invoice. That's the problem.
When an AI feature ships, the inference bill that follows feels like weather — something that happens to you. It isn't. Every dollar on that invoice traces back to a design decision somebody made without knowing they were making it.
Which model handles which request. How much context gets stuffed into every prompt. Whether identical calls get cached or recomputed. Whether retries back off or hammer. Each of these was decided — usually by default, usually by the first engineer to get the feature working.
The platform team inherits the sum of those defaults. By the time the bill is big enough to notice, the defaults are load-bearing.
The fix isn't a cheaper model. It's making the invisible decisions visible, so they can be made deliberately.
Tomorrow: the one routing question that cuts most bills by a third.