Inference Economics

I’ve been thinking about this a lot lately.

From what I understand, the frontier labs are pricing inference aggressively. Whether that’s because it’s subsidized, strategically priced, or simply where the market is today almost doesn’t matter.

What does matter is that there are now open-weight models whose benchmarks aren’t far behind the frontier, but cost a fraction of the price to run.

I think this becomes one of the biggest engineering problems over the next few years.

OpenAI and Anthropic have raised and spent enormous amounts of capital. Over time, those businesses will need sustainable economics.

So how do you increase revenue? Increase the price. Create more expensive plans (already seeing a trend: $20 → $200 → $2k?)? Increase the rate per token?

The prices we’re at today are already becoming untenable for some companies. Their token cost is on pace with or outpacing their actual headcount costs. If this is already unsustainable, what happens when the subsidizing ends?

We also need to consider others in the arena. DeepSeek and Moonshot ship open-weight models that get you good enough intelligence for a fraction of what the frontier labs charge.

And because they’re open-weight, companies can run them on infrastructure they control instead of shipping proprietary data to a third-party API.

So enterprises aren’t going to give up their AI productivity gains, but they’re not going to keep paying frontier models for every workload. The existence of capable open-weight models broke that window open. Enterprises will need to take a hard look at their own token economics.

That starts with understanding how they’re actually charged. Let’s look at first-party rates.

$5 per million tokens input and $25 per million output.

Let’s say an agent call has 2,000 tokens of prompt input (tools, system prompt, context) and it outputs 400 tokens.

Two cents isn’t anything to cry about.

Until the product succeeds.

Multiply that by 250,000 calls a month and suddenly you have a $5k monthly inference bill.

Take those same inputs and outputs and stack them next to “good enough” intelligence?

At $1 input and $2.50 output (some open-weight models go even lower) you’re looking at $750. An 85% reduction!

With economics like that I believe it’s inevitable that every company that uses inference at scale will be looking to serve it as efficiently as possible, and that ultimately might not be with the frontier labs (although gpt-5.6-luna at $0.20/$1.20 short context is insane value. and that’s after an 80% cut at the end of July. the race to the bottom has begun)

Don’t get it twisted. The question isn’t:

“what is the cheapest model?”

It’s:

“Which parts of my product actually need frontier intelligence?”

Every one of those decisions changes the economics.

I don’t think the next generation of AI infrastructure wins by having the smartest model.

I think it wins by knowing when you don’t need the smartest model.

It feels inevitable that “token economics” becomes as normal a conversation as cloud spend became over the last decade.

I’m starting to dig into this.

If your company has a meaningful inference bill and you’re willing to share an anonymized usage export, I’d love to analyze it.

I’m trying to get good at understanding where AI dollars actually go, where they don’t need to go, and when self-hosting makes economic sense (or doesn’t).

Because eventually, every AI company runs into the same equation:

$1 of inference to 1,000,000 people is still $1,000,000.