Is AI getting cheaper or more expensive? I’m not sure that’s the right question. Models get cheaper and new ones replace old ones, but we aren’t buying the same thing anymore. Token usage changes as reasoning and prompts change, and most importantly, we change.

A few years ago, we’d ask AI to explain a function. Then we asked it to write the function, followed by the feature. Today, we can give a coding agent an issue and ask it to explore the repository, understand the problem, implement a solution, run the tests, fix what it broke, and open a pull request. As our AI literacy improves, so do our expectations.

That makes comparing the cost of AI over time surprisingly difficult. Even if inference gets cheaper, we might spend more of it because we keep finding more ambitious things to do. More capability keeps changing the baseline against which we measure the price.

We spend the efficiency gains

The phone in your pocket has vastly more computing power than the computers that helped put people on the Moon. We didn’t pocket all those improvements and continue running increasingly efficient versions of the same software. We spent them. Applications became richer while layers of abstraction accumulated.

Developers traded efficiency for productivity, and things that once fit on a floppy disk can now require hundreds of megabytes. More available compute didn’t necessarily give us cheaper computing. Instead, it changed what we considered normal. And AI could follow the same path.

Today’s expensive frontier capability becomes tomorrow’s commodity capability, and eventually some of it might run locally on hardware we already own. But by then, our expectations will have moved too. The local model will do what we wanted two years ago, while the frontier model does something we only recently realized was possible. So we keep following the frontier.

Good is good enough

We tend to talk about AI literacy as if it were one skill, but there are at least two. The first is knowing what AI can do. You learn that instead of asking for a function, you can ask for an outcome. Instead of giving an LLM a document, you can let an agent search a hundred of them. Instead of asking one question, you can ask it to research a subject and challenge its findings before producing a report.

The second is knowing what good means to you. It’s easy to ask an LLM to write a better strategy, but articulating what makes one strategy better takes real work. You can ask a coding agent to implement a feature just as easily. Before you can evaluate the result, though, you need to define the architectural tradeoffs that matter and which shortcuts you don’t want it to take.

Our ability to consume intelligence might be developing faster than our ability to evaluate it. But being able to tell when good is good enough is exactly what allows us to spend less! The obvious answer to rising AI costs is to use a small model for simple work and a larger one when the task demands it. Run common tasks locally, then save expensive frontier inference for the few problems that genuinely require it. Beautiful. Now decide which is which.

How do you know that the model costing one tenth as much is good enough for your work? For one, public benchmarks tell us something about the relative capabilities of models, but your work isn’t a benchmark. Increasingly, the model isn’t even the whole system. A coding agent’s performance depends on the model and its harness, along with the instructions and tools available to it. And then there is the task you give it.

Then there’s you: your requirements, preferences, tolerance for particular failures, and definition of good. Two people can both be doing “office work” while expecting entirely different things from AI. This isn’t like checking whether your PC can run a game. Millions of people run the same game, so someone can benchmark a particular configuration and tell you how many frames per second you’ll get. Nobody can benchmark your work quite as easily.

Convenience favors the frontier

We’ve solved a similar problem with computers in a wonderfully unsophisticated way. Most people don’t establish evaluation criteria for buying a laptop. They don’t benchmark their workload against seven processors or measure memory pressure for a month. They certainly don’t calculate the value of three seconds saved during a particular operation. They buy a current Mac or a current PC within their budget.

There’s a good chance a five-year-old machine would do everything they need perfectly well, but figuring that out takes effort. On the other hand, overprovisioning is convenient. Organizations make the same tradeoff by standardizing on a few configurations. They don’t determine that Alice needs 13.7 GB of memory while Bob needs 21.2. Some employees get more computer than they need because operational simplicity has value.

AI is likely to work the same way. An organization might rigorously evaluate an agent handling millions of customer interactions because saving a fraction of a cent per interaction could justify substantial optimization work. But what about presentations and emails? What about research, spreadsheets, strategy documents, or code written by hundreds of developers working on different systems? It might simply be cheaper and easier to give everyone access to a very good model, even if it’s more intelligence than they need.

This changes the economics of cheaper inference. The choice isn’t merely frontier model: €100 versus cheaper model: €20. It’s closer to frontier model: €100, click here versus cheaper model: €20, plus figure out what your work requires, define acceptable quality, create representative tasks, codify your preferences, evaluate several models, configure them, maintain everything, and repeat whenever your work changes. That €80 buys more than intelligence. It buys you the convenience of not having to know how much intelligence you need, knowing you likely have enough.

That convenience might keep us paying for frontier inference even when cheaper models could do our work perfectly well. Local inference doesn’t automatically fix this. Imagine that five years from now a model running on your laptop is as capable as today’s best cloud models. Would you use it, or would you pay for whatever GPT-12-level system exists then because everyone says it’s dramatically better? Without knowing what good enough means for your work, you can’t know that the local model is enough.

Find the point where better stops mattering

There will always be a better model, just as there will always be a faster car than yours. Somewhere, someone also has a faster computer and a better camera. Are you missing out by not owning them? Missing out by what measure? If your car gets you where you need to go comfortably and reliably, another car being faster doesn’t automatically make yours insufficient.

Yet that’s roughly how we talk about AI models. A new model appears at the top of the benchmarks and becomes the default. We try it, and it feels smarter, so we switch. It could be substantially better. The missing question is whether that difference matters for what we do. Without an answer, you substitute “best for me” with “best available”, and there’s no end to that inference race.

This is where evals become interesting for reasons beyond improving AI systems. We usually use them to ask whether a new version got better or our instructions improved quality. We might also measure whether a tool helped an agent complete more tasks. But evals can answer another question: can I use a less capable model without making my work worse?

Instead of asking which model achieves the highest score, you’re looking for the point at which additional capability stops making a meaningful difference to you. You’re looking for your minimal viable model: the least capable model that reliably meets your requirements. Enough is a requirement you can test, rather than a feeling you borrow from a leaderboard.

Creating the evals takes work. You need representative tasks and a definition of good, captured in criteria that reflect your expectations. But much of that work is an upfront investment. When a new model appears next month, you don’t have to spend days reading benchmarks, scrolling through opinions, trying random prompts, and wondering whether you’re missing out. Run your evals. If the new model improves your work enough to justify its cost, switch. If it doesn’t, stay. When a cheaper model appears, run them again. Maybe this time you can downgrade.

Evaluation literacy compounds

You don’t write an eval suite once and preserve it in amber. Your work and expectations change as you discover new things you want AI to do. So your evals evolve with them. You add scenarios or refine criteria. You remove things that no longer matter and codify another piece of what you’ve learned about your preferences.

Along the way, you get better at evaluating AI. The first time someone asks you to define what “good” means, you might find the question surprisingly difficult. But after doing it repeatedly, you start recognizing what matters to you. And the more often you do it, the better you get at it.

And the return on evals isn’t simply money saved on inference minus the cost of running evals. There’s another cost hiding in the inference race: uncertainty. Every new model invites the same questions. Should I switch? Am I missing something? Will my existing instructions still work? Do I need to migrate my setup? Is my current model suddenly obsolete? Without evals, every release invites another round of experimentation and vibes. With evals, you can get an answer for your work, and that certainty has value too.

Perhaps inference will become dramatically cheaper, or our increasingly ambitious expectations will consume every efficiency gain. Local models might eventually handle most of what we currently send to the cloud. I don’t know, and you don’t need to predict any of it to make a sensible decision today. There will always be a smarter model and another benchmark leader. You don’t need to know which model is best. You need to know how much model is enough. So, what’s your minimal viable model?