As models get better, your agent tools should get smaller. That sounds backwards, because for years we’ve built developer tools by adding capabilities. We started with a specific problem, built a tool to solve it, and gradually expanded it to support more scenarios. The more problems a tool could solve, the more valuable it became. With AI coding agents, that logic doesn’t necessarily hold anymore.
Agent tools, such as skills and MCP servers, often exist to compensate for gaps in what an agent can do on its own. They supply knowledge and instructions the model lacks, or capabilities it couldn’t otherwise access. But unlike traditional developer tools, they’re built on a foundation that keeps evolving, and as that foundation improves, some of the gaps our tools were built to fill disappear. What made your agent tool valuable yesterday might make it a liability tomorrow.
The foundation is moving
Imagine you’ve built a skill that teaches an AI coding agent how to work with your technology. It explains the architecture and recommends patterns, and it walks the agent through common workflows with examples. When you built it, you evaluated the agent without the skill, identified where it struggled, and added instructions to address those gaps. Your evaluations showed that the skill helped the agent complete tasks. Great! You shipped it.
Six months later, the models have changed. They’re better at understanding unfamiliar APIs and reasoning through complex workflows. Some of the instructions that were essential when you built the skill might be unnecessary now if the agent already knows how to complete the task without them. Worse, they might steer it toward an approach that’s no longer the best one.
Yet your skill still loads into the agent’s context and influences its decisions. A traditional tool generally stays useful until the problem or its requirements change. An agent skill, however, can lose value simply because the model using it became more capable. And if you’re not measuring that value, you might never notice.
More isn’t always better
It’s tempting to think that additional instructions can only help. After all, if the agent already knows something, what’s the harm in telling it again? Unfortunately, LLMs don’t work that way. Every instruction you add competes for the agent’s attention. It consumes context and can influence which approach the agent chooses. Instructions can overlap or conflict with each other, and they can constrain an agent that could have found a better solution on its own.
The consequences show up in quality and cost. Your skill might make the agent process more tokens or lead it through unnecessary tool calls and reasoning steps. Even when the final result is correct, you pay more to get it. More concerningly, the skill might make the outcome worse. An agent following outdated or overly prescriptive instructions can produce an inferior solution to the one it would have found without them.
This doesn’t mean that fewer instructions are always better. Specialized knowledge and organization-specific conventions can remain essential, and so can access to external systems. The question is whether the information you’re adding gives the agent something it actually needs. Add only what’s necessary to help the agent succeed, whatever size that makes your skill.
Start with the agent, not the skill
There’s a common trap when building agent extensions: assuming from the start that an extension is needed. You know your technology and the mistakes developers commonly make with it. You can imagine dozens of instructions and examples that might help an AI coding agent use it correctly, so you package that knowledge into a skill. But how do you know the agent needs it?
You don’t, until you’ve seen what the agent can accomplish on its own. So before building anything, give the agent representative tasks, measure the quality of its outcomes and the cost of completing them, and inspect its trajectory to understand where it struggles. Only then decide whether an intervention is necessary.
Perhaps the agent lacks specific knowledge about your API. Perhaps it struggles to discover the right CLI command, or it knows what to do but keeps choosing an inefficient approach. These are different problems that might require different solutions. A skill might help, but so might clearer documentation and tool descriptions, or a change to the underlying API. If you start by asking how to improve your skill, you’ve already assumed the answer. Don’t become so invested in your solution that you forget the problem you’re trying to solve.
Make pruning part of development
Building a useful skill isn’t a one-time exercise, and neither is proving that it works. The baseline you measured before building the skill describes the agent as it was then, and each model release can change it. So as models improve, revisit the assumptions that justified your skill in the first place. A new model release is a natural moment to reevaluate. Agent harnesses evolve too, sometimes daily, so evaluating on a practical, periodic cadence works better than reacting to every small change. The process itself is straightforward:
- Reestablish the baseline. Evaluate what the agent can accomplish without your skill using representative tasks and agent profiles.
- Identify the remaining gaps. Examine outcomes, task costs, and trajectories to understand where the agent still struggles.
- Compare those gaps with your skill. Determine which instructions and capabilities address problems that still exist.
- Prune what’s no longer needed. Remove redundant instructions, simplify workflows, or retire the entire skill if it no longer contributes meaningful value.
- Verify the result. Evaluate the changes to ensure that removing functionality hasn’t introduced regressions.
Keep in mind that evaluation results are contextual. A skill that improves outcomes with one model and harness might provide no benefit with another. If you’re building for a broad audience, evaluate across the agent profiles your audience actually uses, meaning the combinations of harness, model, and other extensions they work with. There’s no universal verdict on whether a skill is useful.
Similarly, the absence of a measurable improvement doesn’t automatically prove that an instruction is unnecessary. Perhaps your evaluations don’t cover the scenario it was designed to address, and that’s worth investigating before you remove it. Everything in your skill should serve a purpose, and if you can’t demonstrate that something helps, you should at least question why it’s there.
Learn to let go
For developers used to building tools, pruning can feel counterintuitive. We spend time understanding problems and designing solutions, and we take pride in what we’ve built. Removing functionality can feel like throwing away that investment, particularly if the feature was genuinely useful when we created it.
But agent tools operate in a different environment. The models beneath them keep getting more capable, independently of our work. We don’t control that progress, and we shouldn’t pretend it isn’t happening. Holding on to instructions simply because we spent time writing them risks making the experience worse for the very people we’re trying to help.
Measure your skill’s success by whether agents can use your technology to achieve high-quality outcomes at an acceptable cost. How many capabilities the skill contains or how often agents invoke it says nothing about that, and neither does the work you’ve put into it. Sometimes improving that experience means adding something, and sometimes it means changing something. Increasingly, it might mean removing something.
That requires a different mindset from the one we’ve traditionally applied to developer tools. Instead of assuming that every iteration should make our tools more capable, we need to keep asking whether those capabilities are still necessary. Your agent tools should get smaller as models get better, because each new model can close some of the gaps they were built to fill. Build what agents need. Measure whether it helps. And be willing to remove it once they no longer need it.