Imagine you’re building an eval to test whether an AI coding agent can deploy an application to your cloud platform. You want to verify that the generated application actually works, so you need to provision resources, deploy the code, validate its functionality, and clean up afterward. Sounds straightforward. Until you try to automate it. Some operations require navigating a portal. Authentication involves interactive steps. Provisioning means coordinating several services, each with its own configuration and failure modes. Even telling whether the deployment succeeded isn’t always obvious.
You could work around these limitations in your eval harness. But consider what you’ve just discovered: if you struggle to automate your workflow, your users’ agents are likely struggling with exactly the same things. In that case, the obstacles in your harness are a product problem, and working around them in the harness leaves them in place for everyone else.
Agents expose friction we’ve learned to tolerate
Many developer workflows were designed around humans. We navigate portals, copy values between screens, interpret error messages, and wait patiently for operations to complete. Over time, we learn the quirks of our tools and develop workarounds. Agents usually don’t have that experience. They need to discover how to accomplish a task, reach the functionality it requires, coordinate the operations, and determine whether they’ve succeeded. A workflow that requires manual interaction or undocumented knowledge can stop an agent entirely. And even when every operation is technically accessible, coordinating dozens of steps adds complexity and more opportunities to fail.
Agents already run infrastructure workflows at scale. In its 2026 State of AI Agents report, Databricks reported that AI agents create 80% of the databases on Neon and 97% of its database branches. As agents become a common way to work with technology, workflows that assume a human operator turn into a problem for more and more of the people who use them.
Friction also costs money. A human developer might tolerate several minutes of manual configuration during an occasional deployment. Agents, however, run workflows repeatedly, and every unnecessary operation and retry consumes time and inference, as does every round of reasoning the agent repeats along the way. With agents, workflow friction directly increases the cost of completing a task. So a workflow can be technically possible for an agent to execute and still offer poor Agent Experience (AX).
Making a workflow accessible is only the beginning
The obvious fix is to expose the functionality through programmatic interfaces such as APIs, CLIs, or MCP tools, so that agents can perform operations without navigating a graphical interface. That’s a valuable improvement, but it leaves the underlying complexity in place. Consider a deployment workflow that requires creating a project, configuring an environment, provisioning infrastructure, assigning permissions, deploying code, and verifying the result. With an API for each operation, the workflow becomes automatable, but the agent still needs to discover and coordinate every step.
A skill can help by describing the sequence and handling common pitfalls, and it can be a reasonable interim solution. But skills bring dependencies of their own. Users need to find out that the skill exists and install it. Then they need to keep it up to date. You’re betting the usability of your technology on users taking those extra steps. Compare that with a baseline agent that discovers and uses your technology through its existing tools and public documentation. When your technology works for that agent, it works for every user who asks an agent for help, including the ones who have never heard of your skill.
The longer-term opportunity is to simplify the workflow itself. For example, your platform could offer a single operation that handles everything from provisioning to validation, with clear progress information and actionable errors. Rather than teaching every agent how to coordinate a complicated process, you make the process simpler for everyone. And when developers deploy dozens or even hundreds of times a day through agents, improvements in reliability and latency add up, and so do the savings, until they can set your platform apart.
Let agents show you where the friction is
How do you find the workflows that need attention? Start the same way you would when improving Developer Experience: use your technology yourself, then talk to your users and watch how they work. Pick a common task and ask an agent to accomplish it. Use a simple, goal-oriented prompt rather than prescribing the steps. If you’re testing deployment to your platform, name the platform, but let the agent figure out how to use it.
And watch out for your own expertise! When you know your product inside out, it’s tempting to hand the agent the exact commands and configuration it needs, along with links to the right documentation. It’s easy to steer it, without noticing, around problems that less experienced users would run into. The more product-specific knowledge your prompt assumes, the narrower the audience it represents. The test is whether the agent can succeed without your help.
Then watch what happens. Where does the agent get stuck? Does it find the right interfaces? Can it authenticate? Does it understand the errors it gets? Does it repeat unnecessary operations? Can it verify that the task is complete? Look beyond outright failures, too. An agent that eventually succeeds after dozens of unnecessary tool calls is still showing you an expensive, inefficient workflow.
When you find a problem, investigate its cause before you choose a solution. Sometimes the documentation or error messages need work. Sometimes an API is missing, or the workflow itself needs redesigning. And sometimes the problem lies with the agent or its tools rather than your product. Understand the friction first, because a new skill or MCP server is only one of the possible answers.
Don’t let telemetry hide unmet demand
Usage telemetry can help you prioritize improvements, but it has a blind spot. A workflow that agents rarely use looks unimportant. Yet agents might rarely use it precisely because they can’t complete it. Low agent usage can be evidence of a barrier rather than of low demand.
So don’t rely on usage statistics alone. Talk to developers and test representative scenarios yourself, looking for workflows that people want agents to perform but still complete by hand. Your long-term ambition should be to make your technology broadly usable by agents. Start with the most important workflows, and let both observed usage and unmet user needs decide which ones those are.
Make workflow friction part of your feedback loop
You don’t evaluate Agent Experience once and declare it finished. Your product keeps evolving. Models change, and agent harnesses introduce new capabilities and behaviors. A workflow that works well today can expose new bottlenecks tomorrow. So make representative agent workflows part of your ongoing product development. Watch how agents behave, and let the failures and inefficiencies you find guide your improvements.
Not every finding requires immediate action. Sometimes the right decision is to accept a limitation, because addressing it would cost more than the expected benefit. That’s a legitimate product tradeoff, as long as you make it deliberately. The fix might also belong to a different team than the one that found the problem. Improving AX means looking beyond the boundaries of individual tools and product components.
Whatever you decide, treat the friction as feedback rather than something to work around and forget. As agents take on more development tasks, their ability to run your workflows on their own becomes part of how your product works. Exposing APIs and tools is a start. The larger opportunity lies in making those workflows discoverable and reliable for an agent, without wasting its time or your users’ tokens.
The next time you struggle to automate a workflow, whether for an eval or anything else, don’t reach for a more sophisticated harness right away. You might have found exactly where your technology needs to improve. Pick a common task. Let an agent attempt it and watch where it struggles. Then ask yourself whether the problem lies with the agent or with the workflow you’ve asked it to navigate.