agent experience
An eval score is only half the result
Your agent's score improved. Its inference bill increased 15 times. Before you celebrate, decide what the improvement is actually worth.
Your agent eval is more than its result
Your agent completed the task and every grader passed. What happened before the score reached 100%?
A knowledge cutoff is a ceiling, not a snapshot
A model's knowledge cutoff looks precise. But how much does that date actually tell you about what it knows?
Test whether your knowledge skill actually works
Your knowledge skill gives reasonable answers to the questions you've tried. What would you find in the parts of the source you didn't think to test?
What if coding agents had a learning mode?
Coding agents can take us from intent to production in minutes. What happens to the understanding we used to build along the way?