agent experience
A knowledge cutoff is a ceiling, not a snapshot
A model's knowledge cutoff looks precise. But how much does that date actually tell you about what it knows?
Test whether your knowledge skill actually works
Your knowledge skill gives reasonable answers to the questions you've tried. What would you find in the parts of the source you didn't think to test?
What if coding agents had a learning mode?
Coding agents can take us from intent to production in minutes. What happens to the understanding we used to build along the way?
Your product didn't get worse
Customers still like your product and keep renewing. Meanwhile, their expectations may be changing in ways your usual signals won't reveal.
Your coding agent extension needs continuous evaluation
Your extension passed its evals. Days later, its code is identical, but can you still trust the result?