agents
I let LLMs play a 1988 detective gamebook 646 times
In a hot-air balloon with a storm coming, almost every model made the sensible choice and landed. About ten decisions later, that choice cost them the case.
Run Vally evals on GitHub Copilot in VS Code
Your skill worked when you tried it in GitHub Copilot Chat. After you change it next week, how will you know it still does?
Evaluating GitHub Copilot in VS Code just got easier
Your users run GitHub Copilot in VS Code, but most agent evaluation frameworks work with CLI agents. Close that gap, and you suddenly need to automate an IDE that changes every week.
Agents removed the friction that built our technical literacy
An agent fixes a runtime version and an authentication problem on its way to getting your app running, then reports "Done." You saved twenty minutes, and you lost whatever those twenty minutes used to teach you.
If you can't automate your workflow, neither can your users' agents
You want to test whether an agent can deploy an app to your platform. But before you can check the agent's work, your eval harness has to click through a portal and an interactive sign-in. Whose problem is that?