When a model has a knowledge cutoff of June 2025, it’s tempting to imagine that its knowledge represents the state of the world in June 2025. Everything published before then is in, and everything published after is out, as if someone took a snapshot of the internet on June 30 and trained the model on it. But that’s not what a knowledge cutoff means. A knowledge cutoff is a ceiling: the model’s knowledge isn’t newer than a particular date. It doesn’t tell you how recent its knowledge is about the thing you’re asking about. If you’re building or configuring AI agents, mistaking the ceiling for a snapshot can make you repeat knowledge the model already has while leaving its obsolete patterns uncorrected.
Not newer than
Ask a model with a June 2025 cutoff about one technology and its knowledge might extend well into 2025. Ask it about another and the latest information it reliably knows might be from 2024. For a less common project, its useful knowledge might be several years old.
I’ve seen this for example with Dev Proxy. Even the latest models exhibit knowledge of Dev Proxy that is several years out of date. Their advertised knowledge cutoff doesn’t contradict this because it promises not newer than June 2025, rather than current as of June 2025. For any particular topic, you cannot determine how current the model’s knowledge is merely from knowing its cutoff. But establishing the latest version it recognizes doesn’t settle the question either.
What’s in the weights
Suppose you could establish that a model knows version 5 of an SDK. That gives you a quick indication of roughly where its knowledge ends, but it doesn’t mean the model knows everything about version 5. Training data isn’t necessarily a complete snapshot of a product’s documentation. The model might know one feature introduced in version 5 and completely miss another. It might know the new API but still reach for an older pattern when solving a particular task because other knowledge encoded in its weights competes with the newer information.
The advertised cutoff gives you the ceiling. Knowing the latest version the model appears to know gives you a better approximation. Ultimately, though, you still need to observe how the model behaves on the tasks that matter to you.
Who cares?
This uneven knowledge changes what an AI coding agent extension needs to contain. Say you’re building a skill that teaches an agent how to use an SDK. You could put all the current documentation into the skill so that whatever the model already knows shouldn’t matter.
But context windows are finite. Every token you spend explaining something the model already knows is unavailable for the user’s task, source code, conversation, tool output, or other extensions. Additional context isn’t inert either. It becomes part of what the model reasons over, so even correct information can change the resulting behavior.
Ideally, an extension adds what the model is missing and avoids repeating what it already reliably knows. Yet missing knowledge is only half the problem. A model whose framework knowledge is several versions behind doesn’t start from zero. It already has APIs and recommended patterns from older versions, some of which may now be obsolete.
Your extension might therefore need to teach the model what’s new and explicitly counter what it already knows. Don’t use this API anymore. Don’t generate this configuration because the current version handles it differently. A model with no knowledge of a technology gives you an empty space to fill. On the other hand, a model with strong but obsolete knowledge gives you something to push against. And the worst part is, that the advertised cutoff cannot tell you which situation you’re dealing with.
Establish the baseline through behavior
You can ask a model what it knows, but don’t mistake the answer for ground truth. Asking, “What’s the latest version of this SDK that you know?” can indicate roughly where its knowledge ends. The model might correctly identify a version or underestimate its knowledge. It might also confidently generate a plausible answer that doesn’t accurately describe what is represented in its weights. Even a correct answer doesn’t establish that the model knows all the relevant features and patterns from that version.
Before deciding what the extension should contain, evaluate the tasks that matter to you without it. Don’t look only at whether the final answer passes or fails. Watch which APIs and patterns the model reaches for. Which assumptions does it make? Does it mix old and new approaches? Does the same obsolete pattern keep appearing?
You aren’t trying to reconstruct the model’s entire training corpus. You need to understand enough of its existing knowledge to identify the gaps relevant to your work. Knowing the latest version the model reliably knows can orient you, but task-level evaluation remains necessary. There is no guarantee that the model learned everything associated with that version, or that the relevant knowledge will win over competing patterns when it generates an answer. A knowledge cutoff gives you an upper bound, while a model’s useful knowledge of the topic you care about could end anywhere below it. When you build or configure an agent extension, don’t infer the model’s baseline from its advertised cutoff. Look at what it actually does.