What Is a Context Window? AI Memory, Explained
It is the model's short-term memory, and it runs out. Understanding it explains why long chats drift and big documents get cut off.
A context window is an AI model’s short-term memory. It is the amount of text the model can read and keep in mind for a single request, and it is measured in tokens rather than words. Your prompt, the files you attach, the back-and-forth of the chat so far, and the reply the model is about to write all have to fit inside it. Run past the edge and the model simply cannot see what fell off.
So what is a context window, really?
Think of the model as someone working at a desk. The context window is the desk. Anything on the desk is fair game: the model can look at it, quote it, and reason about it. Anything that does not fit on the desk might as well not exist for this task. It is not a filing cabinet the model can rummage through, and it is not long-term memory. It is the space in front of the model right now.
Everything on that desk is measured in tokens, the small chunks of text models actually read. A token is about four characters of English, so a token is roughly three-quarters of a word. If that idea is new, the plain-English guide to tokens is the place to start, and you can watch text turn into tokens in the token counter.
How big are context windows in 2026?
Sizes vary a lot by model. As a rough map of the current landscape: the GPT-4o family sits around 128,000 tokens, Claude models hold about 200,000, and Gemini stretches to a million tokens, with some versions reaching two million. Smaller open models you run yourself are often 8,000 to 128,000. These numbers move as new versions ship, so treat them as a snapshot, and check the current figures in the model comparison before you rely on one.
A word of caution on the headline numbers: a million-token window is impressive, but it is a ceiling, not a promise of quality. It tells you what can fit, not how well the model uses it.
How much text actually fits?
Tokens are hard to picture, so here is the conversion most people want. One token is about 0.75 words. That gives you a few handy anchors:
- 128,000 tokens is about 96,000 words, or roughly 200 pages.
- 200,000 tokens is about 150,000 words, longer than most novels.
- 1,000,000 tokens is about 750,000 words, around a dozen books.
Those are English-prose estimates. Code, spreadsheets, JSON and languages other than English pack more tokens into the same space, so they fit less than the word counts suggest. To check whether a specific document will fit a specific model, drop the numbers into the context window calculator.
Why bigger is not always better
It is tempting to reach for the largest window and paste in everything. That backfires more often than people expect, for three reasons.
First, attention thins out. Research on long inputs keeps finding the same pattern: models are sharpest on what sits at the very start and the very end of the window, and weakest on what sits in the middle. Bury the key sentence on page 90 of 200 and the model may skim past it.
Second, tokens cost money. Most AI APIs bill by the token, so a full window on every call adds up quickly. If you are building on an API, that maths matters, and the breakdown of what AI APIs cost is worth a read.
Third, more tokens mean more waiting. A packed window takes longer to process, so the reply arrives more slowly. For a chat app that is the difference between feeling snappy and feeling sluggish.
Context window versus training data
These two get mixed up constantly, so it is worth being clear. The context window is temporary and specific to your current conversation. The model’s general knowledge came from training, which happened once, before the model was released, and stops at a fixed cutoff date. Ask about last week’s news and a model with an older cutoff will not know unless you paste the details into the window. The desk is today; training is everything it studied before it sat down.
What to do when you hit the limit
When your text is too big for the window, you have three good moves. Summarise the old parts of a long chat instead of carrying every message forward. Break a large document into sections and work through them one at a time. Or use retrieval, where a system fetches only the few relevant passages and hands those to the model rather than the whole library. Each keeps the model focused on what matters and keeps your token bill down.
Understanding the window is one of those ideas that quietly explains a dozen other things: why long chats drift, why a model forgot your earlier instruction, why a huge PDF got cut off. For the rest of that picture, the practical guide to AI literacy ties it together.
Frequently asked questions
What is a context window in simple terms?
It is how much an AI model can hold in mind at once, measured in tokens. Everything you type, everything it has said so far, and any files you attach have to fit inside it. When the total runs past the limit, the oldest parts get dropped.
How many words is a 128K context window?
Roughly 96,000 words, or about 200 pages of a paperback. The rule of thumb is one token to about 0.75 words in English, so 128,000 tokens lands near 96,000 words. Code, tables and other languages use more tokens per word, so they fit less.
Does a bigger context window mean better answers?
Not on its own. A larger window lets you paste more in, but models tend to pay less attention to the middle of a very long input, and every extra token costs money and adds delay. Feeding a model only what it needs usually beats dumping everything in.
What happens when you exceed the context window?
The model cannot see past the limit, so the tool either refuses the request or quietly trims the oldest text to make room. In a long chat that is why the assistant seems to forget what you said near the start.
Is the context window the same as the model's knowledge?
No. The context window is temporary memory for the current conversation. The model's general knowledge was baked in during training and has a cutoff date. The window is the desk it works on today; training is everything it learned before it sat down.
The Scroll Team writes the lessons inside Scroll: Learn AI, a microlearning app that teaches how AI works in one minute a day. We read the papers and release notes so you do not have to.