You start a long chat with an AI assistant. You explain your project in detail in the first message, share your rules, paste a few files and go back and forth for an hour. Then the AI suddenly ignores a rule you set at the start, or asks about something you already explained.
It did not get lazy. You probably ran into its context window.
The context window is one of those concepts that sounds technical but explains a lot of everyday AI behaviour: why long chats drift, why some tools refuse a huge PDF, why API bills grow with conversation length, and why a model with a "1 million token" headline still misses details in the middle of a document.
I treat the context window as the second thing every AI learner should understand, right after tokens. Once you get it, you stop fighting the tool and start working with its limits. This page covers what a context window is in an LLM, how it is measured, what happens when it fills up, why bigger is not always better, and practical tips for ChatGPT, Claude and Gemini.
What is a context window in an LLM?
A context window in an LLM is the fixed amount of text, counted in tokens, that the model can consider at once when generating a response.
Every time you send a message, the app does not just send your latest line. It sends a bundle: the system instructions, the earlier conversation, any files or search results, and your new message. The model reads that entire bundle and writes a reply. The context window is the maximum size of that bundle, plus the reply.
Three things to lock in:
- It is measured in tokens, not words or pages. A token is a chunk of text, often part of a word.
- It is a hard limit per request. If the bundle is too big, something has to be cut, summarised or rejected.
- It is the model's only view of your conversation. A large language model has no hidden memory of your chat between requests. If text is not in the window, the model cannot see it.
That last point surprises many beginners. Chat apps feel like they remember you, but technically the app re-sends the conversation each time. Features like "memory" in some chat apps work by saving notes about you and inserting them into the context window later. Everything still flows through the same window.
When people search "what is context window in AI" or "in Claude" or "in ChatGPT", the answer is the same idea. Only the size and the way each app manages it differ.
Context window explained with a simple analogy
The simplest way to understand a context window is to think of it as the AI's working memory, like the desk you study on, not the library you studied from.
Picture yourself preparing for an exam at a small desk. Your desk can hold only so many open books, notes and question papers at once. Whatever is on the desk, you can read and use right now. To bring in a new book when the desk is full, something else has to go back on the shelf.
Map that onto an AI:
- The desk is the context window.
- The papers on the desk are your prompt, chat history and documents.
- What you learned over years of study is the model's training. That knowledge lives in the model's weights, separate from the desk.
- Clearing the desk is what happens when a chat app drops or summarises old messages.
A bigger desk lets you spread out more material, which helps with long reports or large codebases. But a bigger desk does not make you read faster or pay attention to every corner equally. Keep that in mind, because it comes back when we talk about long-context trade-offs.
Another comparison I use with students: the context window is like the RAM in your phone, and training is like the storage. Apps you are actively using sit in RAM. Close enough apps and the old ones get pushed out.
Context window vs tokens: how is a context window measured?
A context window is measured in tokens, the small text units a model reads, so "context window vs tokens" is really about the size of the container versus the unit used to fill it.
For English text, a common rule of thumb from OpenAI is that one token is roughly four characters, or about three quarters of a word. That gives quick estimates:
| Amount of tokens | Roughly how much English text |
|---|---|
| 1,000 tokens | About 750 words, a long essay answer |
| 8,000 tokens | About 6,000 words, a short report |
| 100,000 tokens | About 75,000 words, a full-length novel |
| 1,000,000 tokens | About 750,000 words, several thick textbooks |
Treat these as estimates. Exact counts depend on the model's tokenizer, and different models split text differently.
Two things matter for Indian users:
- Hindi and other Indian languages often use more tokens per word than English in many tokenizers, so the same amount of meaning can fill the context window faster. Newer tokenizers have improved this, but it is still worth checking with the provider's token counter.
- Code, numbers and tables can be token-heavy. A spreadsheet pasted as text can consume far more tokens than its size on screen suggests.
Most AI platforms offer a tokenizer tool or a token-counting option in their API. If you work with long documents, using one once will calibrate your intuition better than any table. Our page on what tokens are in AI explains tokenization in detail.
What counts toward the context window? (input and output share it)
Everything the model reads and everything it writes in a single request counts toward the context window, so input and output share the same space.
Here is what typically fills it, in order:
- System prompt: hidden instructions from the app or developer about how the assistant should behave.
- Memory or custom instructions: saved preferences or notes some apps add automatically.
- Conversation history: every earlier message from you and the model.
- Attachments and tool results: PDFs, code files, web search results, or data returned by tools such as MCP servers.
- Your new message.
- The model's reply, including any hidden reasoning tokens a reasoning model generates before answering.
That last item catches people out. If a model has a large window and you fill almost all of it with input, there is little room left for a long answer. Many APIs also set a separate cap on output tokens, which is usually much smaller than the full window.
Why agents fill the window fast
AI agents are especially hungry. Each step adds a tool call, a tool result and the model's thinking. A coding agent that reads ten files and runs tests a few times can use a big share of its context window in one task. That is why agent tools put so much effort into managing context.
Context window limit: how big is it in ChatGPT, Claude and Gemini?
Context window limits range from a few thousand tokens in early models to around a million tokens or more in some recent models, and the exact number depends on the model, the product and sometimes your plan.
A rough history, as hedged ranges:
| Period | Typical context window | What that allowed |
|---|---|---|
| Early chat models (around 2020 to 2022) | A few thousand tokens | A short conversation or a few pages |
| 2023 onward | Tens of thousands to a couple of hundred thousand tokens | Long reports, whole research papers, bigger code files |
| Recent models | Some offer around a million tokens or more | Very long documents, many files, large codebases |
How this plays out in popular tools:
- Claude: Anthropic publishes context window sizes for each Claude model in its documentation, and has offered large windows for long documents and code. Claude Code also manages long sessions by summarising older parts of the conversation when it gets close to the limit.
- ChatGPT: the context window depends on which model you pick and your subscription tier. The ChatGPT app may offer less than the same model through the API.
- Gemini: Google has promoted very long context windows for some Gemini models, aimed at whole books, long videos and big codebases.
I am deliberately not listing exact numbers per model. They change several times a year, and a number copied from a blog post can be out of date within months. Always check the current official documentation for the model you use before planning around a limit.
My contrarian take: for most students and everyday users, the advertised maximum barely matters. A normal study chat uses a small fraction of any modern window. What matters more is how well the model uses what is already there.
What happens when the context window is full?
When the context window is full, something has to give: the request is rejected, older content is dropped or summarised, or the model's reply gets cut off.
What you see depends on where you use the model:
- Through an API: if your input plus the requested output exceeds the limit, the request usually fails with an error, or the output stops when it hits the token cap. Developers must trim or summarise input themselves.
- In chat apps: apps often quietly drop or compress older messages so the conversation can continue. The chat looks unbroken, but the model may no longer see your early instructions.
- In coding agents: tools may automatically summarise (sometimes called "compacting") older history when space runs low, keeping key decisions and discarding detail.
- With very large uploads: the app may refuse the file, read only part of it, or use search inside the file to pull relevant chunks instead of reading it all.
Common symptoms that you are near the limit:
- The AI forgets a rule or detail you gave early in the chat.
- It contradicts something it said much earlier.
- Answers on a long document miss sections you know are there.
- Replies end abruptly mid-sentence.
When context is lost, the model may also fill gaps with plausible guesses, which is one route to AI hallucination. The fix is usually simple: start a fresh chat with a short summary of what matters, instead of dragging a very long thread forward.
Is a bigger context window always better?
No, a bigger context window is not always better, because long inputs cost more, run slower and are not always used evenly by the model.
1. Cost
API pricing is usually per token. Send 200,000 tokens of input with every message and you pay for all of them, every time. Some providers charge more per token for very long prompts, and features like prompt caching exist partly to soften this. Check the pricing page of your provider.
2. Speed
More input means more computation before the first word of the answer appears. The attention mechanism at the heart of the transformer compares tokens with each other, and in the standard design that work grows quickly as the input gets longer. Engineers use many tricks to manage it, but very long prompts are still slower.
3. The "lost in the middle" effect
A 2023 research paper titled "Lost in the Middle: How Language Models Use Long Contexts" found that the models it tested performed best when the relevant information was near the start or end of a long input, and noticeably worse when it sat in the middle. Newer models have improved on this kind of test, but the lesson still holds: placement and focus matter.
4. Distraction
Pasting everything "just in case" adds noise. Irrelevant text can pull the model's attention away from what you actually asked.
I would rather give a model 5 pages of the right material than 500 pages of everything.
Context window vs RAG: do you still need RAG with long context?
Yes, you often still need RAG even with a long context window, because retrieval picks the right information while a long window only holds more of it.
RAG (retrieval-augmented generation) searches a large collection of documents, finds the few passages most relevant to a question, and puts only those into the context window. A long context window lets you skip the search and paste a lot directly.
| Long context window | RAG | |
|---|---|---|
| Best for | One big document or a handful of files you need read fully | Thousands of documents, or data that changes often |
| Cost per question | High if you resend everything each time | Lower, since only relevant chunks are sent |
| Risk | Details lost in the middle, slower replies | Retrieval may miss the right passage |
| Setup effort | Very low, just paste or upload | Needs embeddings, a vector database and tuning |
In practice, the two work together. RAG decides what goes in, and the context window decides how much can go in. A college helpdesk bot with 2,000 policy PDFs needs retrieval. A student analysing one 80-page research paper can simply upload it to a long-context model.
For job interviews, this comparison is a favourite question: "If models have million-token windows, is RAG dead?" A strong answer covers cost, latency, freshness of data, the lost-in-the-middle effect and access control. Our AI interview prep section has more questions like this.
How to use the context window well: practical tips
To use a context window well, give the model only what it needs, put the most important parts where it will notice them, and restart long chats before they drift.
Practical habits that work in ChatGPT, Claude, Gemini and coding tools:
- Start a new chat for a new task. One topic per conversation keeps the window clean and focused.
- Summarise and restart. When a chat gets long, ask: "Summarise the key decisions, rules and open questions so far in under 200 words." Paste that summary into a fresh chat.
- Put key instructions at the start and repeat them near the end of very long prompts, given the lost-in-the-middle effect.
- Paste the relevant section, not the whole book. If your question is about Chapter 4, share Chapter 4.
- Ask the model to quote. For long documents, "quote the passage you used" makes it easy to check that the model actually found the right part.
- Use projects or custom instructions for rules you repeat often, so you do not have to re-paste them, but keep them short since they also take up space.
- Watch for drift. If the AI starts forgetting rules, do not argue with it for ten messages. Restart with a clean summary.
These tips overlap a lot with good prompt engineering, because managing context is a big part of prompting well. I tell learners that "context engineering", choosing exactly what goes into the window, is quickly becoming as important as wording the prompt itself.
Key takeaways
- A context window is the maximum amount of text, measured in tokens, that an LLM can consider in one request.
- Input and output share the window: system prompt, chat history, files, tool results and the model's reply all count.
- Sizes range from a few thousand tokens in early models to around a million or more in some recent ones, so check current official documentation.
- When the window is full, apps reject requests, drop or summarise older messages, or cut replies short, which can cause forgetting and hallucinations.
- Bigger windows cost more, run slower and can miss details in the middle, so focused context and RAG still matter.