Module 2.6 · Deepens page 4
Page 4's "desk" has a real, numeric edge on this model: 8,192 tokens. Build a conversation deliberately past that, and read exactly what the library says — not a diagram of a desk running out of room, an actual stack trace.
python bc-06-context-wall.py
this model's context limit: 8192 tokens
this conversation is 9031 tokens long
[transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (9031 > 8192). Running this sequence through the model will result in indexing errors
[transformers] This is a friendly reminder - the current text generation call has exceeded the model's predefined maximum length (8192). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.
generate() returned a result anyway — shape: (1, 9034)
but read the warnings above: it explicitly says results past the limit aren't reliable.The script
print("this model's context limit:", model.config.max_position_embeddings, "tokens")
# build a conversation deliberately longer than that limit
huge_message = "word " * 9000
history = [{"role": "user", "content": huge_message}]
prompt = tokenizer.apply_chat_template(history, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
print("this conversation is", inputs["input_ids"].shape[1], "tokens long")
outputs = model.generate(**inputs, max_new_tokens=5)Notice this model didn't hard-crash — it kept going and warned instead. Some models/setups do raise a hard error here; either way, going past the trained limit is not a supported, reliable state. Download bc-06-context-wall.py ⬇
Change 9000 down to 100 in bc-06-context-wall.py ⬇ and re-run. Predict first: do the warnings disappear entirely, or just change wording?
So what
The context window isn't a metaphor — it's a real number, and the library tells you when you've walked past it.