Module 3.8

Same script, one line changed

Same prompt, same tokenize → generate → decode harness from bc-01, run through three different sets of weights. Only the model id string changes. Watch what that one line is actually responsible for.

PROMPT: "Hi, how are you?" — GREEDY DECODING, MAX 30 NEW TOKENS

The script

One constant, one variable

bc-08-same-script.py

from transformers import AutoTokenizer, AutoModelForCausalLM

# the only thing that changes between runs is this one string
MODEL_ID = "HuggingFaceTB/SmolLM2-135M-Instruct"
# MODEL_ID = "distilbert/distilgpt2"
# MODEL_ID = "openai-community/gpt2"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(MODEL_ID)

inputs = tokenizer("Hi, how are you?", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=30)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Every other line is identical to bc-01. AutoTokenizer/AutoModelForCausalLM exist specifically so the harness doesn't need to know which architecture it's loading. Download bc-08-same-script.py ⬇

Try this

Uncomment the Qwen/Qwen2.5-0.5B-Instruct line from bc-07's table into bc-08-same-script.py ⬇ instead. It's instruct-tuned like SmolLM2 — predict whether it reads more like the base models' output or SmolLM2's, before you run it.

So what

The harness never changed — only the weights did, and that alone was the entire difference in behavior.