Module 5.13

The upgrade decision map

SmolLM2-135M-Instruct answers bc-09's arithmetic question wrong, every time. Three real ways forward — not "always go bigger," a real tradeoff each. Click a route.

Route 1, in code

Same script, bigger model id

terminal

pip install llama-cpp-python   # Route 2: run a quantized GGUF file, no torch/transformers needed
ollama run qwen2.5:0.5b       # Route 2, alternative: Ollama manages the download and quantization for you

Route 1 is bc-08 with a bigger MODEL_ID — no new concept, just more RAM and disk. Route 3 swaps the whole tokenizer/model pair for one HTTP request to a hosted API, at a real per-token cost instead of a real RAM cost.

So what

"Get a better answer" was never one decision — it's RAM, disk, or dollars, and you get to pick which one you spend.