AI ENGINEERING: DEEP DIVE · MODULE 01 — THE MATH UNDERNEATH · 01 A hands-on series · math + code

Module 1.1

Everything is a list of numbers

Before attention, before transformers, before any of it: a sentence becomes a list of numbers, and "similar meaning" becomes "pointing the same direction." Drag through a few vector pairs and watch what "similarity" actually computes to.

TWO VECTORS, a AND b
a b
a · b24
‖a‖5.00
‖b‖5.00
cos θ0.96
θ16.3°
vector a vector b angle between them

The math

Three formulas, worked with the numbers on screen

Dot product
a · b  =  a₁b₁ + a₂b₂
= (4)(3) + (3)(4)
= 24
Vector norm
‖a‖  =  √(a₁² + a₂²)
= √(4² + 3²)
= 5.00
Cosine similarity
cos θ  =  (a · b) / (‖a‖‖b‖)
= 24 / (5.00 × 5.00)
= 0.96

Norm and dot product are the only two ingredients. Cosine similarity is just their ratio — it cancels out length and keeps only direction.

The code

The same three lines, every time

toy_vectors.py

This is the entire computation. No library hides anything here — np.dot and np.linalg.norm are doing exactly the sums above.

The Nimbus thread

The same computation, on real-shaped embeddings

Nimbus support corpus (fixed, 6 sentences — used across this whole series)
  • 1. "Nimbus exports fail if the workspace quota is exceeded."
  • 2. "API keys rotate every 90 days and must be reissued from Settings."
  • 3. "…4 more support sentences, reused verbatim in later modules."
Query

"Why did my Nimbus export fail?"

Query embedding (8 illustrative dimensions)
Doc 1 embedding
a · b = 0.76 ‖query‖ = 0.89 ‖doc‖ = 0.86 cos θ = 0.99
High similarity — this is the sentence a vector search would return first.

Illustrative embedding values, not real model output — the point is the operation, not the exact numbers.

So what

Attention, retrieval, everything downstream in this series is this exact same dot-product-over-norms computation — just repeated millions of times, over higher-dimensional vectors.

Go deeper (optional)

Where the dot product's geometry actually comes from

This page showed the dot product as a formula. 3Blue1Brown's classic explanation is about why that formula measures projection — the "duality" between a vector and the linear function it defines. Longer than most clips here (15 min), but it's the definitive treatment of this exact idea.

3Blue1Brown — "Dot products and duality," Essence of Linear Algebra ch. 9. Supplementary, not required to follow this page.