Part I · Maths · Chapter 4

Vectors and matrices

A vector is a list of numbers; a matrix is a table of them. Almost every computation inside a neural network is a matrix multiplied by a vector, so this chapter is where AI's arithmetic lives.

Why it matters

A search system can turn every passage of a document set into a list of 384 or 1,024 numbers and find relevant passages by comparing those lists. A language model is, in arithmetic terms, a long series of matrix multiplications. GPUs exist to do this one operation very fast.

Vectors: lists of numbers

A vector is an ordered list of numbers. A flat described by (area 60, bedrooms 2, floor 3) is a 3-number vector. Each position has a fixed meaning, and the count of numbers is the vector's dimension.

A 2-number vector can be drawn as an arrow from the origin (0, 0): the vector (3, 1) points 3 to the right and 1 up. Higher dimensions cannot be drawn, but every rule below works the same way in 384 dimensions as in 2.

Adding and scaling

  • Add position by position: (3, 1) + (1, 2) = (4, 3). As arrows: put the second arrow's tail on the first arrow's tip.
  • Multiply by a single number (a scalar): 2 × (3, 1) = (6, 2). Same direction, twice as long. −1 × (3, 1) = (−3, −1) points the opposite way.

Length

The length of (3, 4) comes from Pythagoras: √(32 + 42) = √(9 + 16) = √25 = 5. In any dimension, square every entry, add, take the square root.

|a| = √(a12 + a22 + … + an2)Also called the norm, or the L2 norm.

Dividing a vector by its length gives a unit vector, of length 1, pointing the same way: (3, 4) / 5 = (0.6, 0.8). Check: 0.36 + 0.64 = 1. This is called normalising, and embedding models usually return unit vectors.

The dot product

You met it in chapter 2: multiply matching positions, then add. Worked in 3 dimensions:

(1, 2, 3) · (4, 5, 6) = 1×4 + 2×5 + 3×6 = 4 + 10 + 18 = 32

The dot product has a geometric meaning that makes it the measure of similarity used throughout AI:

a · b = |a| × |b| × cos θθ (theta) is the angle between the two arrows. cos θ is 1 when they point the same way, 0 at right angles and −1 when opposite.

Cosine similarity

Rearranging gives the cosine similarity, which ignores length and keeps only direction:

cos θ = a · b|a| |b|

Worked with a = (3, 1) and b = (1, 2):

  1. Dot product: 3×1 + 1×2 = 5.
  2. Lengths: |a| = √10 = 3.162, |b| = √5 = 2.236.
  3. Cosine: 5 / (3.162 × 2.236) = 5 / 7.071 = 0.707, which is the cosine of 45°.

If both vectors are already unit length, the denominator is 1 and the cosine is just the dot product. That shortcut is why search systems normalise their vectors.

Try it

Drag two vectors

Drag the coloured dots. Watch the cosine fall from 1 to 0 as the arrows move to right angles, and to −1 when they point in opposite directions. Make one arrow longer without turning it: the dot product changes but the cosine does not.

Try it

Find the cosine similarity of (1, 0, 1) and (1, 1, 0).

Answer

Dot product 1 + 0 + 0 = 1. Both lengths are √2. Cosine = 1 / (√2 × √2) = 1/2 = 0.5, an angle of 60°.

Matrices: tables of numbers

A matrix is a rectangle of numbers with rows and columns. Its shape is written rows × columns, so

W = 2013a 2 × 2 matrix. W21, the entry in row 2, column 1, is 1.

A matrix times a vector

Each row of the matrix takes a dot product with the vector, and each result becomes one entry of the output. With W above and x = (1, 2):

  • Row 1: (2, 0) · (1, 2) = 2 + 0 = 2.
  • Row 2: (1, 3) · (1, 2) = 1 + 6 = 7.

So Wx = (2, 7). A 2 × 2 matrix turns a 2-vector into another 2-vector. In general, an m × n matrix turns an n-vector into an m-vector: it can change the dimension.

Seen as arrows, a matrix moves every point of the plane: it can stretch, squash, rotate or flip space. The matrix with 1s on the diagonal and 0s elsewhere, the identity, leaves every vector where it was.

Matrix times matrix

To multiply two matrices, take every row of the first with every column of the second. Worked by hand:

123456 × 789101112 = 5864139154
  • Row 1, column 1: 1×7 + 2×9 + 3×11 = 7 + 18 + 33 = 58.
  • Row 1, column 2: 1×8 + 2×10 + 3×12 = 8 + 20 + 36 = 64.
  • Row 2, column 1: 4×7 + 5×9 + 6×11 = 28 + 45 + 66 = 139.
  • Row 2, column 2: 4×8 + 5×10 + 6×12 = 32 + 50 + 72 = 154.

The shape rule: (m × n) times (n × p) gives m × p. The inner numbers must match, because each row of the first needs exactly one partner for each entry of a column of the second. A 2 × 3 times a 3 × 2 is fine; a 2 × 3 times a 2 × 3 is not defined. Order matters: in general AB is not BA.

Try it

Step through a matrix multiply

Press Next cell to compute C one entry at a time. The highlighted row of A and column of B are the two lists being dotted. Edit any entry of A or B and step again.

Try it

Multiply 1201 by 3012. Then multiply them in the other order.

Answer

First order: row 1 gives (1×3 + 2×1, 1×0 + 2×2) = (5, 4); row 2 gives (0×3 + 1×1, 0×0 + 1×2) = (1, 2). Other order: row 1 gives (3×1 + 0×0, 3×2 + 0×1) = (3, 6); row 2 gives (1×1 + 2×0, 1×2 + 2×1) = (1, 4). Different results, so order matters.

Transpose

The transpose AT swaps rows and columns: a 2 × 3 becomes a 3 × 2. You will see it in attention (chapter 18), where QKT computes the dot product of every query with every key in one multiplication.

Counting the cost

Each output cell of an (m × n) × (n × p) product needs n multiplications, and there are m × p cells, so mnp multiplications in all. For the worked example that is 2 × 3 × 2 = 12. For one 4,096 × 4,096 matrix times a 4,096-vector it is about 16.8 million. That is why AI runs on GPUs, which do thousands of multiplications at once.

In AI

  • A layer of a neural network computes y = Wx + b: a matrix times the input, plus a bias vector (chapter 12). Each row of W is one neuron's weights.
  • Processing a batch of inputs at once stacks them as rows of a matrix, so many matrix-vector products become one matrix-matrix product.
  • An embedding is a vector that stands for a piece of text; texts with similar meaning get vectors with high cosine similarity (chapter 23).
  • A tensor is the same idea with more than two index directions, for example a batch of sequences of vectors.

Embedding models produce vectors of a fixed length: 384 numbers for a small open model, 1,024 or more for larger ones. A vector database stores each passage's vector next to its text and ranks passages by cosine distance, which is 1 minus the cosine similarity. Many systems report 1 - distance as the score, so the score you see in a retrieval trace is exactly the cosine similarity from this chapter.

Build it

In Lab 3 you embed a small document set and rank its passages by cosine similarity yourself, then use the best ones to answer questions.

Self-check

What shape is the product of a 4 × 3 matrix and a 3 × 5 matrix?

4 × 5. The inner 3s match and disappear.

Two unit vectors have dot product 0.9. What is their cosine similarity?

0.9. For unit vectors the lengths are 1, so the cosine equals the dot product.

Why does cosine similarity suit comparing texts better than the raw dot product?

It ignores length and compares only direction. Vector length can reflect things like text length rather than meaning, so removing it makes the comparison about what the text is about.

How many multiplications does a 384 × 384 matrix times a 384-vector need?

384 × 384 = 147,456: one per entry of the matrix.

Further reading

Free, checked on 2026-10-21. All resources for this chapter.