ZestCalc

Explainer

Data as Vectors and Matrices

Before a model can learn anything, every example has to become a list of numbers. See a table, an image, and a sentence turn into matrices, and why a whole batch goes through a model as one matrix product.

An example is a vector

A model never sees a house, a photo, or a sentence. It sees numbers. So the first step in any machine learning project is deciding which numbers describe one example. For a house, that might be its area, its number of bedrooms, and its age. Those three numbers, in a fixed order, form a feature vector:

x=[120312]x = \begin{bmatrix} 120 & 3 & 12 \end{bmatrix}

Stack one feature vector per example as the rows of a matrix and you have the dataset. Pick a row or a column below to see both ways of reading it.

85230310
120312455
64145220
15045610
98320380

House 2 is one example: the row vector [120312]\begin{bmatrix}120 & 3 & 12\end{bmatrix}, a single point in 3-dimensional feature space.

Together the features form X, a 5 × 3 matrix: 5 examples, 3 features each.

Each row is one house and each column is one feature. Price is the value to predict, so it's kept apart from the features.

With nn examples and dd features, the data matrix XX is n×dn \times d. The value to predict, here the price, is kept apart as a vector yy with nn entries. Almost every formula in machine learning reads XX one of two ways:

  • A row is one example, a single point in dd-dimensional space. Models make one prediction per row.
  • A column is one feature measured across every example. Statistics such as a feature's mean and spread are computed down the columns, which is why XTX^{T}, the Matrix Transpose Calculator, turns up so often: XTXX^{T}X compares every feature with every other.

An image is a matrix

A grayscale image is already a grid of numbers, one brightness value per pixel. This one is 12×1212 \times 12, small enough to read every entry.

Row 3, column 6 holds 235. Read row after row into one long vector, this pixel is entry 30 of 144.

Row 3 as a vector

[0015023523523523523523515000]\begin{bmatrix}0 & 0 & 150 & 235 & 235 & 235 & 235 & 235 & 235 & 150 & 0 & 0\end{bmatrix}
A 12 × 12 image of the digit 7, the kind of picture a digit classifier reads. Every pixel is a number from 0 (blank) to 255 (full ink).

A simple digit classifier doesn't keep the grid. It reads the pixels row after row into one long vector of 12×12=14412 \times 12 = 144 entries, so the image becomes one row of the data matrix, just like a house. A color image has three such grids, one each for red, green, and blue.

Flattening has a cost: in the long vector, a pixel's neighbors above and below are 12 entries away, so the model has to relearn which pixels sit next to each other. Image models such as convolutional networks keep the grid instead. They're still built from the same multiply-and-add operations, applied to small patches at a time.

A sentence is a matrix of word vectors

Words aren't numbers, so language models give each one a vector. The text is split into tokens, and each token's vector, its embedding, is looked up in a large table that the model learns during training.

“cat” becomes the vector [0.80.4−0.50.1]\begin{bmatrix}0.8 & 0.4 & -0.5 & 0.1\end{bmatrix}.

Four tokens, four dimensions each: the sentence is a 4 × 4 matrix.

Each token is looked up in a table of learned vectors. These are made up and only four wide; real models use hundreds or thousands of dimensions.

A sentence of nn tokens becomes an n×dn \times d matrix with one row per token, the same shape as a table of examples. Training pushes tokens that are used in similar ways toward vectors that point in similar directions, so the angle between two embeddings says how related they are. That angle comes from the dot product, which the Dot Product and Angle Calculator works through one entry at a time.

A whole batch at once

A linear model scores one example by weighting its features and adding them up, which is a dot product of the feature vector xx with a weight vector ww. Scoring every example at once is then one matrix-vector product:

y^=Xw\hat{y} = Xw

Most models produce more than one number per example: a score for each of kk classes, or kk values handed to the next layer of a neural network. The weights become a d×kd \times k matrix WW, and the whole batch goes through in a single product.

Examples n4
Features d3
Outputs k2
X4×3  W3×2=Y4×2\underset{4 \times 3}{X}\;\underset{3 \times 2}{W} = \underset{4 \times 2}{Y}
W
X

Entry (1, 1) of Y is row 1 of X dotted with column 1 of W: 3 products added together.

One product handles the whole batch. Each entry of Y is one row of X dotted with one column of W.
Y=XW,(n×d)(d×k)→n×kY = XW, \qquad (n \times d)(d \times k) \to n \times k

Each entry of YY is one row of XX dotted with one column of WW. The inner sizes have to match, the shape rule from the Matrix Multiplication Calculator. A model built for dd features can't take examples with a different number, and a mismatched shape is the most common error in machine learning code.

Writing models this way isn't only tidy. Matrix products are exactly what graphics cards are built to compute quickly, so stating a model as products of matrices is what makes training on millions of examples practical.