An example is a vector
A model never sees a house, a photo, or a sentence. It sees numbers. So the first step in any machine learning project is deciding which numbers describe one example. For a house, that might be its area, its number of bedrooms, and its age. Those three numbers, in a fixed order, form a feature vector:
Stack one feature vector per example as the rows of a matrix and you have the dataset. Pick a row or a column below to see both ways of reading it.
House 2 is one example: the row vector , a single point in 3-dimensional feature space.
Together the features form X, a 5 × 3 matrix: 5 examples, 3 features each.
With examples and features, the data matrix is . The value to predict, here the price, is kept apart as a vector with entries. Almost every formula in machine learning reads one of two ways:
- A row is one example, a single point in -dimensional space. Models make one prediction per row.
- A column is one feature measured across every example. Statistics such as a feature's mean and spread are computed down the columns, which is why , the Matrix Transpose Calculator, turns up so often: compares every feature with every other.
An image is a matrix
A grayscale image is already a grid of numbers, one brightness value per pixel. This one is , small enough to read every entry.
Row 3, column 6 holds 235. Read row after row into one long vector, this pixel is entry 30 of 144.
Row 3 as a vector
A simple digit classifier doesn't keep the grid. It reads the pixels row after row into one long vector of entries, so the image becomes one row of the data matrix, just like a house. A color image has three such grids, one each for red, green, and blue.
Flattening has a cost: in the long vector, a pixel's neighbors above and below are 12 entries away, so the model has to relearn which pixels sit next to each other. Image models such as convolutional networks keep the grid instead. They're still built from the same multiply-and-add operations, applied to small patches at a time.
A sentence is a matrix of word vectors
Words aren't numbers, so language models give each one a vector. The text is split into tokens, and each token's vector, its embedding, is looked up in a large table that the model learns during training.
“cat” becomes the vector .
Four tokens, four dimensions each: the sentence is a 4 × 4 matrix.
A sentence of tokens becomes an matrix with one row per token, the same shape as a table of examples. Training pushes tokens that are used in similar ways toward vectors that point in similar directions, so the angle between two embeddings says how related they are. That angle comes from the dot product, which the Dot Product and Angle Calculator works through one entry at a time.
A whole batch at once
A linear model scores one example by weighting its features and adding them up, which is a dot product of the feature vector with a weight vector . Scoring every example at once is then one matrix-vector product:
Most models produce more than one number per example: a score for each of classes, or values handed to the next layer of a neural network. The weights become a matrix , and the whole batch goes through in a single product.
Entry (1, 1) of Y is row 1 of X dotted with column 1 of W: 3 products added together.
Each entry of is one row of dotted with one column of . The inner sizes have to match, the shape rule from the Matrix Multiplication Calculator. A model built for features can't take examples with a different number, and a mismatched shape is the most common error in machine learning code.
Writing models this way isn't only tidy. Matrix products are exactly what graphics cards are built to compute quickly, so stating a model as products of matrices is what makes training on millions of examples practical.