Machine learning through linear algebra
Most of machine learning is a handful of linear algebra ideas used at scale. Each page here takes one of them, shows it working, and links to the linear algebra it rests on.
1.Data is linear algebra
An example is a vector, a dataset is a matrix, and comparing two examples is a dot product.
- ExplainerData as Vectors and MatricesHow tables, images, and sentences become matrices, and why a whole batch goes through a model as one product XW.
- CalculatorCosine Similarity CalculatorCompare up to eight vectors by the angle between them, with exact cosines, a similarity grid, and the closest match for each. Includes toy word vectors and king − man + woman.
- CalculatorCovariance Matrix CalculatorPaste up to 100 samples of 6 features and get the means, centered data, exact covariance matrix, and correlation matrix, with a draggable scatter and covariance ellipse.
2.Learning from data
Fitting a model means solving for the best weights or walking downhill toward them, and eigenvalues decide how hard that is.
- CalculatorLinear Regression CalculatorFit a linear model to up to 6 features with an intercept. Exact coefficients from the normal equations, predictions, residuals, MSE, R², and optional ridge regularization.
- ExplainerGradient DescentTraining as walking downhill on a loss bowl: the learning rate limit 2/λ_max, zig-zags in narrow bowls, momentum, and why standardizing features helps.
- CalculatorPCA CalculatorPrincipal component analysis of up to 100 samples and 6 features: covariance or correlation matrix, eigenvalues, loadings, explained variance, and the data projected onto the kept components.
- ExplainerLow-Rank ApproximationA matrix as a sum of rank-one pieces σuvᵀ: compress an image with its top singular values, explain ratings with two tastes, and see why LoRA updates are cheap.
3.Neural networks
A layer is a matrix with a bend after it, training runs the same matrices backward, and attention turns dot products into weights.
- ExplainerNeural Network LayersA layer is σ(Wx + b): a matrix, a shift, and a bend. One neuron as a decision line, a plane carried through a layer, and two layers untangling two moons.
- ExplainerBackpropagation in Matrix FormHow a network gets the gradient for every weight in one backward pass: the error flows back through Wᵀ, weight gradients are outer products δxᵀ, and a batch is one product.
- CalculatorAttention CalculatorScaled dot-product attention for up to 8 tokens: exact scores QKᵀ, scaling by √d_k, softmax weights, an optional causal mask, and each output as a weighted average of the values.