A matrix is a sum of rank-one pieces
The singular value decomposition writes any matrix as , which the SVD Calculator computes for small matrices. Multiplied out, that is a sum of simple pieces:
Each piece is a column times a row , scaled by a singular value . A column times a row has rank one: every row of it is a multiple of the same row. The number of pieces is the rank of , which the Matrix Rank Calculator finds. The singular values come sorted, , so the first pieces are the biggest.
Keep the biggest pieces
Stop the sum after pieces and you get , a matrix of rank . The Eckart–Young theorem says no other rank- matrix is closer to , measuring distance as the square root of the summed squared differences (the Frobenius norm). The error is set exactly by the pieces you dropped:
The sum of the squared singular values is often called the matrix's energy. When the singular values fall off fast, a few pieces hold nearly all of it.
Storing means storing columns of , columns of , and singular values: numbers instead of . For a 96 × 96 image at rank 8 that's 1,544 numbers instead of 9,216.
What compresses well
Low rank means the rows are built from a few shared patterns. Smooth gradients, flat regions, and anything lined up with the rows and columns compress well. The plaid is a row pattern plus a column pattern, so it has rank 2. A diagonal edge is the opposite: each row is the previous one shifted by a pixel, and no small set of shared patterns draws it, so its singular values fall off slowly.
Real image formats such as JPEG use other transforms that do better on photographs. The SVD's strength is generality: it finds the best low-rank summary of *any* table of numbers, and in machine learning that's usually what you want.
Ratings and recommendations
A table of who rated what is a matrix too. People's tastes aren't random: someone who loves one action film probably likes the next. So the ratings matrix is close to low rank, and its top pieces read as hidden factors, here roughly "likes films at all" and "action or romance".
Recommender systems use this the other way around. Most of a real ratings table is empty, and fitting a low-rank matrix to the ratings that do exist fills in the blanks with predictions. That's matrix factorization, the method made famous by the Netflix Prize.
LoRA: fine-tuning with a low-rank update
A large language model has weight matrices with thousands of rows and columns. Fine-tuning it for a new task changes each weight matrix by some update . LoRA (low-rank adaptation) bets that this update is close to low rank, and learns it directly as a product of two thin matrices:
With much smaller than , that's numbers to train instead of . The original weights stay frozen, and the small update can be stored, swapped, or merged back into .
A full update has d² = 16,777,216 weights. LoRA with rank 8 learns 2dr = 65,536, which is 0.39% as many.
The same truncation is also how PCA reduces dimensions: the PCA Calculator keeps the top components, which are the top right singular vectors of the centered data.