ZestCalc

Least Squares Calculator

Find the best fit when no exact answer exists, for a line through points or for any Ax ≈ b. See what it minimizes, as real squares you can drag around.

Points

Enter each point as x and y, up to 30 points, or start from an example.

P1=P_{1} =
P2=P_{2} =
P3=P_{3} =
P4=P_{4} =
P5=P_{5} =

Result

Updates as you type.

Best-fit line
y=45x+15y = \frac{4}{5}x + \frac{1}{5}
Total squared error
45\frac{4}{5}
R²
89\frac{8}{9}

What least squares means

When a system has more equations than unknowns, there's usually no exact answer. Least squares finds the x^\hat{x} that comes closest, by making the total of the squared errors as small as possible:

min⁡x ∣Ax−b∣2\min_x \ |Ax - b|^2

The best-known case is fitting a line. Each point (xi,yi)(x_i, y_i) asks for mxi+c=yim x_i + c = y_i, and with more than two points these rarely all hold. The best-fit line is the one whose squared vertical distances add up to the least.

Squaring the errors keeps positive and negative errors from canceling out, and it gives a clean formula. It also means that one point far from the others can pull the whole line toward it.

The normal equations

Least squares has a geometric meaning. Everything AA can reach is its column space, and if bb isn't in it, the closest reachable point is the projection of bb. The leftover residual r=b−Ax^r = b - A\hat{x} is then perpendicular to every column of AA, so ATr=0A^T r = 0. Rearranging that gives the normal equations:

ATA x^=ATbA^T A\, \hat{x} = A^T b

This is a square system that can always be solved. If the columns of AA are independent, the answer is unique. If they aren't, as the Matrix Rank Calculator can show, many x^\hat{x} share the same smallest error, and this calculator shows the one that sets the free unknowns to 0.

Reading the result

For a line fit, R2R^2 measures how much of the spread in yy the line explains:

R2=1−∑(yi−y^i)2∑(yi−yˉ)2R^2 = 1 - \frac{\sum (y_i - \hat{y}_i)^2}{\sum (y_i - \bar{y})^2}

A value of 1 means that the line passes through every point. A value near 0 means that the line does little better than a flat line at the average height.

This calculator solves the normal equations in exact fractions, so rounding can't affect it. Programs that work with rounded numbers usually avoid them, because ATAA^T A makes rounding errors worse, and use the QR factors from the Gram-Schmidt and QR Calculator instead. The projection of a single vector, the simplest case of all this, is on the Vector Projection Calculator.