Fitting by minimising the sum of squared residuals — the special case of maximum likelihood when errors are Gaussian and equal.Least squares fitting is a mathematical procedure for finding the best-fitting curve or line through a set of data points. The 'best' fit is defined as the one that minimizes the sum of the squares of the vertical distances (residuals) between the data points and the fitted curve. This approach is ubiquitous in science and engineering because it provides a principled way to extract parameters from noisy measurements, from calibrating instruments to testing physical theories.
The method was first published by Adrien-Marie Legendre in 1805, and Carl Friedrich Gauss claimed to have used it earlier, around 1795, for astronomical calculations.
For a given model (e.g., a straight line y = mx + b), you compute the residual for each data point: the difference between the observed value and the model's prediction. Squaring each residual removes the sign, penalizes large deviations more heavily, and makes the sum differentiable. The parameters (m and b) are then adjusted using calculus (or numerical methods) until the sum of squared residuals reaches a minimum. This minimum corresponds to the most likely parameter values if the measurement errors are independent and normally distributed with equal variance.
Used in virtually every field of quantitative science: fitting calibration curves in chemistry, determining Hubble's constant from supernova data, and estimating parameters in machine learning regression models.
The name 'least squares' comes from the fact that the solution minimizes the sum of the squared residuals, not the sum of the absolute residuals, which would be more robust to outliers but is harder to compute analytically.





