Skip to main content
Edlintics

Ordinary least squares

Ordinary least squares is the standard method for fitting a regression line, choosing coefficients that minimise the sum of squared residuals.

Also known asOLS

ByHoang TruongUpdated

FrameworkOrdinary least squares (OLS)

What it is

See it move

Loading infographic...

The infographic is a scatter plot with a fitted regression line, showing advertising spend on the horizontal axis and profit on the vertical axis. OLS selects the line by squaring each vertical gap between every observed profit value Y and its predicted value ŷ, then choosing the line that makes the total Σ(Y − ŷ)² as small as possible.

The formula

LaTeX
min⁡β^0, β^1∑i=1n(yi−β^0−β^1xi)2\min_{\hat{\beta}_0,\,\hat{\beta}_1} \sum_{i=1}^n (y_i - \hat{\beta}_0 - \hat{\beta}_1 x_i)^2

Variables

Sum of squared residuals — the quantity OLS minimises
Observed outcome for observation i
OLS estimate of the intercept
OLS estimate of the slope
Observed explanatory variable value for observation i

Squaring residuals penalises large errors more heavily and stops positive and negative gaps from cancelling each other.

LaTeX
β^1=∑(xi−xˉ)(yi−yˉ)∑(xi−xˉ)2\hat{\beta}_1 = \frac{\sum(x_i - \bar{x})(y_i - \bar{y})}{\sum(x_i - \bar{x})^2}

Variables

OLS slope estimate
Explanatory variable value for observation i
Sample mean of the explanatory variable
Outcome variable value for observation i
Sample mean of the outcome variable

The OLS slope equals the sample covariance of x and y divided by the sample variance of x.

LaTeX
β^0=yˉ−β^1xˉ\hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x}

Variables

OLS intercept estimate
Sample mean of the outcome variable
OLS slope estimate
Sample mean of the explanatory variable

The fitted line always passes through the point (x̄, ȳ).

Ordinary Least Squares (OLS) — Regression Method