Guide to Statistics: Probability & Statistics Facts, Formulae and Information

Print this page

Simple Linear Regression

To fit the straight line $$y=\alpha +\beta x$$ to data $$(x_i,\,y_i),~i=1, 2, \ldots, n~$$ by the method of least squares the estimates of slope, $$\beta$$, and intercept, $$\alpha$$, are given by: \[b=\frac{\sum{x_iy_i}-\frac{1}{n}\left(\sum{x_i}\sum{y_i}\right)}{\sum{x_i^2}-\frac{1}{n}\left(\sum{x_i}\right)^2} = \frac{s_{xy}}{s_{xx}},~~~a=\bar{y}-b\bar{x}\] If we assume that the $$x_i$$ are known and that they have normal distributions with means $$\alpha +\beta x_i$$, and constant variance $$\sigma^2$$, written as $$y_i\sim N\left(\alpha +\beta x_i,\,\sigma^2\right)$$, then if $$x_0$$ is a fixed value \[b\sim N\left(\beta,\,\frac{\sigma^2}{s_{xx}}\right)\] \[a\sim N\left(\alpha ,\,\sigma^2\left\{\frac{1}{n}+\frac{\bar{x}^2}{s_{xx}}\right\}\right)\] \[a+bx_0\sim N\left(\alpha +\beta x_0,\,\sigma^2\left\{\frac{1}{n}+\frac{\left(x_0-\bar{x}\right)^2}{s_{xx}}\right\}\right)\] A common alternative is to use $$\hat{\alpha}$$ for a and $$\hat{\beta}$$ for b.

contents

close this window