Simple Linear Regression is the method for finding the "line of best fit" between the dependent variable, y, and the independent variable, x.

Simple: only one independent variable

Linear in the Independent Variable: the independent variable only appears to the first power.

Linear: also means linear in the parameters, since no parameter appears to the first power.

The Least Squares Regression Line is the line which minimizes the sum of the square or the error of the data points.  It is an averaging line of the data.  (See the graph below.) 

Regression analysis tries to fit a model to one dependent variable based on one or more independent variables.  In most cases, there will be error.  See the graph below for an example of Simple Linear Regression.

Notice that the graph to the right shows several features:
  • The actual data points (x,y) are the blue dots.
  • The Least Squares Regression Line of the dependent (y) variable based on the independent (x) variable is shown in black.  
  • The errors (residuals) are the vertical distances between the observed values of y and the predictions of the "line of best fit," which are shown in red.

The goal, in general, is to minimize the errors from the actual data to the regression line.  The least squares line minimizes the sum of the square of the errors.
 

 

Regression in general can be used for 3 main purposes:

 

This Simple Linear Regression Tutorial shows:

The Regression Diagnostics Tutorial shows when it is appropriate to use this method.

First, let's look at an example to see the output that should be expected. 

Regression Tutorial Menu    Dictionary

STATS @ MTSU