Index: The Book of Statistical ProofsStatistical Models ▷ Univariate normal data ▷ Simple linear regression ▷ Definition

Definition: Let $y$ and $x$ be two $n \times 1$ vectors.

Then, a statement asserting a linear relationship between $x$ and $y$

$\label{eq:slr-model} y = \beta_0 + \beta_1 x + \varepsilon \; ,$

together with a statement asserting a normal distribution for $\varepsilon$

$\label{eq:slr-noise} \varepsilon \sim \mathcal{N}(0, \sigma^2 V)$

is called a univariate simple regression model or simply, “simple linear regression”.

• $y$ is called “dependent variable”, “measured data” or “signal”;

• $x$ is called “independent variable”, “predictor” or “covariate”;

• $V$ is called “covariance matrix” or “covariance structure”;

• $\beta_1$ is called “slope of the regression line”;

• $\beta_0$ is called “intercept of the regression line”;

• $\varepsilon$ is called “noise”, “errors” or “error terms”;

• $\sigma^2$ is called “noise variance” or “error variance”;

• $n$ is the number of observations.

When the covariance structure $V$ is equal to the $n \times n$ identity matrix, this is called simple linear regression with independent and identically distributed (i.i.d.) observations:

$\label{eq:mlr-noise-iid} V = I_n \quad \Rightarrow \quad \varepsilon \sim \mathcal{N}(0, \sigma^2 I_n) \quad \Rightarrow \quad \varepsilon_i \overset{\text{i.i.d.}}{\sim} \mathcal{N}(0, \sigma^2) \; .$

In this case, the linear regression model can also be written as

$\label{eq:slr-model-sum} y_i = \beta_0 + \beta_1 x_i + \varepsilon_i, \; \varepsilon_i \sim \mathcal{N}(0, \sigma^2) \; .$

Otherwise, it is called simple linear regression with correlated observations.

Sources:

Metadata: ID: D163 | shortcut: slr | author: JoramSoch | date: 2021-10-27, 07:07.