Consider a simple linear regression where the real model is given by $y_i=\beta_1+\beta_2 x_{i} + \epsilon_i$ where $(\epsilon_i)$ are iid, centered, with variance $\sigma^2$. Which of the following statements are always true?

• 1. Assume that $R^2=1$ in a simple linear regression, it means that all the $(x_i,y_i)$ are on the same line.
• 2. Even if the adjusted coefficient of determination $R_a^2=1$, it may happen sometimes that $Y \neq \hat Y$.
• 3. The adjusted coefficient of determination may be negative.

Consider a multiple linear regression model $Y=X\beta +\epsilon$ where $\mathrm{rank}(X)=p$ and $\PE[\epsilon]=0$ and $\Var(\epsilon)=\sigma^2 I$ are satisfied. We write $X=[X_1|\ldots|X_p]$. Denote by $\hat \beta$ the OLS estimator of $\beta$ obtained from the regression of $Y$ on $\{X\beta,\ \beta \in \rset^p\}$. Define $P_X=X(X'X)^{-1}X'=[h_{i,j}]_{1\leq i, j\leq n}$. Which of the following statements are true?

• 4. We always have $\sum_{i=1}^n \hat \epsilon_i=0$, even when none of the vectors $X_i$ is equal to the vector $\mathsf{1}$.
• 5. $\hat Y$ is always orthogonal to $\hat \epsilon$, even when none of the vectors $X_i$ is equal to the vector $\mathsf{1}$.
• 6. Define $\tilde \beta \in \argmin_{\beta} \sum_{i=1}^n |Y_i-x'_i \beta|$. Then the SSE associated with $\hat \beta$ is not always smaller that the SSE associated to $\tilde \beta$.