In least squares LOOCV, the LOOCV error can be computed from a single fitted model using residuals and leverage.

In least squares LOOCV, the LOOCV error can be computed from a single fitted model using residuals and leverage.

• A. True
• B. False
• C. Only for ridge regression
• D. Only if residuals are zero

Answer: (A)

In least squares LOOCV, you can compute the leave-one-out error from a single fitted model using the residuals and the leverages because of how removing a point affects the fit in linear models. Let e_i be the residual for observation i from the full model, and h_ii be the i-th diagonal element of the hat matrix H = X(X'X)^{-1}X'. The prediction you would get for observation i if you left it out is y_hat(-i) = y_hat_i − e_i/(1 − h_ii). Therefore the LOOCV error for that observation is (e_i/(1 − h_ii))^2. Summing these across all observations gives the LOOCV error, which can be computed from the residuals and leverages of the single fitted model without refitting. The factor 1/(1 − h_ii) shows how high-leverage points (large h_ii) have a bigger impact on LOOCV.

In least squares LOOCV, you can compute the leave-one-out error from a single fitted model using the residuals and the leverages because of how removing a point affects the fit in linear models. Let e_i be the residual for observation i from the full model, and h_ii be the i-th diagonal element of the hat matrix H = X(X'X)^{-1}X'. The prediction you would get for observation i if you left it out is y_hat(-i) = y_hat_i − e_i/(1 − h_ii). Therefore the LOOCV error for that observation is (e_i/(1 − h_ii))^2. Summing these across all observations gives the LOOCV error, which can be computed from the residuals and leverages of the single fitted model without refitting. The factor 1/(1 − h_ii) shows how high-leverage points (large h_ii) have a bigger impact on LOOCV.