The training MSE decreases as model flexibility increases.

The training MSE decreases as model flexibility increases.

• A. True
• B. False
• C. Depends on data
• D. Not necessarily

Answer: (A)

Increasing model flexibility expands the set of functions the model can represent. When you train, you minimize the mean squared error on the training data over all those possible functions. Because the larger class includes the smaller one, the minimum training error cannot be worse than before and is often lower, as you can fit the training data more closely with the extra parameters. So the training MSE tends to decrease as you allow more complexity. Of course, this can raise overfitting risk and hurt performance on new data, but for training error alone, more flexibility typically reduces it.

Increasing model flexibility expands the set of functions the model can represent. When you train, you minimize the mean squared error on the training data over all those possible functions. Because the larger class includes the smaller one, the minimum training error cannot be worse than before and is often lower, as you can fit the training data more closely with the extra parameters. So the training MSE tends to decrease as you allow more complexity. Of course, this can raise overfitting risk and hurt performance on new data, but for training error alone, more flexibility typically reduces it.