Bias-variance tradeoff describes the tradeoff statisticians must make between a model with low [[variance]] and one with low [[bias]]. Intuitively, variance is the amount by which our estimates would change if we train on a different training dataset. Bias refers to the error introduced by simplifying a real world problem to a tractable statistical model. We can assume that bias is related to a given model's failure to fit the **training data**. Variance is associated with the model's inability to fit the **testing data**. Consider that the [[mean squared error]] can be decomposed to the variance of the estimator plus the bias of the estimator. Both are non-negative terms, and so for our MSE, which is assumed to be fixed, if variance decreases, bias must increase. The more flexible the model, the more variance you can expect, which implies lower bias. $E \Big [ (Y_0 - \hat f(X_0)^2) \Big] = \text{Var} \Big [ \hat f(X_0) \Big] + \text{Bias} \Big[ \hat f(X_0)) \Big]^2 + \text{Var} (\epsilon)$ where $Y_0$ and $X_0$ represent test data (as opposed to training data). Bias is analogous to [[precision]] and variance to [[accuracy]].