The Bayes classifier is a type of [[classification]] model that minimizes the Bayes error rate. The Bayes classifier creates the Bayes decision boundary that splits the predictor space between classes. ## Bayes error rate The Bayes error rate is given by one minus the maximum over all $j$s of the probability that $Y=j$ conditional on the predictor values $X$. $1 - E \Big [ \overset{max}{j} P(Y=j | X) \Big]$ However, to get the true probability, we must know the distribution of $Y$. [[k-nearest neighbors regression]] is an approach for approximating the Bayes classifier. ## naive Bayesian classifier The "naive" assumption is that the attributes are independent. While rarely true in practice, the assumption is fairly weak and the classifier still works well. Use [[additive smoothing]] to add 1 to each case when there are zero values. ## Bayesian belief network A Bayesian belief network is a probabilistic graphical model using a [[directed acyclic graph]] to model conditional probabilities.