The activation function determines the value that is passed to the next layer in the neural network. Critically, it is a nonlinear function of the inputs (which include all of the activations in the previous layer multiplied by their weights). Common activation functions include - [[sigmoid]] - [[rectified linear unit]] - [[tanh]]