The p-value gives the probability of observing a result as extreme as given the null hypothesis is true. A low p-value indicates a low probability of observing the data if the null hypothesis were true. Reject the null hypothesis when the p-value is less than $\alpha = 1 - \text{the confidence level}$. The smaller the p-value in relation to $\alpha$ the stronger the evidence for rejecting the null hypothesis. The p-value can be calculated as the probability of observing the test statistic given the null hypothesis is true. For a hypothesis test of the mean $\mu$ from a [[normal distribution]], the p-value can be calculated as $\begin{align} \text{p-value} &= P(\bar X > \bar x) \\ &= P(\frac{\bar X - \mu_0}{\sigma/\sqrt{n}} > \frac{\bar x - \mu_0}{\sigma/\sqrt{n}}) \\ &= P(Z > \frac{\bar x - \mu_0}{\sigma/\sqrt{n}}) \end{align} $ The p-value is the area to the right (in this case) of the test statistic $\bar X$. Use [[R]] to find the area under the corresponding tail. ```R pnorm(test_statistic, lower.tail=FALSE) ``` ## Interpretation of a p-value Consider a coin. You flip 3 times (n=3), and all results are heads. You suspect an unfair coin. The null hypothesis would be that the coin is in fact fair. What is the probability you would see three heads in a row if the coin were fair (i.e., the null hypothesis is true)? Pr(heads, heads, heads) = .5 * .5 * .5 = .125, which is also your p-value. Is that enough evidence to declare the game fixed? It is the discretion of the researcher, and the researcher's audience, to determine if a p-value of .125 is sufficient to draw that conclusion. Often, a fixed value of .05 is used to declare results 'statistically significant' and thus publishable.  ## History [John Tukey](https://en.wikipedia.org/wiki/John_Tukey) [AT&T Bell Labs](https://en.wikipedia.org/wiki/Bell_Labs)