P-values are part of a method in science called ‘hypothesis-testing’. Hypothesis testing involves defining a null-hypothesis: a statement that would be true if nothing happened in your experiment. For example, imagine you want to test whether people who ate chocolate were heavier than those who didn’t: the null hypothesis would be that there is no difference in weight between people eating and not eating chocolate.

We’d also state an alternative hypothesis. This reflects what we think might be going on: for example, the alternative hypothesis would be that there is a difference in weight between people eating and not eating chocolate.

The p-value provides evidence against the null hypothesis. If it’s low, such as below 0.05, then it’s quite unlikely that the null hypothesis is correct, so we reject it. 

But P-values are really hard to understand, and even leading scientists struggle to explain what they mean. They tell you the probability of observing data at least as extreme as the data you observed in an experiment, assuming that the null hypothesis is true. It’s easy to think that they also tell you the probability that the null hypothesis is true, but that’s not quite right, because the probability we calculate assumes that the null hypothesis is true.

Some people think that the widespread use of and misunderstanding about p-values leads to problems with the reproducibility of scientific findings. You can learn more about problems with reproducibility in science in this video.

Don’t lose faith in science though – it’s one of the best ways that we have to learn about and improve the world. And partly because it’s so important, there is a new field called ‘metascience’, that aims to better understand and improve the way that science is done.

Next Section »