The method that can "prove" almost anything - James A. Smith
830,301 Views
3,203 Questions Answered
Let’s Begin…
In 2011, a group of researchers conducted a study designed to find an impossible result. Their study involved real people, truthfully reported data, and commonplace statistical analyses. So how did they do it? The answer lies in a statistical method scientists often use to try to figure out whether their results mean something, or if they’re random noise. James A. Smith explores p-hacking.
Additional Resources for you to Explore
We’d also state an alternative hypothesis. This reflects what we think might be going on: for example, the alternative hypothesis would be that there is a difference in weight between people eating and not eating chocolate.
The p-value provides evidence against the null hypothesis. If it’s low, such as below 0.05, then it’s quite unlikely that the null hypothesis is correct, so we reject it.
But P-values are really hard to understand, and even leading scientists struggle to explain what they mean. They tell you the probability of observing data at least as extreme as the data you observed in an experiment, assuming that the null hypothesis is true. It’s easy to think that they also tell you the probability that the null hypothesis is true, but that’s not quite right, because the probability we calculate assumes that the null hypothesis is true.
Some people think that the widespread use of and misunderstanding about p-values leads to problems with the reproducibility of scientific findings. You can learn more about problems with reproducibility in science in this video.
Don’t lose faith in science though – it’s one of the best ways that we have to learn about and improve the world. And partly because it’s so important, there is a new field called ‘metascience’, that aims to better understand and improve the way that science is done.
About TED-Ed Animations
TED-Ed Animations feature the words and ideas of educators brought to life by professional animators. Are you an educator or animator interested in creating a TED-Ed Animation? Nominate yourself here »
Meet The Creators
- Educator James A. Smith
- Director Anton Bogaty
- Narrator Addison Anderson
- Music Salil Bhayani, cAMP Studio
- Sound Designer Amanda P.H. Bennett
- Director of Production Gerta Xhelo
- Editorial Director Alex Rosenthal
- Producer Bethany Cutmore-Scott
- Editorial Producer Elizabeth Cox
- Fact-Checker Jennifer Nam