# The method that can "prove" almost anything - James A. Smith

• 750,674 Views
• TEDEd Animation

#### Let’s Begin…

In 2011, a group of researchers conducted a study designed to find an impossible result. Their study involved real people, truthfully reported data, and commonplace statistical analyses. So how did they do it? The answer lies in a statistical method scientists often use to try to figure out whether their results mean something, or if they’re random noise. James A. Smith explores p-hacking.

Create and share a new lesson based on this one.

TED-Ed Animations feature the words and ideas of educators brought to life by professional animators. Are you an educator or animator interested in creating a TED-Ed Animation? Nominate yourself here »

## Meet The Creators

• Educator James A. Smith
• Director Anton Bogaty
• Music Salil Bhayani, cAMP Studio
• Sound Designer Amanda P.H. Bennett
• Director of Production Gerta Xhelo
• Editorial Director Alex Rosenthal
• Producer Bethany Cutmore-Scott
• Editorial Producer Elizabeth Cox
• Fact-Checker Jennifer Nam
Additional Resources for you to Explore
P-values are part of a method in science called ‘hypothesis-testing’. Hypothesis testing involves defining a null-hypothesis: a statement that would be true if nothing happened in your experiment. For example, imagine you want to test whether people who ate chocolate were heavier than those who didn’t: the null hypothesis would be that there is no difference in weight between people eating and not eating chocolate.

We’d also state an alternative hypothesis. This reflects what we think might be going on: for example, the alternative hypothesis would be that there is a difference in weight between people eating and not eating chocolate.

The p-value provides evidence against the null hypothesis. If it’s low, such as below 0.05, then it’s quite unlikely that the null hypothesis is correct, so we reject it.

But P-values are really hard to understand, and even leading scientists struggle to explain what they mean. They tell you the probability of observing data at least as extreme as the data you observed in an experiment, assuming that the null hypothesis is true. It’s easy to think that they also tell you the probability that the null hypothesis is true, but that’s not quite right, because the probability we calculate assumes that the null hypothesis is true.

Some people think that the widespread use of and misunderstanding about p-values leads to problems with the reproducibility of scientific findings. You can learn more about problems with reproducibility in science in this video.

Don’t lose faith in science though – it’s one of the best ways that we have to learn about and improve the world. And partly because it’s so important, there is a new field called ‘metascience’, that aims to better understand and improve the way that science is done.
TED-Ed
Lesson Creator
New York, NY
06/21/2021 •
3 Responses

Create and share a new lesson based on this one.

TED-Ed Animations feature the words and ideas of educators brought to life by professional animators. Are you an educator or animator interested in creating a TED-Ed Animation? Nominate yourself here »

## Meet The Creators

• Educator James A. Smith
• Director Anton Bogaty
• Music Salil Bhayani, cAMP Studio
• Sound Designer Amanda P.H. Bennett
• Director of Production Gerta Xhelo
• Editorial Director Alex Rosenthal
• Producer Bethany Cutmore-Scott
• Editorial Producer Elizabeth Cox
• Fact-Checker Jennifer Nam

## More from Math In Real Life

Thinking & Learning

## Can you survive the creation of the universe by solving this riddle?

lesson duration 06:24

772,230 views

Mathematics

## Can you solve the rogue submarine riddle? Difficulty level: Master

lesson duration 06:19

987,095 views

Thinking & Learning

## Can you solve the giant spider riddle?

lesson duration 05:07

2,520,626 views