What is a p-value?

• A. The probability that the null hypothesis is true.
• B. The probability of observing data as extreme as the observed, assuming the null hypothesis is true; used to assess statistical significance.
• C. A measure of the size of the effect.
• D. The ratio of the observed to expected frequencies.

Answer: (B)

A p-value is the probability, assuming the null hypothesis is true, of obtaining data that are as or more inconsistent with the null as what you actually observed. It captures how surprising your results would be if there were really no effect.

Think of it like this: you start with a model that there’s no real difference or no effect. If you collect data and the results are very unlikely under that model, the p-value is that small probability of getting something this extreme just by chance. It’s not the probability that the null is true, and it doesn’t tell you how big the effect is—those are separate ideas called significance and effect size.

For example, if you’re testing whether a treatment changes an outcome, a small p-value means the observed data would be unlikely if the treatment really had no effect, so you might conclude the effect is real (subject to your chosen significance level). If you have a large sample, even tiny differences can give small p-values, which is why it’s important to consider practical significance and effect size along with the p-value.

Whether you’re looking at a one-sided or two-sided test changes what counts as “as extreme,” but the underlying idea remains: the p-value measures compatibility of the data with the null hypothesis, not the truth of the null itself.