The p-value is the area in the tail beyond your statistic
From a z or t score, the p-value is the probability of a result at least that extreme under the null hypothesis. A two-tailed test counts both tails, so it doubles the one-tail area.
Φ is the cumulative normal distribution. For a t score, swap in the t distribution with your degrees of freedom; the two-tail doubling is the same. A one-tailed test uses a single tail, so its p-value is half the two-tailed one.
Significance thresholds and what they signal
| p-value | Verdict at α = 0.05 | Strength of evidence against the null |
|---|---|---|
| p ≤ 0.001 | Significant | Very strong |
| p ≤ 0.01 | Significant | Strong |
| p ≤ 0.05 | Significant | Moderate (the common cutoff) |
| 0.05 < p ≤ 0.10 | Not significant | Weak / suggestive only |
| p > 0.10 | Not significant | None — fail to reject the null |
α = 0.05 is a convention set by Fisher, not a rule. Physics uses about 0.0000003 (five sigma) for a discovery; some clinical work uses 0.01.
What the p-value does not tell you
- Not the chance the null is true. A p-value is the probability of the data given the null, not the probability of the null given the data. Reversing the two is the most common error.
- Not the size of the effect. A tiny effect can hit p < 0.05 with a large enough sample. Always report an effect size alongside the p-value.
- Not a cliff at 0.05. A p-value of 0.049 and 0.051 are practically identical. Treat the number as continuous, not pass/fail.
- "Not significant" is not "no effect." A high p-value means you failed to find evidence against the null, which is not the same as proving it true.
Common questions
What does a p-value of 0.05 mean?
It means that if the null hypothesis were true, there is a 5% chance of getting a result this extreme or more extreme by chance alone. It does not mean there is a 95% chance your hypothesis is correct.
When is a result statistically significant?
When the p-value is at or below the significance level you set, usually 0.05. A p-value of 0.03 is significant at 0.05; a p-value of 0.08 is not. The threshold is a convention, not a law of nature.
What is the difference between one-tailed and two-tailed?
A two-tailed test asks whether the result differs from the null in either direction and splits the significance across both tails. A one-tailed test looks in one predicted direction only, which halves the p-value. You must pick the direction before seeing the data.
