Statistical Significance

Definition

A result is statistically significant when the p-value from a hypothesis test falls below the pre-determined significance level (commonly 5%). This indicates the observed effect is unlikely to be due to sampling variability alone, given the null hypothesis. Statistical significance does not necessarily imply practical or substantive importance.

Why It Matters

Statistical significance is the gatekeeper of scientific publishing, yet it is frequently misinterpreted. A significant p-value does not prove that the null hypothesis is false, nor does it measure the probability that the result is due to chance. It also says nothing about the size of the effect or its real-world relevance. Modern best practice emphasises reporting effect sizes and confidence intervals alongside p-values to give a fuller picture of the evidence.

Example

A drug trial reports a statistically significant reduction in symptom severity (p = 0.03). While this meets the conventional threshold for significance, the actual reduction is only 2 points on a 100-point scale, with a 95% confidence interval of [0.5, 3.5]. The result is statistically significant but clinically negligible, illustrating why significance alone should never drive conclusions.

Related Terms

• P-Value
• Effect Size
• Confidence Interval
• Null Hypothesis
• Type I Error

Software Notes

• SPSS: Significance is reported automatically in virtually all test output tables under "Sig." or "Asymp. Sig." columns.
• R: summary(model) and test functions return p-values. Use report() from the report package for APA-formatted significance statements.
• Stata: Regression and test output tables show significance in "P>|t|" or "Prob > F" columns. test commands after regression display joint significance tests.