Bayesian Inference

Definition

Bayesian inference is a statistical framework in which probability expresses a degree of belief rather than a long-run frequency. Bayesian methods update a prior distribution for parameters using observed data via Bayes' theorem to obtain a posterior distribution. This posterior captures both the evidence in the data and the analyst's prior knowledge.

Why It Matters

Bayesian inference naturally incorporates uncertainty and produces full probability distributions for quantities of interest. It is increasingly used in policy evaluation, clinical trials, and decision analysis because it allows researchers to incorporate existing knowledge, update beliefs sequentially as new data arrive, and make direct probability statements about hypotheses. Unlike frequentist methods, Bayesian analysis can handle small samples, complex hierarchical structures, and missing data more flexibly.

Example

A pharmaceutical company has preliminary evidence from an early-phase trial suggesting a new drug has a 65% probability of efficacy. A subsequent large-scale trial yields strong positive data. Using Bayesian updating, the company combines the prior belief (65%) with the new trial likelihood to obtain a posterior probability of efficacy of 92%. This direct probability statement — "there is a 92% chance the drug is effective" — is impossible under frequentist inference but is natural and actionable under the Bayesian framework.

Related Terms

• Confidence Interval (the Bayesian analogue is the credible interval)
• P-Value
• Odds Ratio
• Meta-Analysis (Bayesian meta-analysis is common)

Software Notes

• SPSS: Bayesian analysis is available in SPSS Statistics 28+ via Analyze > Bayesian Statistics. Covers one-sample and two-sample t-tests, ANOVA, linear regression, and binomial tests. Requires specification of prior distributions.
• R: rstanarm::stan_glm(y ~ x, data = df, family = gaussian()) for Bayesian regression with default weakly informative priors. brms::brm(y ~ x, data = df) offers a more flexible interface. bayesplot::mcmc_areas() visualises posterior distributions.
• Stata: bayes: regress y x fits a Bayesian linear regression with default priors. bayesstats summary reports posterior means, credible intervals, and probabilities. bayesgraph produces diagnostic and posterior plots.