Glossary
Stepwise Regression
A systematic approach to variable selection that iteratively adds or removes predictors based on a criterion such as AIC, BIC, or p-values. Beginning from either a full or null model, the algorithm evaluates whether adding or dropping a single predictor improves the criterion....
Definition
A systematic approach to variable selection that iteratively adds or removes predictors based on a criterion such as AIC, BIC, or p-values. Beginning from either a full or null model, the algorithm evaluates whether adding or dropping a single predictor improves the criterion. The process stops when no further improvement is possible. Stepwise regression is a useful exploratory tool but can be unstable and is increasingly supplemented by regularisation methods.
Why It Matters
Variable selection is essential when many candidate predictors exist but only a subset is relevant. Stepwise regression provides an automated, criterion-driven selection process that can be a practical starting point. However, its instability (small data changes can yield different models) and tendency to overfit mean that results should be validated on holdout data or through cross-validation.
Example
A marketing analyst has 30 potential predictors of customer churn. Forward stepwise selection based on AIC starts with no predictors and iteratively adds the variable that most improves the model, ultimately retaining 8 predictors. The resulting parsimonious model balances predictive power with interpretability.
Related Terms
Software Notes
- SPSS: Analyze > Regression > Linear; select "Stepwise" as the method
- R:
stepAIC()from theMASSpackage; e.g.,stepAIC(full_model, direction = "both") - Stata:
stepwisecommand; e.g.,stepwise, pr(0.10): regress y x1-x30