Education Research

Client Context

A multi-academy trust (MAT) with 14 secondary schools wanted to evaluate whether a new mathematics intervention programme improved Key Stage 4 attainment. Pupil-level data from 6,200 students across two cohorts were available, including prior attainment, demographics, and free school meal status.

The Challenge

The intervention was rolled out to all schools simultaneously, meaning there was no natural control group. The MAT needed a quasi-experimental approach that could account for selection effects and provide evidence strong enough to present to the Department for Education and Ofsted. They also needed to understand whether the programme worked better for particular subgroups (Pupil Premium students, students with low prior attainment).

Our Approach

We used a difference-in-differences design comparing KS4 maths attainment before and after the intervention, relative to national trends. To strengthen causal inference, we also implemented a propensity score-matched comparison using students from similar non-participating schools. Multilevel modelling accounted for the hierarchical data structure (students nested within schools). Subgroup analyses examined differential effects by Pupil Premium status and prior attainment quartile. All analyses were conducted in Stata with reproducible do-files.

Results

The difference-in-differences estimate showed a statistically significant improvement of 0.18 standard deviations (95% CI: 0.07 to 0.29) in maths attainment, equivalent to approximately one grade at GCSE. The effect was larger for Pupil Premium students (0.27 SD) than for non-Pupil Premium students (0.12 SD), and the interaction was significant (p = 0.03). The MAT used these findings to secure continued funding and to share best practice across its schools.

Client Testimonial

"The robustness of the analysis gave us the evidence base we needed. Ofsted were impressed that we could demonstrate real impact, not just good intentions."

— Director of Education, Horizon Academies Trust

[Explore More Case Studies → /case-studies/]

[Discuss Your Education Research → /contact/]