Student Growth Percentiles (SGPs) are a common measure of student progress over time. They are often used in evaluations of teachers, schools, and programs. They are also useful for tracking students over time, assessing teacher effectiveness, and comparing school outcomes among different groups of schools.
However, SGPs are prone to excessive measurement error at the individual level. Moreover, they are difficult to interpret in aggregated forms, such as SGPs for a single school or teacher. To overcome this problem, we propose a data-driven approach to SGP analysis that models longitudinal item-level data with latent regression MIRT models. This approach allows us to study the distributional properties of true SGPs, a necessary step for investigating and interpreting estimated SGPs at the school or teacher level.
The relationship between student covariates and true SGPs is complex. Although many student-level variables have been shown to correlate with a true SGP at the individual level, they do not account for much of the variability in true SGPs across subjects and grades. Instead, the relationships between true SGPs and student background variables are likely to result from multiple mechanisms.
1. Group Variability in Student Covariates
The unequal distribution of student covariates across subjects and grades may create a problem for estimating true SGPs using only those covariates. Specifically, if students have different backgrounds at the individual level, then there will be different average SGPs in each subject and grade. This suggests that the average true SGPs should be adjusted to match the observed distribution of student covariates, but it is unclear how to do this without compromising on accuracy or efficiency.
2. Excessive Measurement Error in Estimated SGPs
The relationships between student characteristics and true SGPs are complex, and they make it difficult to interpret estimated SGPs at the teacher or school level. To overcome this problem, we propose combining longitudinal item-level data with latent regression models and Monte Carlo methods to study the joint distribution of true SGPs, student covariates, and test scores.
3. The Relationship Between Student Covariates and True SGPs is Complex
The variation in the individual-level distribution of student covariates explains part of the variance in true SGPs. This may be due to a variety of mechanisms, including student-level factors that are related to both the true SGPs and the observed student covariates, or contextual effects that are correlated with the students’ surroundings (e.g., neighborhoods or classroom dynamics).
4. The Relationship Between True SGPs and Student Covariates is Complex
While the relationship between student characteristics and true SGPs is complex, it is clear that aggregating estimated SGPs to the teacher or school level overcomes the excessive measurement error problem at the individual level. However, if the relationships between student characteristics and true SGPs persist at this level, then the aggregated SGPs will be characterized by additional sources of variance. This is the case for estimated SGPs that are based on a narrower range of student characteristics, such as those relating to motivation or skill levels.