The reliability and validity measures of state assessment scores are dependent upon the accuracy reported by the makers of the OAA, PARCC, and AIR assessments. Reliability measures of state assessments across all years in this study and test subjects were reported to be strong with Cronbach’s alpha values greater than 0.7. SEM values were fairly low, depending on performance level scaled score values. Norm-referenced and standardized state assessments are considered the gold standard for quantifying student achievement. This is why OST scores were used in this study. Demographic data of students were self-reported by parents. The school identified gifted students and English learners. Economically disadvantaged data were determined via student participation in the free or reduced lunch program. The validity of parent-reported measures cannot be controlled. The use of HLM for data analysis is the preferred method of analyzing multilevel datasets as it accounts for the shared variance inherent within hierarchical data. The purpose of using HLM in this study was to determine the impact of STEM programming on achievement when students are nested within level-2 (classrooms) which are clustered into level-3 (grade levels). The level-1 (student) variables of OST scores, gender, race, and socioeconomic status were used to determine the variances of in-between groups. With participation in STEM or general education programs being a predictor of the level-1(student) slope, the use of HLM was used to determine the level of moderation between STEM programming and predicting for all level-1 (student) predictors. In conclusion, the percent of variance in regard to STEM programming can be calculated as a moderator in the OST-score and demographics relationships, including but not limited to gender. The hypothesis in this research study was students participating in an integrated middle school STEM program will demonstrate more academic achievement in ELA, math, and science than students participating in a traditional general education setting when correcting for variances among other level-1 (student) demographic variables.

Citation

Chine, D. R. (2022). Measuring Student and School Effects on Academic Achievement in a STEM Program. In J. Johnson & A. Kaban (Eds.), A Pathway to Success? A Longitudinal Study Using Hierarchical Linear Modeling of Student and School Effects on Academic Achievement in a Middle School STEM Program (pp. 51–76). ISTES Organization.