Study 1
Fetsco and Kain (n.d.) used a quasi-experimental design to evaluate the GEAR UP – Academic Mentoring in Mathematics program. The following outcomes were assessed: 1) obtaining a grade of C or above in algebra I, 2) obtaining a grade of C or above in geometry, and 3) passing a state-mandated math test (Arizona Instrument to Measure Standards, or AIMS). Grades were calculated by averaging numeric grades for each semester (a score above 2.0 was considered a C or above). AIMS scores were calculated as pass–fail based on meeting or exceeding the state standard. Study samples differed for each outcome assessment.
The treatment groups comprised all GEAR UP students who graduated in 2012 from a large urban high school in southwestern Arizona. The comparison groups for the algebra and geometry grade outcomes consisted of students who took the same math courses at the same school, but graduated in 2011, before the GEAR UP program was implemented. In addition, as a result of an unanticipated change in state testing, a separate comparison group from the class of 2012, who did not receive the GEAR UP program, was used to assess the AIMS math test scores outcome. However, this was a limitation to the study, as the comparison group students were not separated from the treatment group during the school day, and despite not having access to mentors, they may have experienced some diffusion of benefits.
For the algebra I grades outcome assessment, the treatment group consisted of 114 students who were 94.7 percent Hispanic and 55.3 percent female. Approximately 6.1 percent were English Language Learners (ELLs), and 6.1 percent had Individualized Education Plans (IEPs). The comparison group consisted of 86 non-GEAR UP students who completed the same algebra I course the previous year without mentors. These students were 96.5 percent Hispanic and 51.2 percent female. Approximately 35.0 percent were ELLs, and 9.3 percent had IEPs. There were statistically significant differences between groups; the treatment group had higher Terra Nova Math and Language scores, whereas the comparison group had a greater number of ELLs. Propensity score analysis was conducted to account for these and a number of other covariates such as gender, race, ELL status, special education status, and Terra Nova Normal Curve Equivalents (NCEs) from the Math, Language, and Reading subtests.
For the geometry grades outcome, the treatment group consisted of 92 students who were 97.8 percent Hispanic and 58.7 percent female. Approximately 6.5 percent were ELLs, and 9.8 percent had IEPs. The comparison group consisted of 43 non-GEAR UP students who completed the same geometry course the previous year without mentors. These students were 97.7 percent Hispanic and 58.1 percent female, and 9.3 percent had IEPs. Of the comparison group, 27.9 percent of the students were ELLs, which was statistically significant. An ANCOVA was conducted to predict receiving a C or above, using Terra Nova Math NCEs as a covariate.
For the outcome regarding AIMS math test scores, the treatment group consisted of 197 GEAR UP students who were about 96 percent Hispanic and 52.8 percent female. About 2.5 percent were ELLs, and 6.1 percent had IEPs. The comparison group consisted of 211 non-GEAR UP students who were enrolled at the same time as the treatment group. These students were 27.4 percent Hispanic and 45.5 percent female. Of this comparison group, 8.1 percent were ELLs, and 9.0 percent had IEPs. Propensity score analysis was conducted to account for gender, race, ELL status, special education status, and Terra Nova NCEs from the Math, Language, and Reading subtests.
While not included in the CrimeSolutions review of this study, a survey measure was also used to assess confidence in passing a college-level algebra class and confidence in completing the college preparatory curriculum in high school math. These items were assessed using a 5-point Likert scale ranging from strongly disagree to strongly agree. These ratings were then dichotomized into students who agreed or strongly agreed versus those who did not (the results of the survey are discussed in the Other Information section). The two survey items assessing confidence in math were also assessed using a smaller subset of these groups (n = 149; treatment and n = 149; control). The survey items were administered by algebra II teachers during the third year of the program, and all students in participating classrooms took the survey. No subgroup analyses were conducted.
