Evidence Rating: No Effects | One study
Date:
This is a school-based intervention designed to help students with attention deficit hyperactivity disorder (ADHD) develop, practice, and generalize academic and social skills by using volunteer mentors to deliver skills training to students. This program is rated as No Effects. Academic functioning and parent/teacher ratings of student behavior reflecting ADHD symptoms did not differ statistically significantly for youths in the intervention group, compared with the control group.
A No Effects rating implies that implementing the program is unlikely to result in the intended outcome(s) and may result in a negative outcome(s).
This program's rating is based on evidence that includes at least one high-quality randomized controlled trial.
Program Goals/Target Population
Challenging Horizons Program – Mentoring (CHP-Mentoring) is a school-based intervention for young adolescents with attention deficit hyperactivity disorder (ADHD), which aims to improve academic and social functioning within the context of a mentoring relationship. Students from the target population are paired with school staff who serve as adult mentors and use elements of the broader CHP curriculum to help develop, practice, and generalize academic and social skills. Eligible participants are male and female middle school students (6th to 8th grade) who have a prior diagnosis of ADHD or exhibit at least four of nine symptoms of ADHD, according to the criteria from the Diagnostic and Statistical Manual of Mental Disorders (DSM).
Program Components
The CHP-Mentoring intervention is based on the CHP model. CHP includes organizational interventions to address disorganization of materials and assignments, study and note-taking skills training to help students improve comprehension of written and spoken information, and an interpersonal skills group to help students improve their relations with peers and adults.
The CHP-Mentoring version uses teachers and other school staff as adult mentors who work one-on-one with students during regular school hours. At the beginning of the school year, each student in the program is assigned a mentor. Although mentors choose to fill the role voluntarily, informed matches based on prior student–staff relationships are regarded by the program as potentially more beneficial (Evans et al. 2016). Mentors focus on establishing strong supportive relationships with youths while implementing a subset of the CHP interventions. Mentors meet with mentees approximately once a week at varying times during the school day such as before class, during homeroom, or during study halls. Mentors track student data, including accuracy of assignment recording and adherence to the program’s criteria for student organization.
Mentors meet with program consultants every 2 weeks to review student data tracked by the mentor and discuss intervention modifications as needed. Mentors and program consultants are provided with a CHP-Mentoring manual. The manual includes information on all procedures for each intervention, decision rules for modifying and progressing through interventions, and forms for tracking progress. Following the mentor–consultant meetings, mentors are encouraged to schedule feedback meetings with their assigned students to review progress and discuss any revisions made to the students’ intervention plans.
Key Personnel
Mentors are recruited from staff at the school in which the program is implemented. Mentors serve on a voluntary basis, but may receive a small stipend for participation. Although mentors work one-on-one with students, some mentors may have more than one student with whom they work during the school year. Program consultants are school-employed mental health professionals or doctoral students in a clinical or school psychology program who meet with the mentors biweekly. Additionally, a program supervisor, typically a graduate student or postdoctoral fellow, provides oversight and supervision of consultants and mentors.
Program Theory
The Challenging Horizons Program is designed to strengthen academic and social behaviors in students with ADHD. The program is identified as a training rather than as a behavior-management intervention. This is reflected in the intervention’s focus on teaching skills, enabling practice of the learned skills, and promoting generalization of skills (but without manipulating behavioral contingencies in the school setting). Furthermore, schools serve as the host setting for the program, as this allows direct application of many of the skills that are the focus of the program in both academic and social interactions.
Overall, the findings from the study by Evans and colleagues (2016) showed that students in the Challenging Horizons Program (CHP)-Mentoring condition did not differ statistically significantly over time from control youths on the measured outcomes.
Study 1
DBD: ODD Symptoms
Students in the CHP-Mentoring program did not differ statistically significantly in change over time from students in the control group on teacher-reported scores on the ODD Symptoms subscale of the DBD.
CPS: Academic Competence
Students in the CHP-Mentoring program did not differ statistically significantly in change over time from students in the control group on teacher-reported scores on the Academic Competence subscale of the Classroom Performance Survey (CPS).
Percent of Assignments Turned In
Students in the CHP-Mentoring program did not differ statistically significantly in change over time from students in the control group on teacher-reported percentage of assignments turned in.
IRS: Academic Progress
Students in the CHP-Mentoring program did not differ statistically significantly in change over time from students in the control group on teacher-reported scores on the Academic Progress subscale of the Impairment Rating Scale (IRS).
GPA
Students in the CHP-Mentoring program did not differ statistically significantly in change over time from students in the control group on their GPA.
COSS: Task Planning
Students in the CHP-Mentoring program did not differ statistically significantly in change over time from students in the control group on parent-reported scores on the Task Planning subscale of the Children’s Organizational Skills Scale (COSS).
HPC: Inattention/Avoidance
Students in the CHP-Mentoring program did not differ statistically significantly in change over time from students in the control group on parent-reported scores on the Inattention/Avoidance subscale of the Homework Problem Checklist (HPC).
HPC: Poor Productivity
Students in the CHP-Mentoring program did not differ statistically significantly in change over time from students in the control group on parent-reported scores on the Poor Productivity and Nonadherence with Homework Rules subscale of the HPC.
DBD: Inattention
Students in the CHP-Mentoring program did not differ statistically significantly in change over time from students in the control group on teacher-reported scores on the Inattention subscale of the Disruptive Behavior Disorders Rating Scale (DBD).
DBD: Hyperactivity/Impulsivity
Students in the CHP-Mentoring program did not differ statistically significantly in change over time from students in the control group on teacher-reported scores on the Hyperactivity/Impulsivity subscale of the DBD.
Study 1
Evans and colleagues (2016) used a stratified random assignment research design to evaluate the Challenging Horizons Program (CHP) in nine urban, suburban, and rural middle schools. Students in sixth to eighth grades were recruited in three cohorts over 3 academic years. Recruitment was conducted in the spring of the academic year prior to the start of the intervention by sending letters home to all parents at participating schools, having school staff directly inform parents of some students about the opportunity to participate, and posting flyers in participating schools. Interested parents/caregivers underwent an initial telephone screening followed by an evaluation to determine students’ eligibility for the study. To be eligible, students had to 1) attend a participating school; 2) meet Diagnostic and Statistical Manual of Mental Disorders [4th ed. text revision (DSM-IV-TR)] diagnostic criteria for attention deficit hyperactivity disorder (ADHD); 3) demonstrate impairment in academic, social, or other relevant domains based on either parent or teacher report; 4) have an IQ score of 80 or greater; and 5) not meet diagnostic criteria for pervasive developmental disorder or serious psychiatric disorder such as schizophrenia or bipolar disorder.
After screening for eligibility and completing baseline assessment, students were randomized to one of three study conditions: CHP-Mentoring, CHP-After School, or a Community Care control condition. This review focuses on the comparison of the CHP-Mentoring condition to the control condition. In the CHP-Mentoring condition, each student received a small portion of the CHP-After School intervention from a mentor who was a teacher or other staff member in his or her school. Mentors agreed to meet with their students weekly, with most mentors meeting their students in the morning before classes, during homeroom, at lunch, or during study halls. Participants in the control condition received a list of locally available child and family psychosocial and pharmacological intervention options at the start of the school year.
Participants were 110 sixth- to eighth-grade students in the CHP-Mentoring program and 104 students in the control condition. Mentors and students met an average of 25.17 times and had an average of 1.84 feedback sessions. The average mentor–mentee session lasted 12.12 minutes. Of the mentor–student pairings, 30 percent involved one intervention, 50 percent involved two interventions, 18 percent involved three interventions, and 2 percent involved four interventions. Seventy-five percent of mentor–mentee interventions focused on organizational skills, 53 percent focused on homework-recording accuracy, 30 percent focused on daily report cards, 20 percent focused on missed assignment checks, 10 percent focused on study skills, and 3 percent focused on another type of intervention. The average number of consultant–mentor meetings was 13.39, and the average meeting duration was 19.59 minutes.
Outcome measures were collected from parents at baseline, at four equally spaced times during the intervention year, and then 6 months after the end of the intervention (i.e., halfway through the following school year). These measures included 1) scores from the Task Planning subscale of the Children’s Organizational Skills Scale (COSS), a 58-item scale assessing youth organization, time management, and planning difficulties; and 2) scores from the Inattention and Avoidance of Homework and the Poor Productivity and Nonadherence with Homework Rules subscales of the Homework Problems Checklist, a 20-item rating scale assessing homework performance. Outcome measures were also collected from teachers at all time points except at baseline. These measures included 1) the Academic Competence subscale of the Classroom Performance Survey (CPS), a 15-item rating scale assessing academic and interpersonal competence; 2) reports of the percentage of assignments turned in; 3) the Academic Progress subscale of the Impairment Rating Scale (IRS), a 7-item rating scale assessing areas of impairment at school; and 4) the Inattention, Hyperactivity/Impulsivity, and Oppositional Defiant Disorder (ODD) subscales of the Disruptive Behavior Disorders Rating Scale (DBD), a 26-item checklist for symptoms of ADHD and ODD. Grades for each participating student were also collected at baseline (i.e., the last quarter of the previous school year), at four points during the intervention year (i.e., for each quarter of school), and at four points during the year following the intervention (i.e., for each quarter of the follow-up year) and converted into GPAs for core subject areas (English/language arts, social studies, math, and science). Parents were also asked about their child’s use of medication and other treatment for ADHD at each assessment point.
Intervention effects on youth functioning were assessed using hierarchical linear modeling (HLM) via a linear mixed effect model. All analyses used an intent-to-treat (ITT) approach (i.e., including data from all participants randomized into a condition regardless of level of participation in intervention activities). The basic models included group assignment, time, and group-by-time interaction as predictors of outcomes, with control for baseline achievement scores, intelligence estimate, family income, and parental education. Medication use was included as a time-varying covariate.
The Challenging Horizons Program – Mentoring manual, which includes information on all procedures for each intervention, decision rules for modifying and progressing through interventions, and forms for tracking progress, is available to mentors and consultants and can be obtained from the program developer.
Additional Outcomes
Additional outcome measures in the study by Evans and colleagues (2016) included parent reports on the following measures: 1) Organized Actions and Memory Materials Management subscales of the Children’s Organizational Skills Scale (COSS); 2) the Relation with Children and Academic Progress subscales of the Impairment Rating Scale (IRS); 3) the Inattention, Hyperactivity/Impulsivity, and ODD subscales of the Disruptive Behavior Disorders Rating Scale (DBD); and 4) the Social Skills, Problem Behavior, and Academic Competence subscales of the Social Skills Improvement System (SSIS), a rating scale assessing the domains of social skills, problem behavior, and academic competence. Teachers also provided additional reports on the following measures; 1) the Interpersonal Competence subscale of the Classroom Performance Survey (CPS); and 2) the Relation with Children subscale of the Impairment Rating Scale (IRS).
Overall, at follow up, no significant group differences were found in change over time between students in the CHP-Mentoring and control groups for any of these outcome measures.
These sources were used in the development of the program profile:
Study 1
Evans, Steven W., Joshua M. Langberg, Brandon K. Schultz, Aaron Vaughn, Mekibib Altaye, Stephen A. Marshall, and Allison K. Zoromski. 2016. “Evaluation of a School-Based Treatment Program for Young Adolescents with ADHD.” Journal of Consulting and Clinical Psychology 84(1):15–30. doi:10.1037/ccp0000057
These sources were used in the development of the program profile:
Evans, Steven W., Zewelanji N. Serpell, Brandon K. Schultz, and Dena A. Pastor. 2007. “Cumulative Benefits of Secondary School-Based Treatment of Students with Attention Deficit Hyperactivity Disorder.” School Psychology Review 36(2): 256–73. (This study was reviewed but did not meet Crime Solutions' criteria for inclusion in the overall program rating.)
Following are CrimeSolutions-rated programs that are related to this practice:
This practice provides youth with a positive and consistent adult or older youth relationship to promote healthy youth development and social functioning and to reduce risk factors. The practice is rated Effective in reducing delinquency and improving educational outcomes; Promising in improving psychological outcomes and cognitive functioning; and No Effects in reducing substance use.
Evidence Ratings for Outcomes
Crime & Delinquency - Multiple crime/offense types | |
Education - Multiple education outcomes | |
Mental Health & Behavioral Health - Psychological functioning | |
Mental Health & Behavioral Health - Cognitive functioning | |
Mental Health & Behavioral Health - Social functioning | |
Drugs & Substance Abuse - Multiple substances |
Age: 10 - 14
Gender: Male, Female
Race/Ethnicity: White, Black, Hispanic, Other
Geography: Suburban Urban Rural
Setting (Delivery): School
Program Type: Academic Skills Enhancement, Conflict Resolution/Interpersonal Skills, Mentoring
Current Program Status: Active