Program Profile: Challenging Horizons Program - After-School Version (CHP-After School)

This is an after-school intervention designed to help students with attention deficit hyperactivity disorder (ADHD) develop, practice, and generalize academic and social skills by using volunteer mentors to deliver skills training to students. This program is rated as Ineffective. Academic functioning and parent/teacher ratings of student behavior reflecting ADHD symptoms did not differ statistically significantly between youths in the intervention group and those in the control group.

An Ineffective rating implies that implementing the program is unlikely to result in the intended outcome(s).

This program's rating is based on evidence that includes at least one high-quality randomized controlled trial.

Program Goals/Target Population

Challenging Horizons Program – After School (CHP-After School) is an after-school intervention (located at the school attended by participating students) for young adolescents with ADHD, which aims to improve their academic and social functioning through a relationship with an assigned Personal Counselor, or PC. The PC’s role is to develop a therapeutic relationship with each assigned student and to coordinate CHP interventions that help these students develop, practice, and generalize academic and social skills. Eligible participants are male and female middle school students (6th to 8th grade) who have a prior diagnosis of ADHD or exhibit at least four of nine symptoms of ADHD, according to the criteria from the Diagnostic and Statistical Manual of Mental Disorders (DSM).

Program Components

The CHP-After School intervention is based on the CHP model. CHP includes organizational interventions to address disorganization of materials and assignments, study and note-taking skills training to help students improve comprehension of written and spoken information, and an interpersonal skills group to help students improve their relations with peers and adults.

The CHP-After School version uses assigned PCs (trained undergraduate students) who work with their students in an after-school program. At the beginning of the school year, each student in the program is assigned to a PC, who focuses on establishing therapeutic relationships with youths while implementing CHP interventions and monitoring progress. PCs meet with students for a 2-hour and 15-minute program session twice a week. Each program session is composed of five activities: a meeting between the student and his or her PC, a group intervention targeting social impairment (ISG), recreation/game time, education/study skills group, and individual education time for homework completion. In the ISG component, students identify and work on personal social goals with their PC. In the education group, students focus on skills for studying, notetaking, summarizing, and writing. Skills are introduced with direct instruction followed by application of the skill at home or in class.

PCs track student data, including accuracy of assignment recording and adherence to the program’s criteria for student organization. The work of PCs is overseen by a site supervisor who also leads group activities. PCs are provided with a CHP-After School manual. The manual includes information on all procedures for each intervention, decision rules for modifying and progressing through interventions, and forms for tracking progress. PCs also check lockers, binders, book bags, and student planners to monitor student adherence to the organization criteria.

Key Personnel

The program is staffed primarily by the PCs. These are undergraduate students who receive 9 hours of pre-program training, and 15–30 minutes of individual and 60–90 minutes of group supervision weekly throughout the school year. The site supervisor, typically a graduate student or postdoctoral fellow, supervises the PCs and implements some of the program components, as noted above.

Program Theory

The CHP is designed to strengthen academic and social behaviors in students with ADHD. The program is identified as a training rather than a behavior-management intervention. This is reflected in the intervention’s focus on teaching skills, enabling practice of the learned skills, and promoting generalization of skills (but without manipulating behavioral contingencies in the school setting). Furthermore, schools serve as the host setting for the program, as this allows direct application of the skills that are the focus of the program, in both academic and social interactions.

Study

Evans and colleagues (2016) used a stratified random assignment design to evaluate the Challenging Horizons Program (CHP) in nine urban, suburban, and rural middle schools. Students in sixth to eighth grades were recruited in three cohorts over 3 academic years. Recruitment was conducted in the spring of the academic year prior to the start of the intervention by 1) sending letters home to all parents at participating schools, 2) having school staff directly inform parents of some students about the opportunity to participate, and 3) posting flyers in participating schools. Interested parents/caregivers underwent an initial telephone screening followed by an evaluation to determine the students’ eligibility for the study. To be eligible, students had to 1) attend a participating school; 2) meet Diagnostic and Statistical Manual of Mental Disorders, (4th edition, text revision [DSM-IV-TR]) diagnostic criteria for attention deficit hyperactivity disorder (ADHD); 3) demonstrate impairment in academic, social, or other relevant domains based on either parent or teacher report; 4) have an IQ of 80 or greater; and 5) not meet diagnostic criteria for pervasive developmental disorder or serious psychiatric disorders such as schizophrenia or bipolar disorder. 

After screening for eligibility and completing baseline assessment, students were randomized to one of three study conditions: CHP-After School, CHP-Mentoring, or a Community Care control condition. This review focuses on the comparison of the CHP-After School condition to the control condition. In the CHP-After School condition, each student was assigned to a Personal Counselor (PC) and participated in twice-weekly, after-school sessions involving the five program activities. Participants in the control condition received a list of locally available child and family psychosocial and pharmacological intervention options at the start of the school year. 

Participants were 112 sixth- to eighth-grade students in the CHP-After School program and 104 students in the control condition. No more than two students were assigned to one PC. Students attended an average of 31.85 after-school sessions, with 94 percent of the students attending at least once. 

Outcome measures were collected from parents at baseline, at four equally spaced times during the intervention year, and then 6 months after the end of the intervention (i.e., halfway through the following school year). These measures included 1) the Task Planning subscale of the Children’s Organizational Skills Scale (COSS), a 58-item scale assessing youth organization, time management, and planning difficulties; and 2) the Inattention and Avoidance of Homework and the Poor Productivity and Nonadherence with Homework Rules subscales of the 20-item Homework Problems Checklist. Outcome measures were also collected from teachers at all time points except at baseline. These measures included 1) the Academic Competence subscale of the Classroom Performance Survey (CPS), a 15-item rating scale assessing academic and interpersonal competence; 2) reports of the percentage of assignments turned in; 3) the Academic Progress subscale of the Impairment Rating Scale (IRS), a 7-item rating scale assessing areas of impairment at school; and 4) the Inattention, Hyperactivity/Impulsivity, and Oppositional Defiant Disorder (ODD) subscales of the Disruptive Behavior Disorders Rating Scale (DBD),  a 26-item checklist for symptoms of ADHD and ODD. Grades for each participating student were also collected for the last quarter of the previous school year, each quarter of both the intervention and follow-up school years, and converted into grade point averages (GPAs) for core subject areas (English/language arts, social studies, math, and science). Parents were also asked about their child’s use of medication and other treatment for ADHD at each assessment point. Descriptive information on demographic characteristics for the three groups at baseline was reported, but did not include significance tests for group differences. 

Intervention effects on youth functioning were assessed using hierarchical linear modeling (HLM) via a linear mixed effect model. All analyses used an intent-to-treat (ITT) approach (i.e., including data from all participants randomized into a condition regardless of level of participation in intervention activities). The basic models included group assignment, time, and group-by-time interaction as predictors of outcomes and included the following covariates: achievement scores, intelligence estimate, family income, and parental education (assessed at baseline), as well as medication use (included as a time-varying covariate).

The Challenging Horizons Program – After School manual, which includes information on all procedures for each intervention, decision rules for modifying and progressing through interventions, and forms for tracking progress, is available to mentors and consultants and can be obtained from the program developer.

Additional Outcomes

Additional outcome measures in the study by Evans and colleagues (2016) included parent reports on the following measures: 1) Organized Actions and Memory Materials Manage subscales of the Children’s Organizational Skills Scale (COSS); 2) the Relation with Children and Academic Progress subscales of the Impairment Rating Scale; 3) the Inattention, Hyperactivity/Impulsivity, and ODD subscales of the Disruptive Behavior Disorders Rating Scale (DBD); and 4) the Social Skills, Problem Behavior, and Academic Competence subscales of the Social Skills Improvement System, a rating scale assessing the domains of social skills, problem behavior, and academic competence. Teachers also provided additional reports on the following measures: 1) the Interpersonal Competence subscale of the Classroom Performance Survey (CPS); and 2) the Relation with Peers subscale of the IRS.

Group differences in change over time were generally not found between students in the CHP-After School and control groups, with the exception of parent-reported inattention on the DBD and the Materials Management subscale of the COSS, for which significant differences favoring the intervention group were found.

These sources were used in the development of the program profile:

These sources were used in the development of the program profile: