Evidence Rating: No Effects | One study
Date:
An intensive residential program that provides training and services, including structured one-on-one mentoring, to at-risk youth (ages 16 to 18 years). This program is rated No Effects. The program had a statistically significant, positive impact on employment and GED attainment among participating youth as compared with control group youth. However, the program had no effect on youths’ frequency of arrests, marijuana or other illegal drug use, delinquent behavior, or psychological distress.
A No Effects rating implies that implementing the program is unlikely to result in the intended outcome(s) and may result in a negative outcome(s).
This program's rating is based on evidence that includes at least one high-quality randomized controlled trial.
Program Goals/Target Population
The National Guard ChalleNGe Program is an intensive residential program that provides training and services to promote leadership, academic excellence, job skills, and community service among at-risk youth. The goals of the ChalleNGe Program are to assist youth in obtaining education and employment and in developing positive attitudes and behaviors. The program serves youths ages 16 to 18 who have dropped out of or been expelled from school and are unemployed, drug free, and not heavily involved with the justice system. The program is open to both males and females, and there are no income-based eligibility criteria.
Program Components/Program Theory
The 17-month program is guided by the Positive Youth Development framework and includes three phases: a 2-week PreChalleNGe Phase, a 20-week Residential Phase, and a 1-year Postresidential Phase. The PreChalleNGe Phase includes assessment and an introduction to the program’s rules and expectations, military discipline, and teamwork. Candidates who complete this phase are formally enrolled into the program as “cadets” and move to the second phase. The Residential Phase includes a curriculum structured around eight core components: Leadership/Followership, Responsible Citizenship, Service to Community, Life-Coping Skills, Physical Fitness, Health and Hygiene, Job Skills, and Academic Excellence. During this phase, the program provides assistance with preparing for the General Educational Development (GED) exam or with obtaining a high school diploma. The PreChalleNGe and Residential phases of the program use military structure, discipline, facilities, and staff to accomplish their objectives. Youths are divided into platoons and squads and live in barracks on a military base. The daily schedule is highly structured, and the military staff closely supervises youths at all times.
Toward the end of the Residential Phase, youths work with the staff to arrange postresidential placement, which may consist of employment, education, or military service. Youths who show competency on the core components and have an acceptable placement lined up successfully complete the Residential Phase and move into the Postresidential Phase. This phase includes a structured one-on-one mentoring component. The goals of the Postresidential Phase are to assist youths with their transition back into the community and to support them in maintaining the new attitudes and behaviors they have learned in the program. Youths nominate their own mentors, drawn from family friends, extended family members, godparents, school personnel, or religious leaders. Program staff members then help youths initiate their mentoring relationships as part of the Residential Phase. During the Postresidential Phase, mentors are required to have a minimum of four contacts with their mentees per month. Staff members contact both youths and their mentors at least monthly to solve problems and monitor youths’ progress.
Key Personnel
Military staff lead all training and curriculum sessions. Youths nominate their own mentors for the Postresidential Phase, and military staff screen and train the mentors.
The evaluation of the National Guard ChalleNGe Program conducted by Millenky and colleagues (2011) overall found mixed results. At the 36-month follow-up, youths in the program group were more likely than those in the control group to have been employed and to have obtained a high school diploma or GED. These differences were statistically significant. However, there were no statistically significant differences between groups in number of arrests and in reported levels of delinquent behavior, illegal drug use, marijuana use, and psychological distress. The preponderance of evidence suggests the program did not have the intended effects.
Study 1
Involvement in Any Productive Activity
There was no statistically significant difference between treatment and control groups in their reported involvement in productive activity at the 36-month follow-up.
Number of Arrests
There was no statistically significant difference between treatment and control groups in their reported frequency of arrests at the 36-month follow-up.
Delinquent Behavior
There was no statistically significant difference between the treatment and control groups in self-reported delinquent behavior at the 36-month follow-up.
Frequent Illegal Drug Use
There was no statistically significant difference between the treatment and control groups in reported frequency of other illegal drug use at the 36-month follow-up.
Frequent Marijuana Use
There was no statistically significant difference between the treatment and control groups in reported frequency of marijuana use at the 36-month follow-up.
Psychological Distress
The treatment and control groups did not differ significantly in their reported levels of psychological distress at the 36-month follow-up.
Earning High School Diploma or GED
There was a statistically significant difference between the treatment and control groups in their reported education attainment at the 36-month follow-up. Youths in the program group were significantly more likely than those in the control group to report having obtained a high school diploma or GED, with 72 percent of the program group reporting a GED or high school diploma, compared with 56 percent of the control group.
Employment Status
There was a statistically significant difference between the treatment and control groups in their reported employment status at the 36-month follow-up. Fifty-eight percent of all youths in the treatment group reported being employed in the past 12 months, compared with 51 percent of youths in the control group.
Study 1
Millenky and colleagues (2011) evaluated the National Guard ChalleNGe Program, a residential intervention designed for high school dropouts. To be eligible for the program, youths must have dropped out or been expelled from school and be unemployed, drug free, not currently on probation or parole for anything beyond juvenile status offenses, and not convicted of a felony or capital offense. Youths were recruited for the study through 10 participating program sites in California, Florida, Georgia, Illinois, Michigan, Mississippi, New Mexico, North Carolina, Texas, and Wisconsin. The sites were not randomly selected. Instead, the programs that reported stable staffing and tendency to receive more applicants than they could serve were chosen for the evaluation.
The baseline sample included 3,074 participants, recruited in 18 incoming class cycles from 2005 to 2007. During the study period, random assignment was conducted for a class cycle at a given program site only if the number of qualified applicants was at least 25 greater than the number needed to meet the site’s graduation goal. Participants were randomly assigned to either the ChalleNGe program (2,320 youths) or a waitlist control group (754 youths). A greater number of youths were assigned to the program group than the control group because the primary goal was to fill the number of available program slots.
Follow-up assessments took place at 6, 12, 24, and 36 months after the baseline assessment. Of those who were assigned to the intervention condition, 82 percent completed the 2-week PreChalleNGe phase and 53 percent graduated from the Residential Phase. In addition, 56 percent of all youths in the intervention group reported that they were still in contact with their mentors at the 36-month follow-up.
A subsample of youths was randomly drawn to complete the 36-month survey. This subsample included 1,173 youths, with 722 youths in the program group and 451 youths in the control group. The final composition of this subsample was 42.3 percent white, 33.8 percent African American or Black, 18.1 percent Hispanic, and 5.7 percent other races/ethnicities. The majority of the youths were male (88 percent), and slightly more than half (53.3 percent) were 17 years old.
At the time of random assignment, for the subsample assessed at 36 months, there were no differences between youths in the intervention group and control group on their demographic characteristics, levels of academic achievement, or physical health. However, youths in the intervention group were less likely to receive public assistance and were more likely to have used drugs or alcohol and to have been arrested or convicted of a crime in their lifetime, compared with youths in the control group. The study compared treatment and control groups at the beginning of the intervention and again at the 36-month follow-up.
The regression models were used to test for program effects and were weighted to account for differences across sites in sample size, survey response rate, and program versus control group ratio. The regression analyses also controlled for demographic characteristics (gender, race, and age), highest grade completed at study baseline, whether the participant lived in a two-parent household, and whether the participant applied to the ChalleNGe program because of interest in the military. Pooled estimates of adjusted means were reported in the evaluation. Outcome measures were youths’ self-report measures of their employment status, having earned a high school diploma or GED, involvement in any productive activity (vocational training, for example), number of arrests, delinquent behavior, illegal drug use (other than marijuana, such as LSD, cocaine, crystal meth, amphetamines, and heroin), marijuana use, and psychological distress.
Millenky, Schwartz, and Rhodes (2014) note that fidelity to the ChalleNGe model during the Residential Phase was relatively high. Specifically, field research visits to all program sites suggested that the basic structure of the program was quite similar from site to site and that all sites were implementing the core elements of the program. However, there was some variation across sites in the program environment, approaches to recruitment and discipline, and other elements. Delivery of the intervention during the Postresidential Phase was more uneven across sites. For instance, all mentors were required to receive mentor training, but programs executed this training inconsistently. In addition, some sites were noted to have had few resources to monitor activities during this phase.
Additional Outcomes
Millenky and colleagues (2011) reported potential negative effects of the National Guard (ChalleNGe) Program on selected other outcomes. Specifically, at the 36-month follow-up, youths in the program group were significantly more likely to be overweight based on self-reported height and weight and significantly less likely to report consistent use of birth control if sexually active, compared with those in the control group.
Subgroup Analysis
Millenky, Schwartz, and Rhodes (2014) identified subgroup differences in the impact of the ChalleNGe Program at the 36-month follow-up. When comparing program effects for older youths (those who were 17 or 18 years old at the start of the study) with younger youths (those age 16 at the start of the program), program effects were found to be more positive among older youths for the following outcomes: having obtained a high school diploma or GED, having earned college credit, and employment status. No clear pattern of differences in impacts was detected for the following subgroups that were also compared: those who have ever been arrested or convicted of a crime versus those with no previous involvement in the justice system and those with high versus poor grades in high school. It was found, however, that the impact of the ChalleNGe program on high school diploma receipt was significantly higher among those who at baseline reported having relatively high grades in high school at baseline than it was among those who reported having relatively poor grades.
These sources were used in the development of the program profile:
Study 1
Millenky, Megan, Dan Bloom, Sara Muller–Ravett, and Joseph Broadus. 2011. Staying on Course: Three-Year Results of the National Guard Youth ChalleNGe Evaluation. New York: MDRC.
These sources were used in the development of the program profile:
Bloom, Dan, Alissa Gardenhire–Crooks, and Conrad Mandsager. 2009. Reengage High School Dropouts: Early Results of the National Guard Youth ChalleNGe Program Evaluation. New York, N.Y.: MDRC.
Millenky, Megan, Dan Bloom, and Colleen Dillon. 2010. Making the Transition: Interim Results of the National Guard Youth ChalleNGe Evaluation. New York, N.Y.: MDRC.
Millenky, Megan, Sarah E.O. Schwartz, and Jean E. Rhodes. 2014. “Supporting the Transition to Adulthood Among High School Dropouts: An Impact Study of the National Guard Youth ChalleNGe Program.” Prevention Science, 15(4):448–59.
Perez–Arce, Francisco, Louay Constant, David S. Loughran, and Lynn A. Karoly. 2012. A Cost–Benefit Analysis of the National Guard Youth ChalleNGe Program. Santa Monica, Calif.: RAND Corporation.
Following are CrimeSolutions-rated programs that are related to this practice:
This practice provides youth with a positive and consistent adult or older youth relationship to promote healthy youth development and social functioning and to reduce risk factors. The practice is rated Effective in reducing delinquency and improving educational outcomes; Promising in improving psychological outcomes and cognitive functioning; and No Effects in reducing substance use.
Evidence Ratings for Outcomes
Crime & Delinquency - Multiple crime/offense types | |
Education - Multiple education outcomes | |
Mental Health & Behavioral Health - Psychological functioning | |
Mental Health & Behavioral Health - Cognitive functioning | |
Mental Health & Behavioral Health - Social functioning | |
Drugs & Substance Abuse - Multiple substances |
Youth-initiated mentoring involves a hybrid approach in which youths are empowered to identify, develop, and strengthen natural mentoring relationships from their existing social networks, rather than being assigned a new mentor through a more formal mentoring relationship. The practice is rated Promising for improving psychological, health, school/academic, and social outcomes, and rated No Effects for cognitive functioning outcomes.
Evidence Ratings for Outcomes
Mental Health & Behavioral Health - Psychological functioning | |
Education - Multiple education outcomes | |
Mental Health & Behavioral Health - Social functioning | |
Mental Health & Behavioral Health - Cognitive functioning |
Age: 16 - 18
Gender: Male, Female
Race/Ethnicity: White, Black, Hispanic, Other
Setting (Delivery): Other Community Setting, Residential (group home, shelter care, nonsecure)
Program Type: Academic Skills Enhancement, Leadership and Youth Development, Mentoring, Vocational/Job Training
Targeted Population: Truants/Dropouts
Current Program Status: Active