Program Profile: Harlem (NY) Children's Zone - Promise Academy Charter Middle School

This is a charter middle school that serves predominately low-income, minority students (grades 6–8) in Harlem who are usually 2 or 3 years behind grade level. The program is rated Effective. Students who were admitted to the school showed statistically significant improvements in English Language Arts scores, math scores, and attendance rates, compared with students who were not admitted. However, there were no significant differences between groups on matriculation.

An Effective rating implies that implementing the program is likely to result in the intended outcome(s).

This program's rating is based on evidence that includes at least one high-quality randomized controlled trial.

Program Goals/Target Population

The Harlem Children’s Zone (HCZ) is a 97-block area in the Harlem neighborhood of New York, NY, that combines "no excuses" charter schools with communitywide initiatives to address some of the main problems that underprivileged children face every day, such as inadequate schools, high-crime neighborhoods, and health issues. One important component of the HCZ Project is the Promise Academy Charter Middle School, which aims to provide students in grades 6–8 with a well-rounded, high-quality education.

The Promise Academy middle school serves predominately low-income, minority students in Harlem who are usually 2 or 3 years behind grade level. While the community programs offered by HCZ are available to anyone living near the area, the charter middle school is not open to all students. Because of the limited amount of space available, admission to the Promise Academy is only offered to students whose numbers are randomly selected during the middle school lottery. The school began in 2004 with 100 sixth graders, and by 2009, there were 700 middle school students in the system.

Program Components

The Promise Academy combines structural reforms with wraparound services to provide students with a comprehensive college preparatory program. The Promise Academy provides an extended school day and year, coordinated afterschool tutoring, and additional classes on Saturdays for anyone who needs extra help with homework. In the classroom, strong focus is placed on mastering basic skills, especially literacy and mathematics skills. Students have more than 2 hours of literacy instruction and more than 90 minutes of mathematics instruction every day. Students also explore coursework in the arts and sciences.

In addition, many students spend 10 or more hours a day in school. An average school day runs from 8 a.m. to 4 p.m., which is a 20 percent increase over a typical school day. Students are also given the opportunity to participate in afterschool programs, such as photography or music activities, that run from 4 p.m. to 6 p.m. The academic year is 210 days of school, which includes a mandatory 25-day summer program. Only 180 days of school are required by law.

The Promise Academy also runs a health clinic in the middle school building that provides students with free medical, dental, and mental-health services. Students are screened upon entering the school, and they receive regular check-ups through the Children’s Health Fund. Students also receive incentives for achievements (such as money or trips), nutritious and freshly made meals, bus fare, and other intangible benefits, such as support from committed staff members.

Additional Information

The Promise Academy Charter Middle School is part of the HCZ Project that provides a continuum of services focusing on the developmental needs of children in Harlem. In addition to the middle school programs, the Project includes elementary school programs that are available to students in grades K–5. However, the evaluation research examining the effectiveness of the elementary school programs has not yet reached the rigor of the research looking at the middle school programs.

Study

Dobbie and Fryer (2010) examined the effects of being admitted to or attending the Promise Academy charter schools in the Harlem Children’s Zone (HCZ) on academic outcomes. In New York State, oversubscribed charter schools are required by law to allocate enrollment offers through a random lottery system. The study took advantage of this system by comparing the average outcomes of students who “won” the lottery and were offered admission to the Promise Academy to the average outcomes of students who “lost” the lottery and were not offered admission.

The sample included students from the middle school lotteries from 2005 through 2006 (data was not available from the 2004 middle school lottery). The treatment group included 169 lottery winners in grades 6–8. The lottery winners were 51 percent male, and 84 percent Black and 14.5 percent Hispanic. Seventy-two percent qualified for free lunch. The control group included 289 grade 6–8 youth who did not win the lottery. The group was 43.3 percent male, and 82.6 percent Black and 15.4 percent Hispanic. Sixty-five percent qualified for free lunch. There were no statistically significant differences between the groups.

The study used an intent-to-treat (ITT) analysis, which meant the treatment group included all students who were offered admission to Promise Academy, even if they did not attend. The ITT analysis therefore captured the effect of being offered admission, not of actually attending. The study also included treatment-on-treated (TOT) analysis, which looked at the average effect of enrollment in the Promise Academy on those students who won the lottery and chose to attend. The results for the TOT analysis are provided in the Other Information section of the profile.

Data was collected from files at HCZ as well as administrative data on student demographics and academic outcomes from the New York City Department of Education (NYCDOE). Academic outcomes in math and English Language Arts (ELA) were measured by the New York State tests that are conducted in the winters of third through eighth grade. The math portion of the exam includes questions on number sense and operations, algebra, geometry, measurement, and statistics. The section on ELA assesses students on information and understanding, literary response and expression, and critical analysis and evaluation. The study authors constructed measures of absenteeism and matriculation using NYCDOE data. Absenteeism was measured as the total number of absences a student accumulated during the first 180 days of the school year (after the first 180 days, NYCDOE no longer collects absence data from schools). Matriculation was used as an indicator for whether a student was “on-time” given their expected grade. Subgroup analysis was conducted by grade level.

For more information about Harlem Children’s Zone, please see the program’s website: https://hcz.org/

Subgroup Analysis

Dobbie and Fryer (2010) looked at the effects of being admitted (lottery winners) to Promise Academy by grade level. They also examined the effects of enrolling in the charter school. Lottery winners were 11.9 percent more likely to be performing on grade level in math in the sixth grade, 16.3 percent more likely in the seventh grade, and 27 percent more likely in eighth grade, compared with lottery losers across the three grades. These differences were statistically significant. The treatment-on-treated (TOT) analysis, which looked at the effects of actually attending the charter middle school, showed a similar pattern. There was a convergence in test scores between students who attended the Promise Academy middle school and the average white student in NYC. In other words, Promise Academy students had nearly caught up to the average white student in New York City public schools in math test scores after 3 years of schooling. The TOT analysis found that lottery winners who enrolled in the charter middle school scored 0.338 standard deviations higher in sixth-grade math, 0.371 higher in seventh-grade math, and 0.745 higher in eighth-grade math, compared with the control group. The results were statistically significant for all 3 years.

As with math scores, lottery winners, losers, and the average African American student in NYC had similar ELA scores, which were roughly 0.65 standard deviations behind the average white student in the city. ITT analysis showed there were no significant differences in ELA scores between the lottery winners and losers until eighth grade. In the eighth grade, however, Promise Academy students had significantly higher scores than the control group, although the treatment effect was not as strong on ELA scores as on math scores (i.e., lottery winners scored 0.196 standard deviations higher in eighth grade). The TOT analysis found that scores for students enrolled in Promise Academy were

These sources were used in the development of the program profile:

These sources were used in the development of the program profile:

Following are CrimeSolutions-rated programs that are related to this practice:

Targeted Truancy Interventions

These interventions are designed to increase attendance for elementary and secondary school students with chronic attendance problems. The practice is rated Effective for improving attendance.

Evidence Ratings for Outcomes

Education - Attendance/truancy

Mentoring for Youth Development

This practice provides youth with a positive and consistent adult or older youth relationship to promote healthy youth development and social functioning and to reduce risk factors. The practice is rated Effective in reducing delinquency and improving educational outcomes; Promising in improving psychological outcomes and cognitive functioning; and No Effects in reducing substance use.

Evidence Ratings for Outcomes

	Crime & Delinquency - Multiple crime/offense types
	Education - Multiple education outcomes
	Mental Health & Behavioral Health - Psychological functioning
	Mental Health & Behavioral Health - Cognitive functioning
	Mental Health & Behavioral Health - Social functioning
	Drugs & Substance Abuse - Multiple substances

Dropout Prevention Programs

School- or community-based programs targeting frequently absent students or students at risk of dropping out of school. These programs are aimed at increasing school engagement, school attachment, and the academic performance of students, with the main objective of increasing graduation rates. The practice is rated Effective for reducing rates of school dropouts, and rated Promising for improving test scores/grades, graduation rates, and attendance.

Evidence Ratings for Outcomes

	Education - Dropout
	Education - Academic achievement/school performance
	Education - Graduation
	Education - Attendance/truancy

Why might a practice's outcome ratings differ from the ratings of specific programs encompassed by that practice?