Practice Profile: After-School Programs

Taheri and Welsh (2016) evaluated the impact of after-school programs (ASPs) on delinquency. To be eligible for inclusion in the meta-analysis, ASPs had to be the focus of the intervention. ASPs were defined as organized programs that were implemented after school hours, which targeted youth or adolescents who would have otherwise been unsupervised during this time. Furthermore, the study had to include (1) an outcome measure of delinquency or antisocial behavior; (2) focus on children or adolescents; and (3) use an experimental design or a quasi-experimental design with control groups. Both published and unpublished studies were eligible for inclusion and could include international research; however, studies written in English, German, and French were given priority.  Finally, to be included in the meta-analysis, each study had to provide the necessary information to calculate effect sizes.

Using this inclusion criteria, a variety of search strategies were used to identify eligible studies. Keywords, which included but were not limited to ASP, community crime prevention, youth development, and delinquency, were used to search such databases as Criminal Justice Abstracts, National Criminal Justice References Service Abstracts, Educational Resources Information Clearinghouse, Google Scholar, Dissertation Abstracts, and Academic Search Premiere. References of key literature reviews and evaluation reports of ASPs were also searched. Finally, organizational databases were searched, and leading researchers in relevant fields were contacted to determine if there were any other potentially eligible studies.

The search yielded 59 potential studies. Of these, 12 studies met the inclusion criteria and were included in the meta-analysis. Of the 12 studies, 3 used random assignment and 9 used quasi-experimental designs. The included studies had a total sample size of 19,986, with an average age of 9–16 years old. The majority of the studies were conducted in the United States (N=10), one study was conducted in Canada, and one study was conducted in Sweden. The included studies could be categorized into the following ASP intervention types: academic (N=5), skills training/mentoring (N=5), and recreation (N=2).  The demographics of the sample were not reported.   

To analyze the impact of after-school programs on delinquency, a random effects model was used to account for the heterogeneity across studies. Program effect sizes were weighted on the variance of the effect size and the study sample size. Effect sizes were calculated for each outcome and then averaged to create a mean effect size for each outcome, also known as a standardized mean difference. If an odds ratio or partial r were reported, these were converted to d using formulae in Lipsey and Wilson (2001).

Durlak, Weissberg, and Pachan (2010) evaluated whether ASPs enhanced the personal and social skills of children. To be eligible for inclusion, studies had to evaluate an ASP, defined as a structured program offering activities that occurred during the school year (yet outside of school hours) and were supervised by adults. Furthermore, eligible ASPs had to target the development of personal or social skills of youth, aged 5–18 years old. Personal and social skills could include the development of skills such as problem-solving, self-control, self-efficacy, self-esteem, leadership, conflict resolution, and decision-making. It is important to note that evaluations that only focused on improving academic performance and school attendance were excluded. Both published and unpublished studies were eligible for inclusion and could include international research. Finally, each study had to include a control group and provide the necessary information to calculate effect sizes.

To identify studies, keywords were used to search the following databases: ERIC, PsycInfo, Medline, and Dissertation Abstracts. The American Journal of Community Psychology, Journal of Community Psychology, and Journal of Counselling Psychology were also hand-searched for eligible studies. Finally, reference lists of prior ASP reviews were searched in the Harvard Family Research Project database, which houses after-school program research. This search strategy was limited to studies that ranged from January 1, 1980, to December 31, 2007.

The search yielded 68 studies that were eligible for review. Over half (N=44) were unpublished dissertations or technical reports. In terms of methodological design, 24 studies used a randomized design, whereas 44 used a quasi-experimental design. Furthermore, approximately 46 percent of the programs targeted elementary students, 37 percent targeted junior high students, and 9 percent served high school students. It is important to note that approximately 9 percent of the studies did not indicate the grade level of the target audience. In terms of ethnicity, the samples comprised African Americans, Latinos, Asians or Pacific Islanders, and American Indians. In terms of socioeconomic status, 25 percent of the studies targeted predominately low-income families, 19 targeted mixed-income families, and 45 percent of the studies did not provide information on socioeconomic status.

For the meta-analysis, the following eight outcomes were measured: child self-perceptions, bonding to school, positive social behaviors, problem behaviors, drug use, achievement test scores, school grades, and school attendance. However, for the purposes of the CrimeSolutions review, achievement test scores were not scored. Self-perceptions included measures of self-esteem, self-efficacy, and self-concept. Bonding to school was operationalized as positive feelings and attitudes toward school and teachers. Positive social behaviors measured interactions with others such as cooperation, leadership, and appropriate responses to conflict. Problem behaviors measured whether youth could control their behavior in social situations. Drug use was measured by youth self-report of alcohol, marijuana, or tobacco use. School grades was measured by youth’s grades in primary school subjects or their grade point averages. School attendance was measured by the number of days that students attended school.

Overall, an inverse variance random effects model was used in the analyses to determine the effectiveness of ASPs. Effect sizes were calculated for each outcome and then averaged to create a mean effect size for each outcome, also known as a standardized mean difference.

Lauer and colleagues (2006) evaluated the impact of ASPs and summer school programs in assisting at-risk students in both reading and mathematics. However, for the purpose of the CrimeSolutions review, only the impact of the ASPs was of interest, and thus presented in this profile. To identify studies, specific keywords related to after-school programs were used to search the ERIC and PsycINFO databases. The search strategy was executed in May 2003, according to the following parameters: 1) the study must have been conducted between 1985 and 2003, 2) the study could not include college students, and 3) the study must have been written in the English language. Literature reviews on after-school programs and organizational websites with additional studies on ASPs were reviewed. Both published and unpublished studies were eligible for inclusion (although studies had to be published in the English language and implemented in the United States).

To be eligible for inclusion, studies must have evaluated an ASP delivered to students, in grades K–12, who were at risk for failing out of school. At-risk students were defined as 1) students who were underperforming on standardized tests, classroom activities, or course grades; or 2) students who fit characteristics that were typically associated with low student achievement/dropout. Eligible studies had to include an assessment of student achievement in reading, mathematics, or both. Eligible studies were limited to experimental or quasi-experimental designs that compared youth who participated in an after-school program with a comparison group of youth who did not participate. Finally, each study had to include the necessary information to calculate effect sizes. It is important to note that studies were not included if the program was developed for, and targeted, a specific population of students (e.g., special education students, English language learners).

Following full-text screening of relevant articles, 15 eligible studies examined ASPs. All 15 studies were quasi-experimental designs, and included outcomes related to reading; however, only 9 studies included outcomes related to mathematics. The eligible reading and mathematics studies included a total treatment sample size of 7,246 students, ranging from K–8 grade, with the majority characterized as being low performing, of low socioeconomic status, and from a minority group.

A fixed-effects model was used to determine the impact of after-school programs assisting at-risk students in both reading and mathematics. Effect sizes were calculated for each outcome and then averaged to create a mean effect size for each outcome, also known as a standardized mean difference.