Study 1
Barnoski (2004) used a pseudo-random assignment procedure to assess the effect of Washington State Aggression Replacement Training (WSART) on juvenile recidivism, at the 18-month follow up. The study sample included a total of 1,229 adjudicated youth (n = 704 treatment group; n = 525 comparison group) within 26 juvenile courts in Washington State.
All juveniles were assessed for eligibility by court staff, using prescreen and full versions of the Washington State Juvenile Court Assessment (WSJCA). This instrument determines a youth’s level of risk by prescreen scores on criminal and social history; the full assessment is then given to moderate- to high-risk youth only. The full instrument is organized into nine domains: 1) school, 2) free time, 3) employment, 4) relationships, 5) family (current and prior), 6) drug/alcohol, 7) mental health, 8) antisocial attitudes, and 9) skills (another score was developed to measure aggression). For each domain, a risk or protective factor score was computed. To be eligible for WSART, youth had to be considered moderate or high risk and have at least one of the following: 1) a criminal history risk factor score of at least 1 for a weapon, violent misdemeanor, or felony conviction; 2) an aggression risk factor score of at least 2 out of a possible 13; 3) an attitudes/behavior risk factor score of at least 7 out of a possible 28; or 4) or a skills risk factor score of at least 9 out of a possible 36. Youth who met these criteria were assigned to WSART until the program reached capacity (i.e., either all therapists had full caseloads, or all sessions were full). Court staff assigned the remaining eligible youth to the comparison group. The comparison group did not participate in WSART, but received the usual juvenile court services. Through this approach, the study authors created a sample with similar risk and protective factors.
The average age at adjudication across all courts was 15.2 for youth in the WSART program and 15.5 years for youth in the comparison group. The majority of the sample was male (80 percent in the WSART treatment group and 81 percent in the comparison group). Multivariate statistical techniques were used to control for systemic differences between the program and comparison groups on key characteristics from the WSJCA (such as gender, age, and domain risk and protective factor scores).
Outcomes of interest included recidivism, measured as felonies, violent felonies, and misdemeanor and felonies. Data on recidivism were collected from the state’s Administrative Office of the Courts and the Department of Corrections. To analyze the outcomes of interest, logistic regression was used to control for systemic differences between the treatment and comparison groups. Additional analysis examined how competent delivery of the WSART program affected the intended outcomes. A subgroup analysis was conducted to examine the effects on recidivism reduction by courts that were competently delivering WSART (n = 21), compared with courts that were not competently delivering the program (n = 5).
Study 2
Knoth, Wanner, and He (2019) assessed the effects of the Washington State ART (WSART) program on felony, misdemeanor, and violent recidivism in 13,070 juvenile court youth. Youth were identified through matches between the Juvenile Assessment Research Database and court cases from the Criminal History Database. Juvenile court youth sentenced with local sanctions were screened using the Washington State Positive Achievement Change Tool (PACT). For youth with multiple court cases who therefore received multiple PACT assessments, one of those assessments was randomly selected to be included in the analyses. If the randomly chosen assessment included youth’s WSART participation (but not necessarily completion) between January 1, 2006, and June 30, 2016, they were included in the treatment group. If it did not, the youth were placed in the comparison group. Youth were eligible for WSART if they were considered to be moderate or high risk and had one of the following: 1) a current or prior adjudication for a weapon, violent misdemeanor, or felony conviction; 2) a score of 2 or above on the Aggression domain; 3) a score of 5 or above on the Attitudes/Behavior domain; or 4) a score of 4 or above on the Social Skills domain. The comparison group was considered “treatment-as-usual,” as it included youth who were eligible for WSART but who did not start the program during the same 10-year period and may have started another evidence-based practice. The final matched sample included 6,535 treatment youth and 6,535 comparison youth.
To establish a sample that was similar on all characteristics (such as demographics, offense characteristics, and risk information), the authors used propensity score matching to identify youth in the comparison group who were most similar to youth in the treatment group and match them on observable characteristics, besides participation in WSART. The majority of youth in the WSART treatment group were 15 to 17 years old (77.0 percent) and male (74 percent). With regard to race/ethnicity, the treatment group was white (63.2 percent), Hispanic (15.9 percent), Black/African American (12.8 percent), Asian/Pacific Islander/Native Hawaiian (2.8 percent), American Indian/Alaska Native (3.7 percent), and other race/ unknown (1.6 percent). At the time of the assessment, most comparison group youth were also 15 to 17 years old (77.6 percent) and male (74.3 percent). With regard to race/ethnicity, the comparison group was white (63 percent), Hispanic (16.1 percent), Black/African American (12.8 percent), Asian/Pacific Islander/Native Hawaiian (2.8 percent), American Indian/Alaska Native (3.7 percent), and other race/unknown (1.6 percent). There were no statistically significant differences between the groups on baseline characteristics.
The primary outcome of interest was recidivism, including any recidivism, misdemeanor recidivism, felony recidivism, or violent recidivism. Outcomes were assessed through regression analyses using the matched sample. Recidivism data were collected from the Washington State Institute for Public Policy’s Criminal History Database, which combines data from several state agencies, including court data, residential confinement data, and incarceration in state prisons and community supervision data. Additional subgroup analyses explored the effects of risk level, gender, age, race/ethnicity, trainer competence, and program completion on program outcomes. For each of these subgroups, new models identified matched samples of treatment and comparison group youth with a similar likelihood of participating in WSART.