Screened-Out Meta-Analyses for Practices

We have identified thousands of programs and practices in our literature reviews, but not every intervention makes it onto CrimeSolutions with a rating.

We use rigorous research to inform you about what works in criminal justice, juvenile justice and crime victim services. We set a high bar for the scientific quality of the evidence used to assign ratings. Much of the evidence that we identify is screened out before going to review. See Why Meta-analyses Are Screened Out.

Following is a list of meta-analyses that were screened out during the initial review. 

Why Meta-Analyses Are Screened Out

A meta-analysis is screened out if it does not meet all of the criteria described below.

1. Intervention: The meta-analysis must include at least two program evaluations of the practice of interest.
2. Aggregation: The meta-analysis must report an aggregate effect size. That is, the meta-analysis must aggregate or average the effects of at least two studies into a single estimate of the effectiveness of the treatment.
3. Primary aim: The intervention that is the subject of the meta-analysis must focus on at least one of the following. The program or practice must:
   • Aim to prevent or reduce crime or delinquency, overt problem behaviors (such as aggression or gang involvement), or important risk factors for crime or delinquency (such as school failure);
   • Aim to prevent, intervene, or respond to victimization;
   • Aim to improve justice systems or processes; and/or
   • Target an offender population or an at-risk population (that is, individuals who have the potential to become involved in the justice system).
4. Literature search: The meta-analysis must be based on a comprehensive and systematic search of the literature. To meet this criterion at least minimally, the search must include both of the following components:
   • At least two sources for identifying potentially eligible research must be used. Sources include, for example, different electronic bibliographies, Google type internet searches, hand searches of journals, contacting relevant researchers in the field, and reviewing the references in pertinent studies and prior reviews.
   • There is some indication that unpublished or “grey” literature was sought during the search, whether or not any unpublished research reports eventually met the eligibility criteria for inclusion in the meta-analysis. Unpublished and grey literature are defined here as literature that is not controlled by commercial publishers, including for example, conference presentations, government and non-governmental agency technical reports, and theses/dissertations. Meta-analyses that explicitly state that only published or peer-reviewed research was included are ineligible. Meta-analyses that explicitly or actively excluded unpublished or non-peer-reviewed research are ineligible.
5. Primary outcomes: The meta-analysis must report results on at least one eligible outcome, defined by CrimeSolutions to include:
   • Crime, delinquency, or overt problem behaviors (including aggression, gang involvement, or substance use), which may be presented as individual behaviors, community-level behaviors, crime rates, etc.;
   • Victimization;
   • Justice system practices or policies; or,
   • Risk factors for crime and delinquency, including school failure, psychological problems or mental illness, etc.
     
     Eligible outcomes must fall completely within one of the single construct categories listed below and cannot be combinations of constructs across more than one category. That is, constructs that are combinations of education and mental health outcomes would not be eligible unless the education and mental health outcomes were also reported separately.
6. Control groups: The studies included in the meta-analysis must have appropriate control groups. Specifically, one of the following two conditions must be met:
   • An appropriate counterfactual condition is required for all included studies in a meta-analysis, or
   • In meta-analyses that include studies without control groups, results for controlled studies are reported separately from those of uncontrolled studies.
7. Reporting of results: The meta-analysis must report a common quantitative index of results that represents the magnitude of the treatment effect (i.e., effect sizes) across included studies.
8. Combining relationships: If the meta-analysis reports an average mean effect size calculation (for all studies or for subgroups of studies), only effect sizes in the same metric and measuring the same type of relationship are averaged together. This criterion has to do with the kinds of relationships and effect sizes that are combined rather than the types of programs that are combined. A meta-analysis that includes a diverse group of programs would not necessarily be problematic here. However, a meta-analysis that combined standardized mean difference effect sizes for treatment versus control comparisons with pretest-posttest change effect sizes would be ineligible unless the different effect size means were reported separately.
9. Publication date: At least 50% of the studies included in the meta-analysis must be published or available on or after 1980.
10. Age of samples: All of the studies included in the meta-analysis must involve offender participants that are either all juvenile or all adult. If the meta-analysis includes studies that involve both juveniles and adults, mean effect sizes must be reported separately for the juvenile and adult studies. Meta-analyses that include studies that themselves combine juveniles and adults in the same programs without providing any options for splitting the juvenile and adult samples would not be eligible. The age criterion does not apply for: (1) meta-analyses of practices or procedures that focus on areas, hot spots, surveillance, and the like, which do not typically involve identifiable offenders; or for (2) victim interventions.

Return to text.