A practice is a general category of programs, strategies or procedures that share similar characteristics with regard to the issues they address and how they address them. CrimeSolutions’ goal is to answer the question “How effective is this general practice on average across many evaluations?” This page helps show how we get there. See also CrimeSolutions Programs by the Numbers.
From Articles to Ratings - The Practice Funnel
How do we get to the practices that we rate?
- Screen articles and reports
- Identify meta-analyses of practice evaluations and screen those meta-analyses
- Send practices for review
- Assign ratings to one or more outcomes per practice and post to CrimeSolutions
Thousands of Articles Screened
While CrimeSolutions has rated over a hundred practices, tens of thousands of articles have been reviewed and screened. Practices are identified for potential inclusion on CrimeSolutions through literature searches of relevant databases, journals and publications, and nominations from the field.
Of the tens of thousands of articles screened:
- 1.55% resulted in a meta-analysis being identified
- .42% resulted in a meta-analysis being reviewed.
- .02% resulted in a finding of inconclusive evidence.
- .40% resulted in at least one practice outcome rating.
From Meta-Analysis Identified to Practices Rated
After meta-analyses are identified, research staff review materials to determine whether the goals of the evaluated practice fall within the scope of CrimeSolutions. To fall within the scope, the practice must:
- Aim to prevent or reduce crime, delinquency or related problem behaviors (such as aggression, gang involvement or school attachment);
- Aim to prevent, intervene or respond to victimization;
- Aim to improve justice systems or processes; and/or
- Target a population of persons convicted of a crime or an at-risk population (that is, individuals who have the potential to become involved in the justice system).
Prevention practices not explicitly aimed at reducing or preventing a problem behavior must apply to a population at risk for developing problem behaviors.
Historically, for every meta-analysis identified:
- 67% were screened out.
- 29% were reviewed.
- 4% are put on hold for possible future review.
For every practice reviewed:
Practice Outcome Ratings
While programs receive a single rating, CrimeSolutions rates individual outcomes for practices – a single practice can include multiple rated outcomes. As such, a practice may have an outcome rated Effective, and another rated No Effects.
As of November 2022, 121 rated practices have resulted in 396 rated outcomes. The average number of outcomes per practice is 3.3; the median number of outcomes is 3.
Historically, for every practice rated:
- 52% resulted in at least one outcome rating of No Effects.
- 68% resulted in at least one outcome rating of Promising.
- 42% resulted in at least one outcome rating of Effective.
For every outcome rated:
- 29% resulted in a rating of No Effects.
- 40%resulted in a rating of Promising.
- 31% resulted in a rating of Effective.
Practice Ratings by the Extent of Evidence - Single or Multiple Meta-Analyses
Practice ratings are based on all available and applicable meta-analysis. The strongest evidence can come from practices with multiple meta-analyses.
Of the practices rated:
- 65% are based on a single meta-analysis.
- 35% are based on multiple meta-analyses.
Practice Outcome Ratings by Topic
CrimeSolutions' literature review encompasses articles and reports based on meta-analyses incorporating evaluations of programs across the criminal and juvenile justice systems and victim services. However, the availability of rigorous meta-analyses of program evaluations is not equal across the eight primary topic areas covered by CrimeSolutions.
As many practices address multiple issues across the justice system and victim services, each practice may be assigned to multiple topics.