FAQ Listing

The information and evidence ratings included on CrimeSolutions are not static. As additional programs and practices are identified and new research becomes available, CrimeSolutions content will be updated and supplemented to reflect the most current programmatic and research information available. We also rely on users to provide us with critical feedback about the CrimeSolutions website itself. What is useful and what is not? What additional features would you like to see on the site in the future? CrimeSolutions users are welcome to Submit Feedback.

Specific concerns about evidence ratings or information contained within CrimeSolutions may also be submitted via the Submit Feedback online form. If necessary, changes to the information presented on the site will be made, per the processes outlined in Inquiring About or Appealing an Evidence Rating.

CrimeSolutions does not provide direct services.  However where available, we list contact information in the profile for each program and practice. If you are unable to locate this information on our site, please contact us with the title of the program or practice you are interested in, and we will do our best to provide the contact information you are seeking.

Although reviews are conducted on an ongoing basis, CrimeSolutions is not an exhaustive list of all justice-related programs and practices. If you cannot find the program or practice you are looking for, then you can do the following:

• Search for programs or practices that address similar outcomes using filters on the program and practice lists.
• Review the list of programs that have been considered but not rated . If the program is on this list, it means that the evaluation(s) associated with that program did not meet our criteria for review or inclusion.
• Review the list of practices that have been considered but not rated . If the practice is on this list, it means that the evaluation(s) associated with that program did not meet our criteria for review or inclusion.
• Nominate that program or practice in question for review.

Historically, for every article reviewed:

• 8% resulted in an identified program.
• 3% resulted in a review.
• 1% resulted in a finding of inconclusive evidence.
• 2% resulted in a rating.

For every program identified:

• 34% resulted in a review.
• 20% resulted in a rating.
• 45% were screened out.
• 20% are on hold.

For every program reviewed:

• 42% resulted in a finding of inconclusive evidence.
• 59% resulted in a rating.

For every program rated:

• 27% resulted in a rating of No Effects.
• 59% resulted in a rating of Promising.
• 14% resulted in a rating of Effective.

No. A listing on CrimeSolutions does not constitute an endorsement of particular programs or practices. Furthermore, it is not intended to replace or supersede informed judgment or innovation. CrimeSolutions recognizes that rigorous evaluation evidence is one of several factors to consider in justice programming, policy, and funding decisions. OJP also recognizes the importance of encouraging and supporting innovative approaches that may not yet have extensive evidence of effectiveness. 

CrimeSolutions classifies programs  and practices in three levels: “Effective,” “Promising” and “No Effects” based on the strength of the evaluation research that indicates a program or practice achieves its goals (i.e., its justice-related outcomes).

Evidence Ratings

Evidence Rating	One-Study Icon	Multiple-Study Icon	Description
Effective			Programs or Practices have strong evidence indicating they achieve justice-related outcomes.
Promising			Programs or Practices have some evidence indicating they achieve justice-related outcomes.
No Effects			Programs or practices have strong evidence that the program did not have the intended effects or had harmful effects when trying to achieve justice-related outcomes. While programs and practices rated No Effects may have had some positive effects, the overall rating is based on the preponderance of evidence.

A single-study icon is used to identify programs that have been evaluated with a single sample. A program with multiple publications listed in the evidence base may receive a single-study icon because:

• The publications resulted from a study based on a single sample.
• the studies that comprised the program’s evidence base did not demonstrate effects in a consistent direction.

A multiple studies icon is used to represent a greater extent of evidence supporting the evidence rating. The icon depicts programs that have more than one study in the evidence base demonstrating effects in a consistent direction.

Read more about evidence ratings at About CrimeSolutions.

A Program is a specified set of activities combined according to precise guidance in order to achieve a specific purpose. Program profiles on CrimeSolutions tell us whether a specific program was found to achieve its goals when it was carefully evaluated. The results apply to the exact set of activities and procedures used for that one program as it was implemented at the time of evaluation. Thus the program profile tells us that a program is likely to produce the observed result if implemented in exactly the same way.

A Practice is a general category of programs, strategies, or procedures that share similar characteristics with regard to the matters they address and how they do it. Practice profiles tell us about the average results from multiple evaluations of similar programs, strategies, or procedures. The programs, strategies, or procedures within a practice are similar because they share certain defining characteristics that are described for each practice profile on CrimeSolutions. Thus, practice profiles tell us the most typical results across multiple evaluations.

Sometimes a practice on CrimeSolutions will receive multiple evidence ratings because there is sufficient evidence available to draw conclusions about multiple outcomes addressed by that practice. This is because the ratings for practices are based on a larger amount of evidence using techniques of meta-analysis to examine the findings of numerous studies. With more evidence, it is possible to draw conclusions at a more detailed level of analysis. Programs on CrimeSolutions receive a single overall evidence rating because they are often based on only one evaluation study (although up to three may be included). With a smaller evidence base to inform conclusions, CrimeSolutions does not provide outcome-level ratings for programs.

CrimeSolutions includes “No Effects” programs and practices to inform policy makers and practitioners about the current status of available evaluation evidence before planning or implementing similar efforts.

No Effects programs and practices strong evidence that the program did not have the intended effects or had harmful effects when trying to achieve justice-related outcomes. In cases where negative effects were found, CrimeSolutions profiles for programs and practices will identify and describe observed negative effects.

While programs and practices rated No Effects may have had some positive effects, the overall rating is based on the preponderance of evidence.

Read more about Program Review and Rating from Start to Finish Tips for using CrimeSolutions.

You may notice that a practice's outcome ratings may not be the same as the programs that are encompassed by that practice and vice versa. For example, some practices have outcomes that are rated Effective but programs listed in the Related Programs section are rated Promising or No Effects.

This can happen because practice ratings reflect an assessment of the average effectiveness of a specific characteristic(s) common across programs, strategies, or procedures. Practice ratings do not take into account variations in implementation or other program-specific factors. If such variations impact the effectiveness of the essential practice characteristic(s), practice and program ratings may diverge as the program-specific effectiveness may differ from the average effectiveness across multiple programs.

 A program with multiple publications listed in the evidence base may receive a single-study icon because:

• The publications resulted from a study based on a single sample.
• the studies that comprised the program’s evidence base did not demonstrate effects in a consistent direction.