FAQ Listing

For programs, Study Reviewers analyze the most rigorous evaluation research available to determine whether there is evidence that the program achieves its goal(s). Up to three studies, representing the most rigorous evaluation research available, are selected to comprise the program’s evidence base. In some cases, a fourth study may be included in the review if Study Reviewers feel it is necessary to accurately assess a program. The reviewers use a standard Scoring Instrument for each study and assign scores across four dimensions:

• Program’s Conceptual Framework
• Study Design Quality
• Study Outcomes
• Program Fidelity

The individual study scores are then aggregated to determine the final evidence rating.

Read more about the Program Review and Rating from Start to Finish or see the Scoring Instrument.

Potential justice-related programs and practices are identified through:

• Literature searches of relevant databases, journals, and publications; and
• Nominations from experts, practitioners, or others.

Read more about how to Nominate a Program or Practice for CrimeSolutions.

Programs with evidence that is inconclusive as to whether they achieve justice-related outcomes do not receive an evidence rating on CrimeSolutions. However, CrimeSolutions periodically updates a static list of programs that have been reviewed by Study Reviewers, but not assigned an evidence rating due to lack of evidence. A program is placed on the inconclusive evidence list if the study (or studies) reviewed received only Class 5 study ratings indicating that there were significant limitations in the study design such that it was not possible to establish a causal relationship to the program’s justice-related outcomes.

A practice is placed on the inconclusive evidence list if the meta-analysis (or meta-analyses) reviewed received only Class 5 outcome ratings indicating there were significant limitations in the study design of the included evaluations that it was not possible to establish a causal relationship to the practice's justice-related outcomes.

Learn more about Programs and Practices Identified but Not Rated.

Before the evidence for a program or practice is reviewed and rated, the program or practice is screened to determine whether it meets CrimeSolutions' criteria for inclusion on the Web site. This includes a thorough review of the purpose of the program or practice and the strength of the available evidence. Read more about: Program Review and Rating from Start to Finish or Practice Review and Rating from Start to Finish.

At least two Study Reviewers independently assess the evaluation research for each program and the meta-analysis research for each practice under consideration by CrimeSolutions. All Study Reviewers are subject matter and research methodology experts who complete training and certification prior to reviewing studies. Lead Researchers select the evidence base, coordinate the review process, match programs with Study Reviewers based on expertise, and resolve any scoring discrepancies prior to a program being assigned a final evidence rating. Study Reviewers use a standard Scoring Instrument to determine whether a program achieves its goals.

Read more about CrimeSolutions Researchers and Reviewers or the Program Review and Rating from Start to Finish or the Practice Review and Rating from Start to Finish.

The information and evidence ratings included on CrimeSolutions are not static. As additional programs and practices are identified and new research becomes available, CrimeSolutions content will be updated and supplemented to reflect the most current programmatic and research information available. We also rely on users to provide us with critical feedback about the CrimeSolutions website itself. What is useful and what is not? What additional features would you like to see on the site in the future? CrimeSolutions users are welcome to Submit Feedback.

Specific concerns about evidence ratings or information contained within CrimeSolutions may also be submitted via the Submit Feedback online form. If necessary, changes to the information presented on the site will be made, per the processes outlined in Inquiring About or Appealing an Evidence Rating.

CrimeSolutions does not provide direct services.  However where available, we list contact information in the profile for each program and practice. If you are unable to locate this information on our site, please contact us with the title of the program or practice you are interested in, and we will do our best to provide the contact information you are seeking.

Although reviews are conducted on an ongoing basis, CrimeSolutions is not an exhaustive list of all justice-related programs and practices. If you cannot find the program or practice you are looking for, then you can do the following:

• Search for programs or practices that address similar outcomes using filters on the program and practice lists.
• Review the list of programs that have been considered but not rated . If the program is on this list, it means that the evaluation(s) associated with that program did not meet our criteria for review or inclusion.
• Review the list of practices that have been considered but not rated . If the practice is on this list, it means that the evaluation(s) associated with that program did not meet our criteria for review or inclusion.
• Nominate that program or practice in question for review.

Historically, for every article reviewed:

• 8% resulted in an identified program.
• 3% resulted in a review.
• 1% resulted in a finding of inconclusive evidence.
• 2% resulted in a rating.

For every program identified:

• 34% resulted in a review.
• 20% resulted in a rating.
• 45% were screened out.
• 20% are on hold.

For every program reviewed:

• 42% resulted in a finding of inconclusive evidence.
• 59% resulted in a rating.

For every program rated:

• 27% resulted in a rating of No Effects.
• 59% resulted in a rating of Promising.
• 14% resulted in a rating of Effective.

No. A listing on CrimeSolutions does not constitute an endorsement of particular programs or practices. Furthermore, it is not intended to replace or supersede informed judgment or innovation. CrimeSolutions recognizes that rigorous evaluation evidence is one of several factors to consider in justice programming, policy, and funding decisions. OJP also recognizes the importance of encouraging and supporting innovative approaches that may not yet have extensive evidence of effectiveness. 

CrimeSolutions classifies programs  and practices in three levels: “Effective,” “Promising” and “No Effects” based on the strength of the evaluation research that indicates a program or practice achieves its goals (i.e., its justice-related outcomes).

Evidence Ratings

Evidence Rating	One-Study Icon	Multiple-Study Icon	Description
Effective			Program is likely to result in the intended outcomes. An Effective practice outcome indicates, on average, there is strong evidence that implementing a program encompassed by the practice will achieve the intended outcome.
Promising			Program may result in the intended outcomes. A Promising practice outcome indicates, on average, there is some evidence that implementing a program encompassed by the practice will achieve the intended outcome.
No Effects			Programs or practices have strong evidence that the program did not have the intended effects or had harmful effects when trying to achieve justice-related outcomes. While programs and practices rated No Effects may have had some positive effects, the overall rating is based on the preponderance of evidence.

A single-study icon is used to identify programs that have been evaluated with a single sample. A program with multiple publications listed in the evidence base may receive a single-study icon because:

• The publications resulted from a study based on a single sample.
• the studies that comprised the program’s evidence base did not demonstrate effects in a consistent direction.

A multiple studies icon is used to represent a greater extent of evidence supporting the evidence rating. The icon depicts programs that have more than one study in the evidence base demonstrating effects in a consistent direction.

Read more about evidence ratings at About CrimeSolutions.