Yes. Programs and practices from other countries are included in CrimeSolutions. These must have English language citations and are otherwise subject to the same criteria used for programs and practices located within the United States.
We appreciate your interest in becoming involved with CrimeSolutions; but at this time, we are not looking for additional reviewers at this time.
For practices, Study Reviewers analyze all of the eligible and applicable meta-analysis studies to determine whether there is evidence that the practice achieves its goal(s). The Study Reviewers use a standard Scoring Instrument to determine the quality of each meta-analysis and to determine the internal validity of each outcome within the meta-analyses. The quality rating for each meta-analysis is combined with the internal validity rating of each outcome and the statistical significance of the effect size to determine a final evidence rating. If there are multiple meta-analyses that assess a single outcome, the individual outcome ratings from each meta-analysis are then aggregated to determine the final rating for that outcome.
For practices, individual evidence ratings are assessed for each outcome. Therefore, a single practice may have multiple evidence ratings if it addresses multiple outcomes. That is, one practice may be rated “Effective” for achieving one outcome, and “Promising” for achieving another outcome.
For programs, Study Reviewers analyze the most rigorous evaluation research available to determine whether there is evidence that the program achieves its goal(s). Up to three studies, representing the most rigorous evaluation research available, are selected to comprise the program’s evidence base. In some cases, a fourth study may be included in the review if Study Reviewers feel it is necessary to accurately assess a program. The reviewers use a standard Scoring Instrument for each study and assign scores across four dimensions:
- Program’s Conceptual Framework
- Study Design Quality
- Study Outcomes
- Program Fidelity
The individual study scores are then aggregated to determine the final evidence rating.
Potential justice-related programs and practices are identified through:
- Literature searches of relevant databases, journals, and publications; and
- Nominations from experts, practitioners, or others.
Read more about how to Nominate a Program or Practice for CrimeSolutions.
Programs with evidence that is inconclusive as to whether they achieve their intended outcomes do not receive an evidence rating on CrimeSolutions. However, CrimeSolutions periodically updates a static list of programs that have been reviewed by Study Reviewers, but not assigned an evidence rating due to lack of evidence. A program is placed on the inconclusive evidence list if the study (or studies) reviewed received only Class 5 study ratings indicating that there were significant limitations in the study design such that it was not possible to establish a causal relationship to the program’s justice-related outcomes.
A practice is placed on the inconclusive evidence list if the meta-analysis (or meta-analyses) reviewed received only Class 5 outcome ratings indicating there were significant limitations in the study design of the included evaluations that it was not possible to establish a causal relationship to the practice's justice-related outcomes.
Learn more about Programs and Practices Identified but Not Rated.
Before the evidence for a program or practice is reviewed and rated, the program or practice is screened to determine whether it meets CrimeSolutions' criteria for inclusion on the Web site. This includes a thorough review of the purpose of the program or practice and the strength of the available evidence. Read more about: Program Review and Rating from Start to Finish or Practice Review and Rating from Start to Finish.
At least two Study Reviewers independently assess the evaluation research for each program and the meta-analysis research for each practice under consideration by CrimeSolutions. All Study Reviewers are subject matter and research methodology experts who complete training and certification prior to reviewing studies. Lead Researchers select the evidence base, coordinate the review process, match programs with Study Reviewers based on expertise, and resolve any scoring discrepancies prior to a program being assigned a final evidence rating. Study Reviewers use a standard Scoring Instrument to determine whether a program achieves its goals.
Intervention materials, including Web sites, may state that a program or practice has been reviewed and posted on CrimeSolutions. After a profile has been posted on CrimeSolutions, interested parties may Submit Feedback to request the CrimeSolutions logo graphic. The logo may be used as a link to CrimeSolutions or in the materials related to the program or practice. However, the posting of a program summary on CrimeSolutions does not constitute an endorsement, promotion, or approval of the intervention by CrimeSolutions or NIJ.
CrimeSolutions recommends the following citation format for program/practice profiles and summaries:
Office of Justice Programs, National Institute of Justice. Name of program or practice. Retrieved [month, date, year profile was accessed], from CrimeSolutions, [URL of summary].