Inconclusive Practices List

Practices Reviewed but Not Rated
Inconclusive

CrimeSolutions uses rigorous research to inform practitioners and policy makers about what works in criminal justice, juvenile justice, and crime victim services. Because we set a high-bar for the scientific rigor of the meta-analyses we will use to do this, many meta-analyses do not meet our criteria and we are unable to rate the practice. 

For practices listed below, our reviewers determined that the available evidence was insufficient for a rating to be assigned. Practices are included in the list because of the insufficient research evidence about the practice, not because of any known weaknesses in the practices themselves.

Along with each practice listed are the references to the meta-analysis or meta-analyses that were reviewed and an indicator of why each study was rejected. See Reasons for Rejecting Meta-analyses.

Practices Reviewed but Not Rated
Practice TitleTopicMeta-analysis/Reason to Reject
Cognitive-Behavioral Therapy for Aggressive Behavior in YouthJuvenile Justice
Barnes, Tia N., Stephen W. Smith, and M. David Miller. 2014. "School-Based Cognitive-Behavioral Interventions in the Treatment of Aggression in the United States: A Meta-Analysis." Aggression and Violent Behavior 19:311–21.Inadequate Design Quality
Alternative Education ProgramsJuvenile Justice
Cox, Stephen M., William S. Davidson, and Timothy S. Bynum. 1995. "A Meta-Analytic Assessment of Delinquency-Related Outcomes of Alternative Education Programs." Crime & Delinquency 41(2):219–34.Low Internal Validity
Beverage Alcohol Price and Tax Levels on DrinkingCrime Prevention
Wagenaar, Alexander C., Matthew J. Salois, and Kelli A. Komro. 2009. "Effects of Beverage Alcohol Price and Tax Levels on Drinking: A Meta-Analysis of 1003 Estimates from 112 Studies." Addiction 104:179–90.Low Internal Validity
Community-Oriented Policing (COPS)Law Enforcement
Gill, Charlotte, David Weisburd, Cody W. Telep, Zoe Vitter, and Trevor Bennett. 2014. "Community-Oriented Policing to Reduce Crime, Disorder and Fear and Increase Satisfaction and Legitimacy Among Citizens: A Systematic Review." Journal of Experimental Criminology doi:10.1007/s11292-014-9210-yLow Internal Validity
Positive Youth Development ProgramsJuvenile Justice
Durlak, Joseph A., Rebecca D. Taylor, Kei Kawashima, Molly K. Pachan, Emily P. DuPre, Christine I. Celio, Sasha R. Berger, Allison B. Dymnicki, and Roger P. Weissberg. 2007. "Effects of Positive Youth Development Programs on School, Family, and Community Systems." American Journal of Community Psychology 39:269–86.Low Internal Validity
Preventing Repeat Domestic Burglary VictimizationCrime Prevention
Grove, Louise E., Graham Farrell, David P. Farrington, and Shane D. Johnson. 2012. Preventing Repeat Victimization: A Systematic Review. Stockholm, Sweden: The Swedish National Council for Crime Prevention.Low Internal Validity
Prison PrivatizationCorrections
Lundahl, Brad W., Chelsea Kunz, Cyndi Brownell, Norma Harris, and Russ Van Vleet. 2009. "Prison Privatization: A Meta-Analysis of Cost and Quality of Confine Indicators." Research on Social Work Practice 19(4):383–94.Low Internal Validity
Pratt, Travis C., and Jeff Maahs. 1999. "Are Private Prisons More Cost-Effective Than Public Prisons? A Meta-Analysis of Evaluation Research Studies." Crime & Delinquency 45(3):358–71.Inadequate Design Quality
Restorative Justice Conferencing for Adults OffendersCorrections
Strang, Heather, Lawrence W. Sherman, Evan Mayo-Wilson, Daniel Woods, and Barak Ariel. 2013. "Restorative Justice Conferencing (RJC) Using Face-to-Face Meetings of Offenders and Victims: Effects on Offender Recidivism and Victim Satisfaction. A Systematic Review." Campbell Systematic Reviews 12.Lacked Info on Statistical Significance
Sex Offender Registration and NotificationCorrections
Drake, Elizabeth K., and Steve Aos. 2009. Does Sex Offender Registration and Notification Reduce Crime? A Systematic Review of the Research Literature. Olympia, Wash.: Washington State Institute for Public Policy.Low Internal Validity
Training for Community SupervisionCorrections
Chadwick, Nick, Angela Dewolf, and Ralph Serin. 2015. "Effectively Training Community Supervision Officers: A Meta-Analytic Review of the Impact on Offender Outcome." Criminal Justice and Behavior:1-13. doi: 10.1177/0093854815595661Low Internal Validity
Date Created: August 14, 2020

Reasons for Rejecting Meta-Analyses

Inadequate Design Quality: meta-analyses can be rejected if they do not provide enough information about the design or have significant limitation in the design (for example, did not measure the methodological quality of included studies; did not properly weight the results of the included studies; etc.) such that it is not possible to place confidence in the overall results of the review.

Low Internal Validity: meta-analyses can be rejected if they have low internal validity, meaning that the overall mean effect size was based on results from included studies with research designs that are not free from threats that could potentially bias the effect estimate. Randomized controlled trials (RCTS) have the strongest inherent internal validity. The internal validity scores of meta-analyses will be lower as the proportion of the included studies with non-RCT designs (i.e., quasi-experimental designs) increases.

Lacked Sufficient Information on Statistical Significance: meta-analyses may be rigorous and well-designed, but they are rejected if they do not provide sufficient information to determine if the mean effect sizes were statistically significant or non-significant. The statistical significance of a mean effect size is needed to determine an outcome's final evidence rating.