Study 1
Solomon and colleagues (1988) reviewed the impact of the Caring School Community (CSC) program on elementary school children in kindergarten through fourth grade in a San Francisco Bay (Calif.)–area suburban school district using a quasi-experimental design. The researchers divided six schools within the district into two roughly equivalent groups of three on the basis of size, faculty experience, sociodemographic characteristics, achievement levels, and participation interest. One of these groups was randomly selected to receive the program, while the other served as a comparison group.
The evaluation looked at a total of 67 classrooms, including 37 classrooms in the treatment group and 30 classrooms in the comparison group. CSC was provided to children in two cohorts, each of which began the project in kindergarten—the first cohort received CSC in the 1982–83 school year, and the second cohort received CSC in the 1985–86 school year. Each cohort in the program schools and the parallel cohort in the comparison schools took part in the same data collection activities every year. The evaluation concentrated on the findings from the first cohort.
Assessments of classroom practices, activities, and student behaviors were conducted using teacher questionnaires, third grade individual student interviews, structured small-group tasks, and classroom observations. A sign system instrument and a rating system instrument were also developed and used. Observers used the sign system after watching the classroom for a 2-minute period, while a rating system was used after completion of a 2-hour observation visit. After the first year of use (1982–83), the instruments were revised.
The intervention lasted 5 years, with 350 students in the first cohort and 165 students remaining for the full evaluation (approximately half in the treatment group and half in the control group). In each of the 5 years of the evaluation, all of the classrooms were observed during eight separate 2-hour visits. Classroom observers were trained and not aware of the status (treatment or comparison) of any of the schools.
There were some limitations to the study. The observational instruments were changed after the first year. Other limitations include multiple treatment interferences, obtrusive testing, secular trends, and intervening events. Finally, students from different schools were used, but no information is given on other initiatives or activities in any of these schools that may affect outcome behaviors. No subgroup analyses were conducted.
Study 2
Battistich and colleagues (2000) conducted a quasi-experimental study of the CSC program using a convenience sample from six school districts for a total of 24 elementary schools. Of the schools, 12 were located on the West Coast, 4 were in the South, 4 were in the Southeast, and 4 were in the Northeast of the United States. They include urban, suburban, and rural school districts and serve diverse but roughly equivalent populations.
Twelve schools were selected to receive the intervention on the basis of faculty interest and perceived likelihood of being able to implement the CSC program. Twelve comparison schools were matched to intervention schools on school size and student characteristics. A total of 5,500 students were included in their evaluation: 2,250 students in the treatment group and 2,250 students in the comparison group. Assessments were completed at the baseline during the 1991–92 school year before the introduction of the CSC program. The researchers used a 36-month follow-up period.
Student self-reports were used to measure the outcomes. Assessments of drug use and other problem behaviors (including students’ use of cigarettes, alcohol, and marijuana and frequency of involvement in delinquent behaviors) were limited to students at the fifth or sixth grade at each of the schools. Students’ sense of school as a community was also measured by examining levels of student autonomy and influence in the classroom, classroom supportiveness, and school supportiveness. Adjustments were made for gender, ethnicity, and grade.
Classroom observations by observers blind to group assignment were conducted throughout the study. Beginning during the baseline year, four 90-minute observations of each classroom in the program and comparison schools were conducted each year. All observers were trained by the same project staff member; observers scored videotapes of classroom interactions and training visits throughout the school year to maintain reliability. Average overall observer agreement with the criterion scores was 75 percent over all 4 years of the study.
Program effects on all measures were examined using planned contrasts comparing linear changes from baseline at the program and matched comparison schools (between-group comparisons). Multivariate analysis of covariance was used, followed by the univariate planned contrasts. Two sets of analyses were also conducted for problem behavior data: a study-wide analysis to include all 24 schools and a high-change analysis which included five high-change schools (schools with high levels of implementation) and their five matched comparison schools. Subgroup analyses were conducted to determine program effects on five high-change schools, that is, the five schools that most closely followed the program instruction.
