Study 1
Johnson and colleagues (2005) used an elementary school population to examine the effects of the Success in Stages: Build Respect, Stop Bullying® program. A 3x3 experimental design was used, which crossed three experimental groups (one control and two treatments) with three assessments (including two treatment groups in the study allowed for replication of the intervention).
The sample of 1,807 fourth and fifth grade students came from 12 elementary schools located across the United States and in various types of communities (urban, rural, and suburban areas). Schools with higher percentages of students receiving free lunches were given preference for inclusion. Schools were matched on several key variables, including type of community, number of students receiving free lunches, and region of the country. Males comprised 51.1 percent of the total sample. A majority of students were white (59.4 percent), followed by Black (17.5 percent), and Hispanic (13.1 percent). Just over 40 percent of students were eligible for free lunches. An equal number of fourth and fifth grade students participated.
The control group received a pretest and three posttests, which included only assessment questions on roles related to bullying and stage of change. The two treatment groups received a pretest; up to three Internet-based sessions; guides for the school administrators, staff, and family; and three posttests. Treatment and control schools were given the pretest as soon as they were ready to participate. The treatment groups were asked to complete the final two Internet sessions a minimum of 30 days apart. The first posttests for the treatment groups were to be administered a minimum of 7 days after the last Internet session. The second posttests were to be given 4 months after the first posttest for all groups, and the final posttest was to be completed 8 months after the second posttest for all groups. All posttests were to be completed approximately 1 year from each group’s start date. The treatment sessions were completed within a 2-month window, ranging from 1 month to 3 months. The overall program length was 12 months when factoring in posttests. The CrimeSolutions review of this study focused on the outcomes for treatment group 1 (n = 662) compared with the control group (n = 506).
Outcomes on the percentages of students who were no longer categorized in the three roles related to bullying (bully, victim, or passive bystander) at posttest were measured, as was the percentage who were no longer participants in all three roles combined. The bullying-related roles were assessed as 1) Bully: treats other kids in mean ways and/or pushes, hits, or kicks other kids to hurt them; 2) Victim: kids treat them in mean ways and/or push, hit, or kick them to hurt them; 3) Passive bystander: lets kids be mean to other kids and lets kids push, hit, or kick others while doing nothing to prevent or discourage such behaviors. For each role, students were asked a single item to assess their stage of change for that role. Students who responded negatively for both of the behavior items related to a particular role were asked how long they had been acting this way to determine whether they were in the Action stage (meaning they had not participated in any of the three roles related to bullying for less than 6 months) or Maintenance stage (meaning they had not participated in any of the three roles related to bullying for 6 months or longer) for that role. Students who responded affirmatively to either of the behavior items related to a role were asked their intentions to make changes so they would no longer be participating in that role. Students completed all assessments and measures of self-reports on computers.
At baseline, there were 48.9 percent of students involved in roles as bullies, 77.2 percent as victims, and 36.5 percent as bystanders. Many students participated in multiple roles. Further, 12 percent of students were in the Action or Maintenance stages at baseline. The rest of the students were in the Precontemplation, Contemplation, or Preparation stages of change. There was a statistically significant difference between the control and treatment groups at baseline for the bullying role; specifically, there were fewer bullies in the control group. Statistical procedures were employed to control for these baseline differences. There were no other statistically significant baseline differences regarding bully-related role or stage of change between the treatment and control groups.
Data were measured using two sets of analyses: 1) posttest-only comparisons of the number of participants who no longer reported involvement in any or all roles, and 2) a repeated-measures comparison of the number of participants who reported they no longer participated in any or all roles. Posttest-only comparisons used an arcsine transformation difference in proportion of movement into Action or Maintenance. An intent-to-treat analysis was used to examine all records, rather than only those that could be matched. The last-observation-carried-forward method of replacement was used when a posttest value was missing. This method replaced the missing value with the last known observation for that record. Repeated-measures analyses for all students across all time points and random effects logistic models were used to examine any changes across the course of the study and to control for baseline covariates. No subgroup analysis was conducted.
Study 2
Evers and colleagues (2007a) used a 3x2 factorial experimental design that had three experimental groups crossed with two assessments to evaluate the effectiveness of Success in Stages: Build Respect, Stop Bullying® for a middle school population.
Of 2,839 middle school students who began the program, 1,230 student participants could be matched from baseline to posttest. These students were from 12 middle schools (sixth to eighth grades) from a variety of communities (rural, urban, suburban) that were recruited for participation. Similar to Study 1 (Johnson et al. 2005), preference was given to schools with higher percentages of students eligible for free lunches. Schools were then matched based on region of country, type of community, and number of students eligible for free lunches. Just over 50 percent of the student sample were female, and 48 percent were eligible for free lunches. Students who could be matched at baseline were white (40.5 percent), Hispanic (27.4 percent), Black (12.0 percent) and Asian-American (4.4 percent). The largest percentage of students (45.1 percent) were in the seventh grade, and the rest were spread throughout the sixth through eighth grades.
The control group completed only pretest and posttest measures. Students in treatment group 1 received up to three 30-minute, Internet-based, individualized and interactive sessions during the school year, a Staff Guide and a Family Guide, and a posttest. Students in treatment group 2 received the same protocol as treatment group 1 and, in addition, received a pretest before their first intervention. This group allowed for a replication of the interventions and a control for any effects owing to a pretest. The treatment groups completed all sessions within a typical 2-month window ranging from 1 to 3 months. The CrimeSolutions review of this study focused on the outcomes for treatment group 2 (n = 266 students) compared with the control group (n = 483 students).
Using self-report measures, students indicated participating in any of the three roles related to bullying (bully, victim, or passive bystander, which were assessed the same way as described in Study 1) and their intention to stop. The middle school sample had 80.6 percent of students involved in roles as victims, 75.6 percent as bullies, and 64.4 percent as bystanders at baseline (students could be involved in multiple roles). Stage of change was then determined by a single item matched to the roles students engaged in. Students were asked their intentions to make changes such that they would no longer be participating in specific roles. At baseline, only 4 percent of the middle school students were in the Maintenance stage and 1 percent were in the Action stage. The rest of the students were in the Precontemplation, Contemplation, or Preparation stages of change. There were statistically significant differences between the treatment and control groups on grade level and ethnicity at baseline. The statistical analyses controlled for effects of all baseline measures.
Outcome measures included the percentage of students who were no longer participating in each of the three roles related to bullying (bully, victim, or passive bystander) and the percentage who were no longer participants when all three roles were combined, at approximately 1 month posttest. This study used the same statistical procedures from Study 1 (Johnson et al. 2005) when analyzing the data. No subgroup analysis was conducted.
Study 3
Evers and colleagues (2007b) also evaluated the effectiveness of Success in Stages: Build Respect, Stop Bullying® for a high school student population. The methodology was the same as described in Study 2 (Evers et al. 2007a).
Of 2,713 high school students who began the program, 1,203 participants could be matched at posttest. These students were from 13 high schools (grades 9–12) from a variety of communities (rural, urban, suburban) that were recruited for participation. Similar to Studies 1 and 2, preference was given to schools with higher percentages of students eligible for free lunches. Fifty-five percent of the sample was female, with 42 percent of the total high school sample eligible for free lunches. Ninth graders made up the largest percentage of the sample, at 41.6 percent. No other characteristics of the sample were provided.
The control group and two treatment groups completed the same assessments and received the same intervention as described in Study 2. The CrimeSolutions review of this study focused on the outcomes for treatment group 2 (n = 531 students) compared with the control group (n = 309 students). Of the high school students, at baseline, 69.8 percent reported being bullies, 71.2 percent as victims, and 74.7 percent as bystanders (students could be involved in multiple roles). Further, 6.5 percent were in the Maintenance stage and 6.0 percent were in the Action stage. In addition to grade level and ethnicity, there were statistically significant gender and baseline stage-of-change differences between the treatment and control groups. Statistical analysis procedures were used to control for effects of all baseline differences.
This study followed the same methodology as Study 2 (Evers et al. 2007a) with regard to timeline and implementation of the program. The same statistical procedures from Study 1 (Johnson et al. 2005) were also used when analyzing the data. No subgroup analysis was conducted.
