Practice Profile: Bullying Prevention Programs

Gaffney, Ttofi, and Farrington (2019) conducted a new review of the literature to update the meta-analysis by Farrington and Ttofi (2009) that looks at school-based programs designed to reduce bullying and victimization. The new search covered literature available from 2009 through December 2016 and was added to literature captured in the Farrington and Ttofi (2009) search from 1983 through May 2009. Using the same systematic search process as the original meta-analysis, the 2019 meta-analysis included primary studies that 1) described an evaluation of a school-based antibullying program implemented with school-age participants (between 4 and 18 years of age); 2) used an operational definition of school bullying that coincides with existing definitions (that is, Gladden et al. 2014; Farrington 1983); 3) measured school-bullying perpetration and/or victimization using quantitative measures such as self-, peer-, or teacher-report questionnaires; and 4) used an experimental or quasi-experimental design, with one group receiving the intervention and another (control group) not receiving the intervention. To facilitate a search for such programs, Boolean searches were conducted on several online databases. Databases of unpublished reports (e.g., ProQuest) also were searched to include gray literature in the review. Further, primary studies (both included in and excluded by previous systematic reviews) were scanned for any additional evaluations that we may have missed in the systematic searches. There was no sample-size limitation. The search included English and non-English studies and published articles and unpublished manuscripts.

Ultimately, the researchers identified 65 studies published since 2009 and combined them with the 35 studies from the previous meta-analysis, resulting in 100 studies with a total of 103 independent effect sizes. The types of study designs were randomized control (n = 45), quasi-experimental (n = 44), and age cohort (n = 14). There were 101,615 participants in the randomized controlled and quasi-experimental designed studies. The authors provide information on the racial/ethnic and gender breakdown of individual studies when that information is available. The studies involved both male and female students and included students from prekindergarten through high school. Race/ethnicity was not provided for most studies. The included studies looked at programs implemented in the United States and internationally throughout North America, South America, Europe, Asia, Africa, Australia, and New Zealand.

Cohen's d was estimated for primary studies when results were reported in the form of continuous data. For primary studies that presented results as percentages or frequencies of participants identifying as either perpetrating bullying or being victimized, the odds ratio (OR) effect size was estimated. All effect sizes were converted to odds ratios. Further, because clustering is a common phenomenon in educational evaluations, effect sizes were corrected for the inclusion of clusters in primary studies. The results of the meta-analysis were presented using three different models: 1) a random effects model that weights studies in proportion to the between-study variance, 2) a fixed effects model, which assigns greater weight to larger evaluations; and 3) a multiplicative variance adjustment to account for the heterogeneity of effect sizes.

Polanin, Espelage, and Pigott (2012) focused their meta-analysis on the effect of school-based bullying prevention programs on increasing bystander intervention in bullying situations. They included studies of programs implemented in K–12 school settings that explicitly measured the behavior of bystanders (also known as passersby, observers, or witnesses). The measure of bystander intervention included intention to intervene, intention to stop bullying, direct intervention, or difficulty in responding assertively to a bullying situation. The researchers also included a measure of empathy for the victim, such as “feeling sad about students who are bullied” and “unpleasantness when another student is being bullied.”

To be included in the analysis, studies needed to be in English and be peer-reviewed and published or conducted between 1980 and 2010. Additionally, the studies needed to be based solely within a school system and intended purposefully to modify bystander behavior. The authors searched five electronic databases and bibliographies of retrieved documents for English-language studies published or conducted between 1980 and 2010. They identified 12 studies that included a total of 12,874 participants. No information is provided on the racial/ethnic or gender composition of the study participants. Programs were located in Western Europe and the United States.

The authors used the standardized mean difference to determine effect sizes for each study using a continuous scale for outcome measures; all effect size metrics were bias corrected, using Hedge’s small sample correction (g). The authors calculated logged odds ratio effect sizes for studies that used a categorical outcome measure (which were converted into a standardized mean difference). They used a random effects model for their analysis. The weighted mean effect size for bystander intervention is based on 12 studies; the weighted mean effect size for empathy (toward the victim) is based on 8 studies.

Wong (2009) conducted a meta-analysis of school-based bullying prevention programs for students K–12. To be included, studies had to use an experimental design or a pretest/posttest independent-groups design; the analysis excluded studies that used age cohort designs and quasi-experimental designs with no pretest measure. Additionally, studies had to evaluate a program, intervention, or policy in a primary, secondary, or K–12 school, had to assess at least one outcome variable that represented bullying behavior or victimization, and each experimental group had to be at least 20 students. The author searched 14 electronic databases from the inception of the database through April 2008, and the bibliographies of retrieved documents, including English-language published (journal articles and book chapters) and unpublished (dissertations and reports) studies. The studies needed to include a measure of bullying or victimization. Samples could be as small as 20 participants per group. Although there was no date limit for publication, all eligible studies were published between 1996 and 2008. Studies covered programs in North America (11 studies) and Europe or Australia (11 studies). No information is given on the racial/ethnic or gender composition of the samples included.

The standardized mean difference effect size was calculated for studies using continuous outcome measures. For one study, Wong converted the product moment correlation and used the Cox-transformed log odds ratio to calculate effect sizes for studies using dichotomous outcome measures. A cluster adjustment of the effect sizes and standard errors was made to take the nesting of students in classrooms and schools into account. The author used a fixed-effects model to calculate the standardized mean effect size.

The weight mean effect size for bullying is based on 22 effect sizes for bullying outcomes from 19 studies involving 18,903 students. The effect size for reducing victimization is based on 25 effect sizes from 22 studies involving 25,361 students.

Kennedy (2020) conducted a meta-analysis to examine the effectiveness of bullying prevention programs on subtypes of traditional bullying victimization (relational, physical, and verbal victimization). A systemic search of literature published from 1990 to 2018 was conducted using three academic databases: PsycINFO, Academic Search Ultimate, and ERIC. Additionally, the reference lists of other meta-analyses on bullying prevention programs were reviewed. Studies were included if they conducted at least one evaluation of a bullying prevention program implemented on youths under age 18 and were published from 1990 to 2018. Additionally, the studies had to be published in English. Research designs included experimental designs, quasi-experimental designs, and nonexperimental designs that used pretests and posttests only. These nonexperimental designs were included to increase sample size. (Note: This CrimeSolutions review includes those analyses that examined only experimental and quasi-experimental designs.) Finally, studies were required to have reported outcomes that related to at least one bullying subtype (relational, physical, or verbal).

Thirty-three individual studies with 87 effect sizes were identified in the search. Thirty-three of the effect sizes were for relational bullying victimization (38 percent), 31 effect sizes were for physical bullying victimization (36 percent), and 23 effect sizes were for verbal bullying victimization (26 percent). The majority of studies (56 percent) were experimental or quasi-experimental, and the remainder (44 percent) were pretest/posttest design. Most studies (76 percent) examined bullying prevention programs for eighth grade and below, 18 percent were for ninth grade and above, and 6 percent had an overlap in age groups. Additionally, 59 percent of the studies took place in the United States, and 68 percent were published in 2010 or later.

All analyses were conducted using comprehensive meta-analysis software Version 3 (Borenstein et al. 2013). The primary method of analysis was a meta-analysis, using an approach that takes into account multiple outcomes from each study. The effect size statistic is an odds ratio. To minimize variance, the mean effect size was computed by weighting the individual effect sizes, assigning the weights by using the sum of the variance within and between studies. A random-effects model was used. A moderator analysis was conducted to analyze the impact of different study-level moderators.