NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: ED663543
Record Type: Non-Journal
Publication Date: 2024-Sep-19
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Mapping the Research Base for Universal Behavior Screeners
Katie Pelton; Kathleen Lane
Society for Research on Educational Effectiveness
Universal behavior screening is used in schools worldwide to detect students with and at risk for behavioral challenges. As schools prepare to implement screening, they face a major decision: which screening tool to use. Several scholars have published recommendations to support education leaders in selecting a screening assessment (Glover & Albers, 2007; Oakes et al., 2017). These recommendations offer a roadmap, though it may be challenging for schools and districts to implement as information on systematic screening tools has developed substantially in the last decade without a comprehensive listing of these instruments nor a consolidated source of information across instruments. We conducted this scoping review to provide a rigorous -- and accessible -- overview of the research base for universal behavior screening instruments to facilitate educators' decision-making process when selecting a systematic screening tool for the students they serve and identify areas of further refinement for the research community. Our search included three phases with reliability assessed in each phase: (1) an electronic database search using Boolean operators, (2) hand searches through identified journals, and (3) a second electronic database search including additional instruments identified in the first two phases (Figure 1). For articles to be included in this review, the article had to be published in English within a peer-reviewed journal, the sample had to include all students in a given grade or age range in a PK-12 school setting, and include some psychometric analysis of a behavior screening tool. We located 180 articles meeting these criteria. From this search, we identified 56 behavior screening instruments. Wording for instrument and subscale names varies greatly across instruments due to different foci (e.g., internalizing, externalizing) and approaches (i.e., deficit skills, strength-based). We suspect many tools measure slightly different constructs. We discuss the importance of clearly defining intended constructs and encourage further discussion amongst researchers on how to clarify these constructs to support educators in identifying an instrument aligned with their intended use. The most common psychometric analyses included coefficient alpha for internal consistency, correlations between theoretically related variables, and confirmatory factor analysis. Coefficient alpha was by far the most common reliability estimate reported (n = 110; 61.11%) with authors including omega more often in recent years (n = 21; 11.67%), suggesting more researchers in this area are researchers moving away from relying solely on coefficient alpha as the underlying assumptions are rarely met fully in practice (Cronbach & Shavelson, 2004; Sijtsma & Pfadt, 2021). Researchers established validity evidence by comparing screening scores to other theoretically related variables such as other screeners, behavior rating scales, or educational outcomes (e.g., attendance, office discipline referrals [ODRs], academic outcomes). Many studies reported correlation coefficients (n = 84; 46.67%), though correlations are often a prerequisite for more advanced analyses. Other analyses included linear regression, logistic regression with extensions into conditional and receiver-operator characteristic curves, kappa coefficients, t-tests, and structural equation modeling. We hope more researchers will use analysis methods which can appropriately model the known nested structure of the data with students nested within their classrooms (Huang, 2018) or employ methods to correct standard errors (McNeish et al., 2017). Although these analyses are more complex to conduct and interpret, we believe collaboration with methodological experts may result in more rigorous and nuanced understanding of how behavior screening instruments function in schools. Many studies evaluated instruments' internal structure with confirmatory factor analysis (n = 77; 42.78%) and fewer studies reported exploratory factor analysis (EFA; n = 45; 25.00%). Multiple group confirmatory factor analysis (MGCFA) or measurement invariance was even less common (n = 34; 18.89%), though MGCFA has become more common with 31 of the 34 studies published in the last decade. We anticipate these analyses will continue to become more commonplace as researchers work to support equitable educational practices and new criteria are established for screening instruments (e.g., NCII, 2022). In sum, we conducted this review to create a rigorous and comprehensive overview of the field including an extensive list of behavior screening instruments for implementers and map how instruments have been studied to date. We located 56 instruments available for screening and 180 articles examining the psychometric properties of these tools meeting our criteria. The most common analyses examined internal consistency, correlations with related variables, and factor analysis. We encourage researchers investigating systematic screening to continue to evolve with research methodology to utilize the most appropriate and robust analytic techniques for their research questions. We expect in another five to 10 years the field will have continued the ongoing validation process for these instruments to ensure we are accurately and equitably screening for emotional and behavioral disorders (EBD) in our school age population. In this meantime, we are hopeful this systematic review will be useful to researchers as they conduct additional psychometric inquiry and -- perhaps most importantly -- educational leaders to inform their decision as to which screening tools to adopt to inform instruction to support educators in providing positive, productive, and safe learning environments for preK-12 students.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Information Analyses; Reports - Research
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A
Author Affiliations: N/A