ERIC Number: ED663544
Record Type: Non-Journal
Publication Date: 2024-Sep-20
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Measurement Invariance and Predictive Validity of a Free-Access Universal Screening Tool: The SRSS-IE
Kathleen Lane; Katie Pelton
Society for Research on Educational Effectiveness
Educational leaders through the United States have prioritized developing integrated tiered systems to assist educators in meeting students' academic, behavioral, and social and emotional well-being learning needs in an integrated fashion (McIntosh & Goodman, 2016). This is an important shift away from traditional reactive models towards proactive, systemic structures for prevention and search-and-serve mandates (IDEA, 2004). A key feature of these integrated tiered systems of support are the use of practical, reliable systematic screening tools to inform valid inference making with respect to meeting student's multiple needs: academic, behavioral, and social and emotional well-being learning needs. In most tiered systems, educators completed academic and behaviors screening tools at three times per year (fall, winter, and spring) to inform instructional practices and school policies. These data are along with other data collected as part of regular school practices to shape instruction at Tier 1, empower teachers with strategies to maximize engagement, and connect students to validated Tier 2 and 3 interventions when Tier 1 practices are insufficient. In addition to prioritizing the implementation of academic screening tools (Briesch et al., 2022), educational leaders also prioritize the need for universal behavior screening to detect students with and at risk for emotional and behavioral disorders (EBD) at the earliest signs of concern. This is a diverse group of students, including students with both major challenges of childhood and youth: externalizing (e.g., aggressive, noncompliant) and internalizing (e.g., anxious, withdrawn; Walker et al., 2005). The Student Risk Screening Scale -- Internalizing and Externalizing (SRSS-IE; Drummond, 1994; Lane & Menzies, 2009) is one of a few free-access screening tools with substantial research on a range of psychometric properties (Pelton et al., 2024). Initial inquiry indicated scores from both externalizing and internalizing scales have been predictive of critical school outcomes in both academic (e.g., oral reading fluency, course failures) and behavioral (e.g., office discipline referrals, in-school suspensions) domains (e.g., Lane, Oakes et al., 2019; Menzies et al., 2012). Yet, many studies conducted have represented samples from a single district or state. In addition, only recently have studies examined (a) measurement invariance as well as (b) more comprehensive studies examining predictivity validity. In this presentation, we report findings of three recent studies of newly adapted SRSS-IE, for use in K-12 settings. In the first study (Lane, Oakes, Buckman et al., 2023), we present new findings of the Student Risk Screening Scale -- Internalizing and Externalizing (SRSS-IE). We examined (a) SRSS-IE factor structure and (b) the degree to which measurement invariance held across sex, special education status, race, and ethnicity, as well as over time (fall, winter, spring) within each school level (elementary, middle, high). This large-scale study funded by the Institute of Education Sciences (IES) included 124 schools from four U.S. geographic regions in their first year SRSS-IE implementation collected over a ten-year period. Using confirmatory factor analysis procedures accounting for the nesting of students within teachers' classrooms, we confirmed a two-factor structure (internalizing and externalizing) and determined three items may be removed from the SRSS-IE and still result in adequate model fit. In brief, all model comparisons between configural, metric, scalar, and strict models met invariance criteria in fall, winter, and spring administrations. Furthermore, longitudinal models also met these invariance criteria. However, replication with schools in later stages of systematic screening is essential before shifting screening practices. In the second study we examined predictive validity of the newly defined Student Risk Screening Scale -- Internalizing and Externalizing (SRSS-IE 9, with 9 items) when used by teachers in school conducting SRSS-IE screening for the first time. This sample included 24,535 elementary students from 74 schools representing four geographic regions. Results revealed fall externalizing and internalizing latent factors as well as subscale scores (SRSS-E5, SRSS-I4, respectively) from the SRSS-IE 9 predicted important year-end outcomes. For example, scores predicted year-end behavioral outcomes (office discipline referrals, suspensions, and nurse visits), academic outcomes (reading and math scores and course failures), as well as referrals to special education for elementary-age students. Given many school systems currently use the SRSS-IE 12 as regular school practice, we examined these original scores as well, with result indicating externalizing (SRSS-E7) and internalizing (SRSS-I5) subscales were comparable in predictive of these outcomes. In the third study, we examined predictive validity the SRSS-IE 9 when used for the first time by middle (four geographic regions) and high (three geographic regions) school teachers from 43 schools. The sample included 11,773 middle school-aged students and 7,244 high school-aged students. Findings revealed indicated fall SRSS-IE externalizing and internalizing latent factors as well as subscale scores predicted year-end behavioral (office discipline referrals and in school suspensions) and academic (course failures) outcomes for middle and high school students as well as referrals to special education for middle school students. Internalizing scores also predicted referrals to special education for high school students. Results also indicated, externalizing and internalizing scores predicted nurse visits at the middle and high school levels with all models save for the subscale models of internalizing in middle school. Analysis of SRSS-IE 12 subscale scores for externalizing (SRSS-E7) and internalizing (SRSS-I5) also yielded comparable outcomes, with few variations. In our presentation we will discuss educational implications, illustrating how these data are used to inform instruction to meet student's multiple needs in an integrated fashion (Buckman et al., in press). Further, we discuss limitations and directions for future inquiry.
Descriptors: Multi Tiered Systems of Support, Screening Tests, Measures (Individuals), Student Needs, Student Behavior, Behavior Problems, At Risk Students, Gender Differences, Special Education, Racial Differences, Time, Instructional Program Divisions, Individual Differences, Elementary School Students, Academic Achievement, Discipline, Predictive Validity, Middle School Students, High School Students
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Elementary Education; Junior High Schools; Middle Schools; Secondary Education; High Schools
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
IES Funded: Yes
Grant or Contract Numbers: N/A
Author Affiliations: N/A