NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: ED659538
Record Type: Non-Journal
Publication Date: 2023-Sep-29
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Leveraging Integrative Data Analysis to Account for Measurement Non-Invariance in Teacher- and Student-Reported School-Related Positive Mental Health
Alexa Budavari; Heather McDaniel; Antonio Morgan-Lopez; Lissette Saavedra; Anna Yaros; John Lochman; Catherine Bradshaw
Society for Research on Educational Effectiveness
Background: It has become increasingly important to shift education research and programming from a deficit focused perspective to a positive youth development perspective in order to identify and promote factors that support students' positive functioning and outcomes. One outgrowth of positive youth development is Positive Mental Health (PMH), which describes students' positive social, emotional, and cognitive well-being and is associated with numerous short- (e.g., academic achievement, academic engagement) and long-term (e.g., improved mental and physical health) outcomes. The study of school-related PMH is particularly important for student success, as numerous facets of PMH (e.g., thriving, flourishing) have been shown to improve students' trajectories of positive functioning. Recent research has begun to identify factors that promote school-related PMH for students, such as social support and self-esteem, yet further research is needed to identify factors that promote PMH among a diverse group of students. Previous studies have been limited by the sociodemographic make-up of their sample, which has restricted researchers' ability to compare and contrast factors that contribute to changes in PMH in students over time. Furthermore, previous approaches to measuring PMH have often failed to account for potential measurement non-invariance across sociodemographic factors. Failure to account for measurement non-invariance when exploring students' PMH can lead to biased conclusions, as measurement non-invariance and true differences in mean levels of the latent construct may be conflated. Objectives: This study focuses on measures of school-related PMH. Specifically, we leverage data from students randomized to the control condition across 8, tier-2 RCTs to explore measurement non-invariance and latent mean differences in reports of students' school-related PMH. We hypothesize that students' gender identity, race/ethnicity, geographic region, and socioeconomic status will be associated with the way in which teachers and students report on students' PMH. After accounting for partial invariance, we hypothesize that these predictors will also be associated with differences in school-related PMH latent means at baseline. Setting: The 8 preventive intervention RCTs (i.e., Coping Power; Lochman & Wells, 2004) took place across 3 states over the last 20+ years, and the current analyses were conducted solely among students randomized to the control (i.e., school-as-usual [SAU]) condition. The interventions were predominantly delivered in elementary schools, whereas two of the interventions were delivered in secondary schools. Participants: Students in the RCTs were eligible if they demonstrated elevated levels of externalizing behaviors, as rated by teachers or parents. For the current analyses, only students in the control condition (SAU) were included in order to avoid influences of any potential intervention effects, given that evidence suggests the potential for an interaction between school-related PMH and the active components of the preventive intervention. Research Design: The current study uses an Integrative Data Analysis (IDA) approach to harmonize data across several studies. IDA provides a framework to help account for semantic differences among similar measures collected across studies (e.g., differing versions/language of the same measures), provides an approach to harmonize questions and responses across different measures (e.g., combining two conceptually similar subscales from two distinct measures), and provides a methodological framework to account for other cross-study variations in measurement (e.g., distinct populations of students). A benefit of the IDA approach is it leverages a larger sample size of underserved and marginalized populations, thus creating the opportunity to explore more equity-specific questions among students that are generally underrepresented in individual studies. Data Collection and Analysis: We will semantically harmonize questions across multiple measures in which students and/or teachers reported on the students' school-related PMH. From the overall set of measures, a series of steps were taken to identify a subset of subscales or specific items relevant for the current IDA. First, we identified all measures with school-related PMH items that were collected in at least 1 of the 8 studies. Second, we identified a set of constructs that previous literature has identified that comprise different aspects of school-related PMH (e.g., student connectedness, school connectedness, teacher connectedness, school-related self-concept). Third, multiple rater semantic harmonization techniques were employed to map similar items across constructs from these measures. Student-reported PMH items were semantically harmonized across multiple measures, including the BASC, School Attitudes and Bonding, and Perceived Self Competence Scale. Additional multiple rater logical harmonization of response options was conducted due to the varied response options across measures. Teacher-reported PMH was solely collected on the BASC, and thus only required analytic harmonization across versions. Once the item-set is finalized and response options and questions across measures are harmonized, we will employ Moderated Non-Linear Factor Analysis (MNLFA) to explore measurement non-invariance and estimate factor scores that take into account differential item functioning (DIF). We will test for DIF based on the students' gender identity, race/ethnicity, geographic region, and SES for school-related PMH. Results: We will discuss the numerous decision points required when harmonizing student- and teacher-reported PMH across multiple measures. We will report on the significant intercept and loading DIF from our MNLFA focused specifically on teacher-reported PMH, as cross-measure harmonization was not required. Conclusions: Given the association between student PMH and both short- and long-term academic outcomes, it is necessary to explore how the measurement of this construct may differ across a diverse range of students, especially when considering multiple reporters. By accounting for DIF when estimating factor scores and exploring predictors of DIF, we will be able to ascertain which predictors are associated with differences in measurement versus differences in levels of the latent construct. Subsequent research can then be conducted to explore trajectories of students' PMH over time that more adequately reflect the students' underlying PMH by accounting for potential biases in measurement. These trajectories will then enable us to identify factors that promote PMH across all students, as well as factors that are unique to students from underserved and/or marginalized backgrounds. In addition to highlighting methodological innovations of IDA, implications for intervention will be discussed.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A
Author Affiliations: N/A