NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: ED658490
Record Type: Non-Journal
Publication Date: 2022-Sep-23
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
A Data-Centric Approach to Examining Associations between Quality and Outcomes in Early Childhood Education
Jonathan Seiden; Emily Hanno; Luke Miratrix; Thu Pham; Stephanie Jones; Nonie Lesaux
Society for Research on Educational Effectiveness
Background/Context: Quality in early education and care (ECE) programs is often conceived as a combination of structural features such as class size and teacher qualification and process features related to the interactions between and within children and adults in the care setting (Hanno et al., 2021; Howes et al., 2008). From a theoretical perspective, process quality is thought to be particularly important with the quality of child-adult interactions understood to be the basis of lifelong learning (Bronfenbrenner & Morris, 2007). Poor quality has become a common explanation for the mixed and even negative results found in some methodologically rigorous evaluations of state-funded pre-Kindergarten and ECE programs (Mader, 2022; Meloy et al., 2019). This argument suggests that without sufficient funding to improve quality, programs designed to improve child outcomes cannot achieve sufficient quality to succeed. Despite near universal acknowledgement about the importance of quality, concretely defining and measuring quality and building empirical evidence about its importance to child outcomes remains an elusive goal. A Consensus Statement from the PreKindergarten Task Force noted that "children's early learning trajectories depend on the quality of their learning experiences" (Phillips et al., 2017, p. ii). In the same report, Farran (2017, p. 45) acknowledges that "questions about how to define quality in early childhood programs and the impact of different approaches on children's outcomes have engaged the research and policy communities since the 1970s and 1980s, and, in many cases, persist to this day." If quality is a linchpin to the success or failure of ECE programs to improve children's outcomes, we would expect providers exhibiting higher levels of quality to have children who exhibit larger gains in their developmental and pre-academic outcomes than providers with lower quality. Unfortunately, the many studies that have examined this question (typically in a non-causal manner) have often found small, and in some cases, no relationships between current measures of ECE quality and child outcomes both domestically and abroad (Brunsek et al., 2017; Hanno & Gonzalez, 2020; Keys et al., 2013; Raikes et al., 2019; Weiland et al., 2013). Purpose/Objective/Research Question: Most studies of the relationships between quality and outcomes have taken a theory-centric approach to defining and measuring quality (Keys et al., 2013). For example, in their meta-analysis of the associations between the Early Childhood Rating Scale and child outcomes, Brunsek et al. (2017) focus on associations between pre-defined scales of measurement on the ECRS and outcomes. These scales are a valuable way to consolidate data on quality into cohesive domains. However, it possible that some aspects of quality or unmodeled sub-constructs of quality are the most important drivers of child outcomes. This paper seeks to deepen our exploration of this question by instead taking a data-centric approach examining the relationship between a large set of unaggregated facets of quality in pre-kindergarten classrooms and child-level outcomes. Population/Program/Practice: We build on explorations of quality-outcome relationships using data from the Early Learning Study at Harvard (ELS@H), a large longitudinal study of ECE that is representative of the settings experienced by children in the commonwealth of Massachusetts. We combine data on quality and outcomes from three sources: (1) Child-level outcomes representing several domains of early childhood development, for example, the Minnesota Executive Function Scale (Reflection Sciences, Inc., 2021) and the Leiter-3 Cognitive/Social and Emotion/Regulation (Roid & Koch, 2017); (2) Observations of child behavior in each class using the Child Observation in Preschool (COP) (Farran & Anthony, 2014); and (3) Observations of teacher behavior in each class using the Teacher Observation in Preschool (TOP) (Bilbrey et al., 2007). Previous analyses of COP and TOP observation data by Farran et al. (2017) identified a set of "Magic 8" classroom practices associated with gains in child-level outcomes by analyzing the magnitude of associations between practices and gains in 26 pre-K classrooms in an urban school district. These eight practices are now promoted as part of the professional development curriculum of the Partnership for Developing Model Early Learning Centers between Peabody Research Institute and Metro Nashville Public Schools (Partnership for Developing Model Early Learning Centers, 2016). This research seeks to investigate whether these same relationships are found when taking a data-centric approach to modelling. Data Collection and Analysis: Because children do not learn in a vacuum, the first step in our analysis is to generate important contextual variables. We calculate individual level means of behaviors on the COP and include information about the mean and variance of COP variables of peers in each class. We then add TOP variables about mean teacher practices in each class which are shared by all children within a class. For each of our child level outcomes, we calculate a gain score based on differences between the beginning and end of a school year. To analyze our combined dataset, we use a variety of machine-learning techniques focused on variable selection to understand the class- and teacher-level aspects of quality most predictive of gains in outcomes. Our primary method is a cross-validated LASSO approach, which finds optimally predictive models by penalizing coefficients on a linear regression model. We fit separate LASSO models for each of the outcome measures to identify whether we find consistent relationships across outcome measures or whether different aspects of ECE quality might be related to different developmental outcomes. Because we do not wish our results to be reliant on the ML method chosen, we also fit a random forest as a robustness check. Doing so allows us to examine whether the variables selected in LASSO models concord with the variables deemed most important for predictive ability in a random forest. After beginning with a data-centric approach, we end our paper by examining the degree to which the associations our analyses surface fit existing conceptualizations of process and structural quality. We conclude by suggesting future avenues for better measuring ECE quality data and exploring outcome-quality relationships. Because our analysis is explicitly non-causal, future research designs would be required to build on our findings to understand whether the aspects of quality we identify are causally related to improvements in child outcomes.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Early Childhood Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A
Author Affiliations: N/A