ERIC Number: ED663263
Record Type: Non-Journal
Publication Date: 2024-Sep-19
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Method-of-Moment Corrected Maximum Likelihood (Ml) Structural-after-Measurement (SAM) Estimator for n-Level Structural Equation Models
Fangxing Bai; Ben Kelcey
Society for Research on Educational Effectiveness
Purpose and Background: Despite the flexibility of multilevel structural equation modeling (MLSEM), a practical limitation many researchers encounter is how to effectively estimate model parameters with typical sample sizes when there are many levels of (potentially disparate) nesting. We develop a method-of-moment corrected maximum likelihood (ML) structural-after-measurement (SAM) estimator for n-level structures with latent variables that is well-suited to the types of small-to-moderate sample sizes typically seen in education research. We probe the consistency, variability and convergence of the estimators with small-to-moderate n-level samples. The SAM estimator emerges as a practical alternative or complement to conventional ML because it often outperforms ML in small-to-moderate n-level samples in terms of convergence, bias, and variance. The proposed estimators are illustrated through an n-level teacher development example. Consider a prototypical example of a professional development study that probes the theory of action for teacher development. Teachers are assigned to a control or professional development condition (treatment; T) designed to improve teaching (latent mediator; [eta][subscript M]) in ways that advance student achievement (latent outcome; [eta][subscript Y]). Estimation of the indirect effect requires the connection of two disparate nesting structures--a teacher side that uses multiple lessons-nested-within-each-teacher and a student side that uses multiple students-nested-within-each-teacher. Conventional MLSEM applications cannot appropriately capture such disparate structures because their purview is restricted to models that contain the same hierarchical or cross-classified (sub-)structure across levels. Recent advances in n-level SEM provide a theoretical bridge for designs with multiple disparate structures. However, stable estimation of such models requires 100 clusters/teachers with 10-20 individuals each as a minimum. In this study, we build on recent developments of limited information estimators and introduce an alternative SAM estimator for estimating small-to-moderate scale MLSEMs with multiple disparate nesting structures. Methods: To outline our developments, we draw on a n-level SEM formulation of mediation that uses common factor models for covariates, mediator, and outcome and a structural model that connects them. The conceptual flow of our SAM estimator follows this conceptual sequence: (a) commence by estimating the random effects of each level using empirical Bayes estimation for each observed variable based on their individual structures; (b) Next, derive the level specific factor score for each latent variable using regression method; (c) Forecast the covariance matrix among the latent variables at the target level using the factor scores from (b); (d) Refine these covariances utilizing the Croon-based method of moments correction; (e) Finally, utilize the corrected covariance matrix to estimate the structural parameters. Consider teacher quality (mediator) for example. For the teacher quality (mediator) model, following the process, we first obtain the empirical Bayes estimates of the random effect of each level. That is, we decompose teacher quality into three level-specific observed teacher qualities [equation omitted]. The level specific indicators can be further decomposed as follows [equations omitted]. We use m[subscript ljk] as the observed indicators of lesson l in teacher j rated by rater k for the latent mediator, [eta][superscript L2a][subscript M[subscript j]], [eta][superscript L2b][subscript M[subscript k]], and [eta][superscript L1b][subscript M[subscript ljk]] as the teacher-, rater-, and lesson-level components of the latent outcome, [delta][superscript L2a][subscript M], [delta][superscript L2a][subscript M], and [delta][superscript L2B][subscript M] as the teacher-, rater-, and lesson-level factor loadings, [mu][subscript M[subscript j]] and [mu][subscript M[subscript k]] as intercepts that vary across teachers and raters, and [epsilon][superscript M[superscript L2a]][subscript j], [epsilon][superscript M[superscript L2b]][subscript k], and [epsilon][superscript M[superscript L1b]][subscript jk] as the teacher, rater-, and lesson-level error terms. We set the scale by assigning unit variances to the teacher-, rater-, and lesson-level factors. M[superscript L2a]][subscript j] is the teacher level specific teacher quality (mediator) of teacher j, M[superscript L2b]][subscript k] is the rater level specific teacher quality of rater k, and M[superscript L1b]][subscript ijk] is the lesson level specific teacher quality of lesson l within teacher j rated by rater k. We set the scale by assigning unit variances to the teacher- and lesson-level factors. [equation omitted]. Similarly, the outcome model nests students in teachers such that [equations omitted] where y[superscript L2a][subscript j] is the teacher level specific student outcome of teacher j, y[superscript L1a][subscript ij] is the student level specific student outcome of student i within teacher j. The teacher-level structural models are then [equations omitted] with a capturing the treatment-mediator impact and b capturing the mediator-outcome association. Estimation. In our omitted derivations, we derive the SAM estimator for covariances among the latent variables in step one and then adjusts those covariances for bias stemming from the measurement uncertainty (Devlieger & Rosseel, 2017). As a simple example, consider the teacher-level component of the latent variable of teaching quality (mediator; [eta][superscript L2a][subscript M]) and student outcome (outcome; [eta][superscript L2a][subscript Y]). In the context of our multilevel mediation application, the resulting correction for the covariance is [equation omitted] where cov([eta][superscript L2a][subscript Y], [eta][superscript L2a][subscript M]) is the true covariance on level 2a (teacher-level), cov(M~[superscript L2a], ?[superscript L2a]) is the predicted covariance using factor scores, A are the factor score matrices, [delta]'s are the factor loading matrices. More conceptually, the result suggests that (cov[eta][superscript L2a][subscript Y], [eta][superscript L2a][subscript M])) can be obtained by dividing (cov(M~[superscript L2a], ?[superscript L2a])) by (A[superscript L2a][subscript M]R[superscript L2a][subscript m][delta][superscript L2a][subscript M]A[superscript L2a][subscript Y]R[superscript L2a][subscript y][delta][superscript L2a][subscript Y]). Our bias-corrected estimator operates by generating corrected estimates of the covariances among the latent variables from which we can obtain unbiased estimates of the parameters via the typical SEM covariance analysis. The average indicator reliability vector (or matrix) for mediator at the teacher level (R[superscript L2a][subscript m] or [omega][superscript L2a][subscript m]) is determined through a univariate approach. For example, for the first indicator of the mediator, we have [equation omitted]. Here n([superscript L2a])[subscript 1b] indicates the number of lessons rated for teacher j. This comprehensive approach at the teacher-level integrates the empirical reliability of variables along with the average reliability vectors, accounting for the multilevel structure in the estimation of their covariance. Significance: Literature has recognized that well-executed small/moderate scale empirical studies can also offer critical contributions to theory and practice. The combination of small/moderate sample sizes coupled with sophisticated n-level SEMs poses challenges for estimation methods because stable and unbiased estimates typically demand a large sample-to-parameter ratio. The results of this study coupled with those of other studies suggest that the proposed SAM estimator serves as an important alternative/complement when samples contain fewer than about 100 clusters/teachers/schools (depending on model complexity).
Descriptors: Maximum Likelihood Statistics, Structural Equation Models, Sample Size, Faculty Development, Measurement Techniques, Comparative Analysis, Teacher Improvement, Academic Achievement, Correlation, Regression (Statistics), Teacher Effectiveness, Bayesian Statistics, Evaluators, Teacher Evaluation, Lesson Plans
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A
Author Affiliations: N/A