Mediation analysis is an important statistical tool in many research fields, where the joint significance test is widely utilized for examining mediation effects. Nevertheless, the limitation of this mediation testing method stems from its conservative Type I error, which reduces its statistical power and imposes certain constraints on its…

Structured means analysis is a very useful approach for testing hypotheses about population means on latent constructs. In such models, a z test is most commonly used for testing the statistical significance of the relevant parameter estimates or of the differences between parameter estimates, where a z value is computed based on the asymptotic…

The present paper aims to discuss how data from international large-scale assessments (ILSAs) can be utilized and combined, even with other existing data sources, in order to monitor educational outcomes and study the effectiveness of educational systems. We consider different purposes of linking data, namely, extending outcomes measures,…

Understanding students' learning dispositions has been a focus for research in education for many years. A range of alternative approaches to conceptualising and measuring this broad construct have been developed. Traditional psychometric measures aim to produce scales that satisfy the requirements for research; however, such measures have an…

Researchers in the behavioral and social sciences often have expectations that can be expressed in the form of inequality constraints among the parameters of a structural equation model resulting in an informative hypothesis. The questions they would like an answer to are "Is the hypothesis Correct" or "Is the hypothesis…

This study proposes robust means modeling (RMM) approaches for hypothesis testing of mean differences for between-subjects designs in order to control the biasing effects of nonnormality and variance inequality. Drawing from structural equation modeling (SEM), the RMM approaches make no assumption of variance homogeneity and employ robust…

Several methods for testing mediation hypotheses with 2-level nested data have been proposed by researchers using a multilevel modeling (MLM) paradigm. However, these MLM approaches do not accommodate mediation pathways with Level-2 outcomes and may produce conflated estimates of between- and within-level components of indirect effects. Moreover,…

Problem-based learning (PBL) is generally organized in three phases, involving collaborative and self-directed learning processes. The hypothesis tested here is whether learning in the different phases of PBL is cumulative, with learning in each phase depending on that of the previous phase. The scientific concepts recalled by 218 students at the…

There have been many discussions of how Type I errors should be controlled when many hypotheses are tested (e.g., all possible comparisons of means, correlations, proportions, the coefficients in hierarchical models, etc.). By and large, researchers have adopted familywise (FWER) control, though this practice certainly is not universal. Familywise…

Despite the recent increase of structural equation modeling (SEM) in language testing and learning research and Kunnan's (1998) call for the proper use of SEM to produce useful findings, there seem to be no reviews about how SEM is applied in these areas or about the extent to which the current application accords with appropriate practices. To…

Because structural equation models are widely used in testing and assessment, investigation into the accuracy of such models may help raise awareness of the value of reanalysis or replication. We focused on second language testing and learning studies and examined: (a) To what extent is information necessary for replication provided by authors?…

Researchers often have expectations that can be expressed in the form of inequality constraints among the parameters of a structural equation model. It is currently not possible to test these so-called informative hypotheses in structural equation modeling software. We offer a solution to this problem using M"plus." The hypotheses are…

Typical validation studies on standard setting models, most notably the Angoff and modified Angoff models, have ignored construct development, a critical aspect associated with all conceptualizations of measurement processes. Stone compared the Angoff and objective standard setting (OSS) models and found that Angoff failed to define a legitimate…

How students understand, feel about and respond to assessment might contribute significantly to learning behaviour and academic achievement. This paper reviews studies that have used a relatively new self-reported survey questionnaire ("Students' Conceptions of Assessment"--SCoA) about student perceptions and understandings of…

The Lothian Birth Cohort 1936 sample is in a uniquely good position to provide relevant data on social class mobility patterns over most of the last century. These participants, with known ultimate social class attainment, took a validated mental test in the Scottish Mental Survey of 1947 and were followed up at approximately age 70. Then, besides…