The measurement error in principal components extracted from a set of fallible measures is discussed and evaluated. It is shown that as long as one or more measures in a given set of observed variables contains error of measurement, so also does any principal component obtained from the set. The error variance in any principal component is shown…

There is significant controversy over what counts as evidence in the evaluation of social interventions. It is increasingly common to use methodological criteria to rank evidence types in a hierarchy, with Randomised Controlled Trials (RCTs) at or near the highest level. Because of numerous challenges to a hierarchical approach, this article…

Borrowing from marital research literature, a four-square matrix constructed from the twin dimensions of effort and comfort levels is used to describe a typology of campus and community associations. Results from a study using the Optimal College Town Assessment to measure community member perceptions on town-gown relationships are presented next,…

The idea that multiple measures should be used when evaluating teachers is widely accepted. Multiple measures are important not only because education has multiple goals, but because each measure is an imperfect indicator of any given goal. For a variety of reasons, states and districts use multiple measures in one particular way: to make…

Assessment tasks can focus student efforts in what they choose to study. This paper describes the development and initial application of an assessment matrix to support student learning in postgraduate research studies at the University of Adelaide. Students and supervisors in the multi-disciplinary Master programmes also benefit from the clear…

The United States is in a bind. On the one hand, it needs millions of additional citizens with at least one year of successful post-secondary experience to adapt to the knowledge economy. The Gates and Lumina Foundations, as well as President Obama have championed this goal in different ways. On the other hand, it has a post-secondary system that…

Many children born with severe and multiple disabilities have complex communication needs and may use no speech or only minimal speech to communicate. Meaningful assessment of their expressive skills to identify communication strengths along a developmental trajectory is an essential first step toward appropriate intervention. This article…

The purpose of this study was to investigate the application of the parallel analysis (PA) method for choosing the number of factors in component analysis for situations in which data are dichotomous or ordinal. Although polychoric correlations are sometimes used as input for component analyses, the random data matrices generated for use in PA…

Figural matrix items are a popular task type for assessing general intelligence (Spearman's g). Items of this kind can be constructed rationally, allowing the implementation of computerized generation algorithms. In this study, the influence of different task parameters on the degree of difficulty in matrix items was investigated. A sample of N =…

When conducting an exploratory factor analysis, the decision regarding the number of factors to retain following factor extraction is one that the researcher should consider very carefully, as the decision can have a dramatic effect on results. Although there are numerous strategies that can and should be utilized when making this decision,…

In the School of Education at the University of Adelaide, the use of oral evidence is increasingly common as students engage with reflective practices now dominant in teacher-education programs. These experiences offer both a dynamic perspective and a challenge to academic assessors and raise three questions, each of which are addressed in this…

The originally proposed multivariate meta-analysis approach for correlation matrices--analyze Pearson correlations, with each study's observed correlations replacing their population counterparts in its conditional-covariance matrix--performs poorly. Two refinements are considered: Analyze Fisher Z-transformed correlations, and substitute better…

The criteria for assessing convergent and discriminant validity proposed by D. T. Campbell and D. W. Fiske (1959) are shown to be inadequate for either the additive or multiplicative structures of data in a multitrait-multimethod matrix. Model-specific criteria are more promising for assessing convergent and discriminant validity. (Author/SLD)

This paper evaluated multidimensional linking procedures with which multidimensional test data from two separate calibrations were put on a common scale. Data were simulated with known ability distributions varying on two factors which made linking necessary: mean vector differences and variance-covariance (v-c) matrix differences. After the…

Meta-analytic structural equation modeling (MA-SEM) is increasingly being used to assess model-fit for variables' interrelations synthesized across studies. MA-SEM researchers have analyzed synthesized correlation matrices using structural equation modeling (SEM) estimation that is designed for covariance matrices. This can produce incorrect…