Fuzzy-set qualitative comparative analysis (fsQCA) has become one of the most prominent methods in the social sciences for capturing causal complexity, especially for scholars with small- and medium-"N" data sets. This research note explores two key assumptions in fsQCA's methodology for testing for necessary and sufficient…

An error in an essential equation within the article by Williams and Zimmerman is corrected, and the algebraic inequalities are translated into questions a researcher can ask about simple or residualized difference scores. Williams and Zimmerman acknowledge the error and note that main conclusions are not affected. (SLD)

Some of the methods originally used to find relationships between reliability and power associated with a single measurement are extended to difference scores. Results, based on explicit power calculations, show that augmenting the reliability of measurement by reducing error score variance can make significance tests of difference more powerful.…

This report describes a statistically designed experiment that was carried out in an operational setting to determine the contributions of different sources of variation to the unreliability of scoring. The experiment made novel use of partially balanced incomplete block designs that facilitated the unbiased estimation of certain main effects…

Observer disagreement is important because it limits the reliabilities of observational measures. To avoid this limitation, observers should be trained, and criterion-related and intraobserver agreement measures should be used both before and during a study. Several agreement coefficients are examined for their applicability. (Author/BW)

A model and computation procedure based on classical test score theory are presented for determination of a correlation coefficient corrected for attenuation due to unreliability. Delta and Monte Carlo method applications are discussed. A power analysis revealed no serious loss in efficiency resulting from correction for attentuation. (TJH)

The purposes of the present investigation were to examine the dimensionality of the Rotter Internal-External (I-E) scale, to test the invariance of its factor structure before and after an intervention designed to alter I-E, and to demonstrate the use of confirmatory factor analysis. An a priori model based on the content of the Rotter items…

This document contains three papers from the Methodology Project of the Center for the Study of Evaluation. Methods for characterizing test accuracy are reported in the first two papers. "Bounds on the K Out of N Reliability of a Test, and an Exact Test for Hierarchically Related Items" describes and illustrates how an extension of a…

Four different procedures were used for estimating the proportion of persons who would be classified consistently as either passing both of two parallel tests or failing both. These four methods were applied at each of four different mastery level scores for each of three different length tests. Data were based on 50 replications of each procedure…

Generalizability theory extends previous methods of estimating the reliability of rating instruments such that one can estimate the precision of a measurement system for differentiating among students, scales, or any other important dimension. In this study, generalizability theory is applied to faculty ratings of junior and senior dental students…

The paper identifies and addresses four methodological weaknesses common to most previous studies that have used LISREL confirmatory factor analysis to test for the factorial validity and invariance of a single measuring instrument. Specifically, the paper demonstrates the steps involved in: (1) conducting sensitivity analyses to determine a…

The Beginning Teacher Evaluation Study (BTES), Phase II, was a research project on effective teaching behavior--what teachers do that significantly affects what and how pupils learn. The purposes of Phase II were to (1) develop an assessment system for measuring teacher and pupil behaviors and other factors which could influence each of them and…

The IADIS CELDA 2012 Conference intention was to address the main issues concerned with evolving learning processes and supporting pedagogies and applications in the digital age. There had been advances in both cognitive psychology and computing that have affected the educational arena. The convergence of these two disciplines is increasing at a…