Developing models and using mathematics are two key practices in internationally recognized science education standards such as the Next Generation Science Standards (NGSS, 2013). In this paper, we used a virtual performance-based formative assessment to capture students' competencies at both "developing" and "evaluating"…

It is common practice for assessment programs to organize qualifying sessions during which the raters (often known as "markers" or "judges") demonstrate their consistency before operational rating commences. Because of the high-stakes nature of many rating activities, the research community tends to continuously explore new…

The present article considers a fundamental question in evaluation research: "By how much do program effects vary across sites?" The article first presents a theoretical model of cross-site impact variation and a related estimation model with a random treatment coefficient and fixed site-specific intercepts. This approach eliminates…

There has been much research and discussion on the principles of standards-based grading, and there is a growing consensus of best practice. Even so, the actual process of implementing standards-based grading at a school or district level can be a significant challenge. There are very practical questions that remain unclear, such as how the grades…

Sensitivity analysis is the study of how the different input variations of a mathematical model influence the variability of its output. In this paper, we review the principle of global and local sensitivity analyses of a complex black-box system. A simulated case of application is given at the end of this paper to compare both approaches.…

Learning to select optimal behavior in new and uncertain situations is a crucial aspect of living and requires the ability to quickly associate stimuli with actions that lead to rewarding outcomes. Mathematical models of reinforcement-based learning to select rewarding actions distinguish between (1) the formation of stimulus-action-reward…

Smaller universities may produce research which is on a par with larger, elite establishments. This is confirmed by a recently developed mathematical model, supported by data from British and French higher education research-evaluation exercises. The detailed nature of the UK system, in particular, allows quantification of the notion of critical…

This paper describes a variety of confirmatory factor analysis models that provide improved tests of multitrait-multimethod matrices, and compares three different approaches (the original Campbell-Fiske guidelines, an analysis of variance model, and confirmatory factor analysis models). (PN)

The utility of the fail-safe "N" statistic was evaluated by computing it for studies in three organizational research domains in which discrepant conclusions were reached by initial and subsequent meta-analyses. Calculation of the fail-safe "N" may have led to more cautious interpretations. Implications for meta-analyses are…

Two recent indices of fit, the Relative Noncentrality Index (RNI) (R. P. McDonald and H. W. Marsh, 1990) and the Comparative Fit Index (P. M. Bentler, 1990), are shown to be algebraically equivalent in most applications, although one condition in which the RNI may be advantageous for model comparison is identified. (SLD)

Various methods have been suggested for the analysis of data collected in research settings where random assignment of subjects to groups has not occurred. For the purposes of this paper the set of allowable nonrandomized designs is made up of those research designs where data are collected for one or more groups of subjects at two or more time…

To locate matches across pairs of lists without unique identifiers it is sometimes necessary to compare strings of letters. String comparators are used in production computer matching software during the Post Enumeration Survey for the 1990 U.S. census. A string comparator metric is described that partially accounts for: (1) typographical…

The problem of comparing proportions when some data are missing is investigated, and determination is made of what statistical techniques are appropriate under each of several probability models describing the observations likely to be missing. Monte Carlo methods were used to investigate the properties of standard estimators under each of the…

Complex decision-making environments often demand that a range of alternative strategies be identified and compared for their shared value in achieving desired ends. Simple parametric statistical procedures such as analysis of variance and discriminant function analysis can be used in such situations to compare (using selected criteria) the value…

A procedure for evaluating a variety of rater reliability models is presented. A multivariate linear model is used to describe and assess a set of ratings. Parameters are represented in terms of a factor analytic model, and maximum likelihood methods test the model parameters. Illustrative examples are presented. (SLD)