For randomized trials that use text as an outcome, traditional approaches for assessing treatment impact require that each document first be manually coded for constructs of interest by trained human raters. This process, the current standard, is both time-consuming and limiting: even the largest human coding efforts are typically constrained to…

KR-21 reliability and its extension (coefficient [alpha]) gives the reliability estimate of test scores under the assumption of tau-equivalent forms. KR-21 reliability gives the reliability estimate for summed scores for dichotomous items when items are randomly sampled from an infinite pool of similar items (randomly parallel forms). The article…

This study offers a new method to measure navigation disorientation in web based systems which is powerful learning medium for distance and open education. The Needleman-Wunsch algorithm is used to measure disorientation in a more precise manner. The process combines theoretical and applied knowledge from two previously distinct research areas,…

Schools are scrambling to prepare students for the writing assessments aligned to the Common Core State Standards. In some states, writing has not been assessed for over a decade. Yet, with the use of computerized grading of the student's writing, many teachers are wondering how to best prepare students for the writing assessments that will…

The m-rater scoring engine has been used successfully for the past several years to score "CBAL"™ mathematics tasks, for the most part without the need for human scoring. During this time, various improvements to m-rater and its scoring keys have been implemented in response to specific CBAL needs. In 2012, with the general move toward…

Due to budgetary concerns, the FDOE restructured the Writing portion of the FCAT assessment program for 2010. Among the changes was a difference in how the writing essays were scored. In the past, two people independently judged each essay and each assigned a score from 1 to 6. In the event of a disagreement between judges, the average of the…

The Lithuanian Informatics Olympiad is a problem solving contest for high school students. The work of each contestant is evaluated in terms of several criteria, where each criterion is measured according to its own scale (but the same scale for each contestant). Several jury members are involved in the evaluation. This paper analyses the problem…

Describes a procedure for information retrieval testing that is based on the comparison of statistically independent methods of retrieval applied to the same database. The probability ranking principle is discussed, the statistical meaning of relevance is examined, and the methodology is illustrated on a large database of MEDLINE records. (19…

Suggests some ways to overcome the problems connected with the use of the Syntactic Density Score. (DD)

Presents a microcomputer-based qualitative discriminant process (QDP) for scoring and ranking in group support systems that is relevant to group decision-making situations that involve the evaluation of alternatives. Scoring and ranking techniques are described, and a case illustration using the QDP is discussed. (Contains 50 references.) (LRW)

The purpose of this paper is to provide an overview of the analysis of data collected by the National Assessment of Educational Progress (NAEP). In simplest terms, the analysis can be characterized as establishing baseline estimates of the percentages of young Americans possessing certain skills, knowledge, understandings, and attitudes and…

Some little-understood facts about standardized test scores and how they are reported and interpreted are explained. In particular, the practice of score equating ensures that different test forms have comparable scoring. This and other practices are designed to enhance the equity of testing. (MSE)

"Item models" (LaDuca, Staples, Templeton, & Holzman, 1986) are classes from which it is possible to generate/produce items that are equivalent/isomorphic to other items from the same model (e.g., Bejar, 1996; Bejar, 2002). They have the potential to produce large number of high-quality items at reduced cost. This paper introduces…