Precision is a key facet of test development, with score reliability determined primarily according to the types of error one wants to approximate and demonstrate. This article identifies and discusses several primary forms of reliability estimation: internal consistency (i.e., split-half, KR-20, a), test-retest, alternate forms, interscorer, and…

Formula scoring (FS) is the use of a don't know option (DKO) with subtraction of points for wrong answers. Its effect on construct validity and reliability of progress test scores, is subject of discussion. Choosing a DKO may not only be affected by knowledge level, but also by risk taking tendency, and may thus introduce construct-irrelevant…

Studies based on longitudinal data focus on the change and development of the situation being investigated and allow for examining cases regarding education, individual development, cultural change, and socioeconomic improvement in time. However, as these studies require taking repeated measures in different time periods, they may include various…

Stewart questioned Beglar's use of Rasch analysis of the Vocabulary Size Test (VST) and advocated the use of 3-parameter logistic item response theory (3PLIRT) on the basis that it models a non-zero lower asymptote for items, often called a "guessing" parameter. In support of this theory, Stewart presented fit statistics derived from…

The surgical theatre educational environment measures STEEM, OREEM and mini-STEEM for students (student-STEEM) comprise an up to now disregarded systematic overestimation (OE) due to inaccurate percentage calculation. The aim of the present study was to investigate the magnitude of and suggest a correction for this systematic bias. After an…

Described herein are formulas and computational procedures for estimating the mean and second through fourth central moments of universe scores through multiple matrix sampling. Additionally, procedures are given for approximating the standard error associated with each estimate. All procedures are applicable when items are scored either…

The consequences of formula scoring versus number right scoring are examined in relation to the assumptions commonly made about the behavior of examinees in testing situations. The choice between the two is shown to be dependent upon having reduced error variance or unbiasedness as a goal. (Author/JKS)

A rebuttal of Frary's 1969 article in Educational and Psychological Measurement. (MS)

The standard error of measurement (SEM) is a measure of the inconsistency in the scores of a particular group of test-takers. It is largest for test-takers with scores ranging in the 50 percent correct bracket; with nearly perfect scores, it is smaller. On tests used to make pass/fail decisions, the test-takers' scores tend to cluster in the range…

The standard error of measurement as a means for estimating the margin of error that should be allowed for in test scores is discussed. The true score measures the performance that is characteristic of the person tested; the variations, plus and minus, around the true score describe a characteristic of the test. When the standard deviation is used…

This paper presents a scientific rationale for deciding the number of points to use on a grading scale in any given assessment situation. The rationale is applied to two common methods of assessment (multiple-choice and essay tests) and an example of a composite assessment. (Author/SJL)

A brief note presenting algebraically equivalent formulas for the variances of three error types. (Author)

There are difficulties in expressing effect sizes on a common metric when some studies use transformed scales to express group differences, or use factorial designs or covariance adjustments to obtain a reduced error term. A common metric on which effect sizes may be standardized is described. (Author/RL)

Supposing different values of the standard measurement error, the relation of scale coarseness to the total amount of error is studied on the basis of probability distribution of error. The analyses are performed within two models of error and with two criteria of amount of error. (Editor/SJL)

Because there is no commonly accepted view of what makes for good writing, automated essay scoring (AES) ideally should be able to accommodate different theoretical positions, certainly at the level of state standards but also perhaps among teachers at the classroom level. This paper presents a practical approach and an interactive computer…