Background: Digital-first assessments leverage the affordances of technology in all elements of the assessment process--from design and development to score reporting and evaluation to create test taker-centric assessments. Objectives: The goal of this paper is to describe the engineering, machine learning, and psychometric processes and…

Examinees' performances are assessed using a wide variety of different techniques. Multiple-choice (MC) tests are among the most frequently used ones. Nearly, all standardized achievement tests make use of MC test items and there is a variety of ways to score these tests. The study compares number right and liberal scoring (SAC) methods. Mixed…

Assessment instruments composed of two-tier multiple choice (TTMC) items are widely used in science education as an effective method to evaluate students' sophisticated understanding. In practice, however, there are often concerns regarding the common scoring methods of TTMC items, which include pair scoring and individual scoring schemes. The…

A total of 19 graduate students enrolled in a graduate course conducted 6 consecutive administrations of the Wechsler Intelligence Scale for Children, 4th edition (WISC-IV, Canadian version). Test protocols were examined to obtain data describing the frequency of examiner errors, including administration and scoring errors. Results identified 511…

Four essential dimensions of a performance test are detailed: directness of test method, type of criterion, standardization of conditions, and objectivity of scoring. For simplicity these factors are described as if each were dichotomous, when in actuality each is a continuum; a test method may be more or less direct, conditions more or less…

This article discusses practical and theoretical problems related to the measurement of formal operations. The first section of the article discusses problems in measuring formal operations using the clinical interview method. These problems include the lack of both a standardized interview and a uniform scoring procedure. Section two discusses…

Measurement problems which jeopardize the reliability and validity of competency-based writing assessments are analyzed. Methods to stabilize rating criteria and readers' application of them are necessary. Most writing assessment programs use guidelines from norm-referenced test methodology. Use of this method of criteria application based on…

Certain testing procedures will overcome some of the problems associated with the use of essay tests. Essay tests may not validly indicate achievement because the questions included in the test may not fairly represent instructional content. Reliability may be a problem because of variations in examinee response in different situations, in test…

There exist special problems in testing first-grade children. Orally administered yes-no tests reduce the problems found in the other types, but they have their own drawbacks. A solution to some of these drawbacks is the use of the matched-pair scoring technique. For each "yes" item on the test there is included a "reversed" or…

The effects of repeaters (testees included in both administrations of two forms of a test) on the test equating process are examined. It is shown that repeaters do effect test equating and tend to lower the cutoff point for passing the test. (JKS)

Investigated scoring agreement among three different training levels of Visual Motor Integration Test (VMI) diagnosticians. Correlational data demonstrated high interexaminer reliabilities; however, there were gross errors in precision after raw scores had been converted into VMI age equivalent scores. (Author/RC)

The Selected Marker Tests were examined for scoring problems and internal consistency and were administered orally to sixth and seventh graders. Scoring problems were discovered and changes were suggested. The problem was found to be item reliability rather than interrater reliability. (Author/MH)

Procedures for conducting functional level testing have been available for use by practitioners for some time. However, the Title I Evaluation and Reporting System (TIERS), developed in response to the educational amendments of 1974 to the Elementary and Secondary Education Act (ESEA), has provided the impetus for widespread adoption of this…

The purpose of this investigation was to establish the effects of repeaters on test equating. Since consideration was not given to repeaters in test equating, such as in the derivation of equations by Angoff (1971), the hypothetical effect needed to be established. A case study was examined which showed results on a test as expected; overall mean…

Investigated certain measurement properties of the Ethical Judgment Scale. Results revealed findings of questionable validity and unacceptably low reliability for the instrument even though three different scoring methods were used in the analysis. (Author)