Extends the theoretical and practical implications of organizational climate research based upon the data from two studies, Sims and LaFollette (1975) and Litwin and Stringer (1968). (Author/RK)

The estimation accuracy of procedures based on classical test score theory and item response theory (generalized partial credit model) were compared for examinations consisting of multiple-choice and extended-response items. Analysis of British Columbia Scholarship Examination results found an error rate of about 10 percent for both methods, with…

Using data from an individually administered interview schedule (the Consumer Satisfaction Inventory), reliability among nine interviewers was estimated with several statistical methods, including simple percentages of agreement, kappa and weighted kappa, Pearson correlations, t tests on interviewers' means, and generalizability theory techniques.…

Four indexes of agreement between ratings of a person that correct for chance and are interpretable as intraclass correlation coefficients for different analysis of variance models are investigated. Relationships among the estimators are established for finite samples, and the equivalence of these estimators in large samples is demonstrated. (SLD)

Addresses generalizability theory (GT), which offers a flexible framework for assessing dependability of measurement. GT allows for consideration of multiple sources of error, allowing investigators to assess the overall impact of measurement error. Illustrative analyses demonstrate the special advantages of GT for planning studies in which…

Discusses statistical procedures for increasing test unreliability due to guessing in multiple choice and true/false tests. Proposes two new measures of test unreliability: one concerned with resolution of defined levels of knowledge and the other with the probability of examinees being incorrectly ranked. Both models are based on the binomial…

Discusses how meta-analysis allows clinicians to determine objectively both presence and size of an effect or correlation within the existing literature by pooling the results of various studies and performing statistical analyses. Describes the risks and benefits of applying information obtained from meta-analysis into clinical practice.…

The application range of Cohen's Kappa is extended to the field of sequential observation data, where omission mistakes of an observer may often occur. It is shown how the omission mistakes can be incorporated into the calculation of the Kappa coefficient without violating the statistic it is based on. The enhanced coefficient is termed Kappa…

Traditionally, the fixed-length linear paper-and-pencil (P&P) mode of administration has been the standard method of test delivery. With the advancement of technology, however, the popularity of administering tests using adaptive methods like computerized adaptive testing (CAT) and multistage testing (MST) has grown in the field of measurement…

This article follows the development of test items (see "Language Assessment Quarterly", Volume 3 Issue 1, pp. 71-79 for the article "Test and Item Specifications Development"), beginning with a review of test and item specifications, then proceeding to writing and editing of items, pretesting and analysis, and finally selection of an item for a…

This study investigated some issues regarding the validity of the Scholastic Achievement Test (SAT) Subject Test: Korean with Listening. The SAT Korean has been administered just once a year since its inception in 1997. As of March 2006, it had been administered nine times. However, SAT foreign language tests are not as rigorously researched as…

Background: The Thinking Styles Inventory--developed by Sternberg and Wagner based on Sternberg's (1988, 1997) earlier theory of mental self-government--was selected for the research in order to assess thinking styles of student teachers. Another reason is that the theoretical constructs, as well as the inventory generated from the theory, have…

There is significant potential for error in long production processes that consist of sequential stages, each of which is heavily dependent on the previous stage, such as the SER (Scoring, Equating, and Reporting) process. Quality control procedures are required in order to monitor this process and to reduce the number of mistakes to a minimum. In…

This book is a guide to categorizing, measuring, testing, and assessing aspects of language, and is intended for language teachers, speech therapists and other language-related practitioners, and researchers, in conjunction with other resources on research methods and statistics. The first part is a discussion of basic terminology and the varied…

In 1994, the journal "Educational and Psychological Measurement" (EPM) instituted an editorial policy requiring authors to use technically appropriate language and methodological practices in their discussions of validity and reliability. To determine if this policy has had any effect on current publications, 150 validity and reliability…