Many test developers try to ensure the content validity of their tests by having external experts review the items, e.g. in terms of relevance, difficulty, or clarity. Although this approach is widely accepted, a closer look reveals several pitfalls need to be avoided if experts' advice is to be truly helpful. The purpose of this paper is to…

The purpose of this systematic review was to characterize the implementation of reliability generalization meta-analytic (RGM) practices within mathematics education-related empirical research. RGM studies are used to investigate and generalize the reliability of a measure across various studies. An exhaustive literature search was conducted to…

Background: Research into the neuropsychology of pediatric obsessive--compulsive disorder (OCD) reveals inconsistent results, limiting the ability to draw conclusions about possible neurocognitive deficits in youth with OCD. The aim of this study was to conduct a meta-analysis of the available literature. Methods: We identified 36 studies, of…

Examined the historical growth in the popularity of statistical significance testing using a random sample of data from 12 American Psychological Association journals. Results replicate and extend findings from a study that used only one such journal. Discusses the role of statistical significance testing and the use of replication and…

The Hubbard and Ryan article is a little ingenuous in its implications for action. Both meta-analyses and replication have problems of their own; each requires careful attention. Good measurement emphasizes proper sampling techniques and using whatever means possible to analyze data. (SLD)

Agrees with R. Hubbard and P. Ryan that statistical significance testing has had a negative impact in that some users have closed their minds to alternative approaches to conducting research. In marketing, the alternatives are not completely satisfactory, however, and researchers are likely to continue to rely on statistical significance testing.…

Summarizes issues associated with psychological assessment, concluding that: psychological test validity is strong and is comparable to medical test validity; distinct assessment methods provide unique sources of information; and clinicians who rely solely on interviews are prone to incomplete understandings. Suggests that multimethod assessment…

There is scientific and popular dispute about whether there are sex differences in cognitive abilities and whether they are relevant to the proportions of men and women who attain high-level achievements, such as Nobel Prizes. A recent meta-analysis (Lynn, R., and Irwing, P. (2004). Sex differences on the progressive matrices: a meta-analysis.…

Suggests that replication research and meta-analysis are not substitutes for statistical significance testing, but rather, like measures of effect size, they are complements to statistical significance testing. Significance testing does provide a means for determining what might be usefully replicated. (SLD)

Suggest that the call for more pointed graduate education and more affirmative journal policies on replication-extension made by R. Hubbard and P. Ryan is useful, although inadequate and probably pointless. Statistical significance testing appears to be here to stay despite the charge that it is of "marginal scientific value." (SLD)

The 14 chapters of this book focus on the technical advances, advances in applied settings, and emerging topics in the testing field. Part 1 discusses methodological advances, Part 2 considers developments in applied settings, and Part 3 reviews emerging topics in the field of testing. Part 1 papers include: (1) "Advances in…