Three types of methodological issues particularly salient in research involving the assessment of substance use or abuse are discussed with strategies for avoiding problems: (1) the reliability and validity of methods; (2) the variability and episodic course of substance use; and (3) the heterogeneity of individuals with substance use disorders.…

Six articles about evaluating information retrieval (IR) systems and methods are briefly described in this introduction to a special issue and some of the major problems that are encountered in evaluating interactive information retrieval are considered. (two references) (LAE)

In this response to critiques (Mudford, Hogg and Roberts 1997, 1999) of the use of behavior states in research involving individuals with mental retardation, it is argued that the work on behavioral state analysis by Robert D. Guess has contributed to the field at the practical, empirical, and theoretical levels. (Contains references.) (CR)

Discusses how student self reported gains or growth during college have become prominent dependent variables in much of the college impact research. Argues that, as typically used in existing studies, self reported gains or growth do not permit the same level of internal validity as does assessing gains with a pretest-posttest design. Suggests a…

A major challenge for researchers and educators has been to discern the effect of ICT use on student learning outcomes. This paper maps the achievements in Year 10 Science of two cohorts of students over two years where students in the first year studied in a traditional environment while students in the second took part in a blended or e-learning…

"How New York City's Charter Schools Affect Achievement" estimates the effects on student achievement of attending a New York City charter school rather than a traditional public school and investigates the characteristics of charter schools associated with the most positive effects on achievement. Because the report relies on an…

Research on facilitated communication (FC) is discussed, along with flaws in previous validation studies and suggestions for improving validation studies, including: testing first whether the person can do the task required under quiet, relaxed conditions; allowing the person to practice the skill until it can be used during the validation test;…

In educational effectiveness research, multilevel models are increasingly used because these models take the multilevel structure of the data into account. In this paper, the effect of ignoring one or more levels of variation in hierarchical linear regression is explored, using a model with four hierarchical levels (individual pupil, class group,…

This book evaluates methodologies in epidemiologic and related studies for obtaining measurements of exposure to environmental tobacco smoke (ETS). The book is divided into three parts. The first part discusses physicochemical and toxicological studies of environmental tobacco smoke, including physicochemical nature of smoke and in vivo and in…

Two questions are addressed in this document: What is worth knowing about Project Follow Through? and, How should the National Institute of Education (NIE) evaluate the Follow Through program? Discussion of the first question focuses on findings of past Follow Through evaluations, problems associated with the use of experimental design and…

The use of fieldwork in evaluation presents the fieldworker and agencies connected with educational evaluation with a number of critical problems. Four of these problems are identified and discussed: 1) definition of the "problem"; 2) multiplicity of signals, expectations, and responses; 3) dispensation of fieldwork data; and 4) collection of data…

This presentation outlines a perspective on job analysis and comments on research reported at a symposium with reference to that perspective. The perspective discusses two questions pertaining to job analysis that are referred to as the Stage I and Stage II evaluation operations: What should be done? and How well was it done? Stage I evaluation is…

Methodological incarceration occurs when research and investigation are restricted by the traditional concepts of how a study should be designed and are hampered by those notions which dictate what research is. This paper examines several reasons why social science research in general--reading research in particular--is considered by some to be…

External political pressures which influence the role and methodology of evaluation are described in the lead article by Henry M. Brickell. These influences, according to the author, are constantly present and are often more powerful in their effect than the actual evaluation findings. Based upon his experiences as an evaluator of educational…

There is general agreement that human services (i.e., services that depend on direct interpersonal contact between a deliverer and a client) are difficult to evaluate. The author points out some sources of this difficulty and proposes a strategy for the evaluation of human services delivery. (Author/GDC)