Evaluators are often called upon to assess the effects of programs. To assess a program effect, evaluators need a clear understanding of how a program effect is defined. Arguably, the most widely used definition of a program effect is the counterfactual one. According to the counterfactual definition, a program effect is the difference between…

Background and context: The variables that researchers measure and how they measure them are central in any area of research, including computing education. Which research questions can be asked and how they are answered depends on measurement. Objective: To summarize the commonly used variables and measurements in computing education and to…

We describe the development, validation, and use of the "Learning Progression-Based Assessment of Modern Genetics" (LPA-MG) in a high school biology context. Items were constructed based on a current learning progression framework for genetics (Shea & Duncan, 2013; Todd & Kenyon, 2015). The 34-item instrument, which was tied to…

Background: Collecting data on instructional practices is an important step in planning and enacting meaningful initiatives to improve undergraduate science instruction. Self-report survey instruments are one of the most common tools used for collecting data on instructional practices. This paper is an instrument- and item-level analysis of…

The 68-item Research Identity Scale (RIS) was informed through qualitative exploration of research identity development in master's-level counseling students and practitioners. Classical psychometric analyses revealed the items had strong validity and reliability and a single factor. A one-parameter Rasch analysis and item review was used to…

The purpose of the study is to have a relook at the ESL reading comprehension assessment system for Malaysian Year Five students. Traditionally, the ESL teachers have been assessing and reporting on their primary year's students by merely giving a composite grade with some vague remarks. This process has been used and is still being employed in…

We investigated the use of adaptive comparative judgment to evaluate the middle school student learning, engagement, and experience with the design process in an open-ended problem assigned in a technology and engineering education course. Our results indicate that the adaptive comparative judgment tool effectively facilitated the grading of the…

Consideration of cultural and linguistic features is mandated by law and one of the most important contributors to the appropriate placement of diverse learners for special education services. This article describes the results of a pilot project designed to assist rural educators to deliver an equitable and unbiased assessment process for English…

School districts and states across the Regional Educational Laboratory Mid-Atlantic Region and the country as a whole have been modifying their teacher evaluation systems to identify more effective and less effective teachers and provide better feedback to improve instructional practice. The new systems typically include components related to…

Understanding the reason(s) for Differential Item Functioning (DIF) in the context of measurement is difficult. Although identifying potential DIF items is typically a statistical endeavor, understanding the reasons for DIF (and item repair or replacement) might require investigations that can be informed by qualitative work. Such work is…

Internationally, many assessment systems rely predominantly on human raters to score examinations. Arguably, this facilitates the assessment of multiple sophisticated educational constructs, strengthening assessment validity. It can introduce subjectivity into the scoring process, however, engendering threats to accuracy. The present objectives…

Objective: Two studies examined the validity of using the Posttraumatic Growth Inventory (PTGI) to assess growth following the transition to grandparenthood as an aid for social workers seeking to promote strengths-based interventions for this population. Method: In Study 1 (n = 210 grandparent/offspring pairs), first-time grandparents'…

This study presents the development process and initial validation of the NyTid test, a process-oriented movement assessment tool for compulsory school pupils. A sample of 1,260 (627 girls and 633 boys; mean age of 14.39) Swedish school children participated in the study. In the first step, exploratory factor analyses (EFAs) were performed in…

Science communication competence (SCC) is an important educational goal in the school science curricula of several countries. However, there is a lack of research about the structure and the assessment of SCC. This paper specifies the theoretical framework of SCC by a competence model. We developed a qualitative assessment method for SCC that is…

This article describes the development of the Redox Concept Inventory (ROXCI) as a measure of students' understandings and confidence of both the symbolic and particulate domains of oxidation-reduction (redox) reactions. The ROXCI was created using a mixed-methods design in which the items were developed based upon themes that emerged from…