Educational researchers often need to construct precise and reliable measurement scales of complex and varied representations such as participants' written work, videoed lesson segments and policy documents. Developing such scales using can be resource-intensive and time-consuming, and the outcomes are not always reliable. Here we present…

Validity is broadly defined as how well something measures what it's supposed to measure. The reliability and validity of scores from assessments are two concepts that are closely knit together and feed into each other.

In Australia, one of the key findings of the Teacher Education Ministerial Advisory Group was that not all graduating pre-service teachers possess adequate pedagogical content knowledge (PCK) to teach effectively. The concern is that higher education providers working with pre-service teachers are using pedagogical practices and assessments which…

In using the Rasch model to improve the quality of an instrument, analysis purports to determine if the sample collaborates well with the items in the instrument such that the results are measuring a single underlying variable. The relevant properties of Rasch analysis are reliability and validity which are key indicators of the quality of a…

The paper presents first the theoretical foundations used to develop a pre-experimental version of a questionnaire on relationship to work, and then the four stages of its initial validation leading to an experimental version. These stages included: (1) Defining the dimensions and sub-dimensions of the relationship to work concept; (2)…

Test writing is a fundamental component of teaching. With increasing pressure to teach larger groups of students, conduct formal assessment of learning outcomes, and offer online and hybrid classes, there is a need for alternatives to constructed response problem-solving test questions. We believe that appropriate use of multiple-choice (MC)…

Validity evidence based on test content is critical to meaningful interpretation of test scores. Within high-stakes testing and accountability frameworks, content-related validity evidence is typically gathered via alignment studies, with panels of experts providing qualitative judgments on the degree to which test items align with the…

Compared to typical schools, implementation of Positive Behavioral Interventions and Supports (PBIS) remains in its early stages with juvenile justice and alternative education settings, whether residential or day programs. As interest and support grows for implementation within juvenile justice and alternative education settings, the creation of…

The test view is on the Duolingo English Test (DET), an alternative online English proficiency test with a machine-driven characteristic. The review covers essential information of the DET such as test purpose, usage, score-mapping with CEFR scale, price, and publisher. Meanwhile, the test usefulness is discussed with focuses on reliability,…

General education science courses strive to promote scientific literacy and the development of scientific process skills. However, research shows that many general education courses are still designed to stress content mastery. In this study, the number of topics in five semester-long introductory atmospheric science courses was reduced to…

The main problem of the educational evaluation validity is that it just copies the conceptual framework system of validity from educational measurement to its own conceptual system. The validity conceptual system that fits the need of theory and practice of educational evaluation has not been established yet. According to the inherent attributive…

The content validity ratio (Lawshe) is one of the earliest and most widely used methods for quantifying content validity. To correct and expand the table, critical values in unit steps and at multiple alpha levels were computed. Implications for content validation are discussed. (Contains 2 tables and 1 figure.)

Students tend to believe that RateMyProfessors.com (RMP) is an unfailingly valid measure of teaching effectiveness, and in fact viewing that site prior to taking a course can have a direct influence on their future evaluations of faculty. The authors describe an activity whose objective is to have students analyze different types of validity using…

Assessment engineering is a new way to design and implement scalable, sustainable and ideally lower-cost solutions to the complexities of designing and developing tests. It represents a merger of sorts between cognitive task modeling and engineering design principles--a merger that requires some new thinking about the nature of score scales, item…

This paper aims to develop a valid method to assess the key competencies of the exercise physiology profession acquired through work-integrated learning (WIL). In order to develop a competency-based assessment, the key professional tasks needed to be identified and the test designed so students' competency in different tasks and settings could be…