The possible dependency of criterion validity on item formulation in a multicomponent measuring instrument is examined. The discussion is concerned with evaluation of the differences in criterion validity between two or more groups (populations/subpopulations) that have been administered instruments with items having differently formulated item…

Although list-style open-ended questions generally help us gain deeper insights into respondents' thoughts, opinions, and behaviors, the quality of responses is often compromised. We tested a dynamic and a follow-up design to motivate respondents to give higher quality responses than with a static design, but without overburdening them. Our…

The measurement of political solidarities and related concepts is an important endeavor in numerous scientific disciplines, such as political and social science research. European surveys, such as the Eurobarometer, frequently measure these concepts for people's home country and Europe raising questions with respect to the order of precedence.…

This paper presents a study focusing on the feasibility and validity of the SELFIEforTEACHERS self-assessment tool for student teachers, addressing three research questions. Using a mixed methods approach with a convergent parallel design. The study provides insights into quantitative and qualitative aspects of the self-assessment of student…

This article reports on the development of the literary portfolio as an alternative to standard written tests in summative assessment of literary competences in the English as a foreign language (EFL) classroom. It is argued that traditional class tests put a too narrow focus on those aspects of literary competences that are more readily…

Much psychological research depends on participants' diligence in filling out materials such as surveys. However, not all participants are motivated to respond attentively, which leads to unintended issues with data quality, known as careless responding. Our question is: how do different modes of data collection--paper/pencil, computer/web-based,…

The application of knowledge to new contexts (i.e., transfer) is a central aim of learning processes and has become a new focus of testing effect research. In a quasi-experimental design, we investigated the transfer of retrieval practice effects on English vocabulary learning on eighth-grade students (N = 182) by applying a typical testing effect…

Testwiseness may introduce construct-irrelevant variance to multiple-choice test scores. Presenting response options sequentially has been proposed as a potential solution to this problem. In an experimental validation, we determined the psychometric properties of a test based on the sequential presentation of response options. We created a strong…

According to teaching-learning theories, interventions in educational practice require a valid diagnosis of students' learning preconditions. Cognitive dispositions at the beginning of studies are the strongest predictor of knowledge acquisition. In the context of increasing numbers of beginning students with heterogeneous preconditions, valid…

Online-quizzes are an economic and objective method for formative assessment in universities. However, closed questions have been criticized for promoting shallow learning and resulting often in poor learning outcomes. These disadvantages can be overcome by embedding closed questions in effective instructional designs involving feedback. In the…

Optics is a core field in the curricula of secondary physics education. In this study, we present the development and validation of a test instrument in the field of optics, the ray optics in converging lenses concept inventory (ROC-CI). It was developed for and validated with middle school students, but can also be adapted for the use in higher…

This classroom experiment investigates the effects of adding representational pictures to multiple-choice and constructed-response test items to understand the role of the response format for the multimedia effect in testing. Participants were 575 fifth- and sixth-graders who answered 28 science test items--seven items in each of four experimental…

The elicited imitation task (EIT), in which language learners listen to a series of spoken sentences and repeat each one verbatim, is a commonly used measure of language proficiency in second language acquisition research. The "TOEFL® Essentials"™ test includes an EIT as a holistic measure of speaking proficiency, referred to as the…