Although it has long been acknowledged that interviewers play a crucial role in the survey data collection process, there is little research concerning interviewer effects on how respondents perceive the interview. We investigate whether interviewer effects exist regarding how much respondents report having enjoyed the interview and whether these…

The purpose of this study is to introduce a functional approach for modeling unfolding response data. Functional data analysis (FDA) has been used for examining cumulative item response data, but a functional approach has not been systematically used with unfolding response processes. A brief overview of FDA is presented and illustrated within the…

The method of web probing integrates cognitive interviewing techniques into web surveys and is increasingly used to evaluate survey questions. In a usual web probing scenario, probes are administered immediately after the question to be tested (concurrent probing), typically as open-ended questions. A second possibility of administering probes is…

It is common to find mixed-format data results from the use of both multiple-choice (MC) and constructed-response (CR) questions on assessments. Dealing with these mixed response types involves understanding what the assessment is measuring, and the use of suitable measurement models to estimate latent abilities. Past research in educational…

We investigated the effect of careless or insufficient effort (C/IE) responses in a study using Amazon's Mechanical Turk. A factor mixture model was used to identify latent classes based on the pattern of responses with biases and examine the effect of C/IE responses on the fit of the theoretical model.

For large-scale assessments, data are often collected with missing responses. Despite the wide use of item response theory (IRT) in many testing programs, however, the existing literature offers little insight into the effectiveness of various approaches to handling missing responses in the context of scale linking. Scale linking is commonly used…

Test-taking behavior is a potential source of construct irrelevant variance for test scores in international large-scale assessments where test-taking effort, motivation, and behaviors in general tend to be confounded with test scores. In an attempt to disentangle this relationship and gain further insight into examinees' test-taking processes,…

Medical simulations allow trainees to work within teams to develop their self-regulated learning (SRL) and socially-shared regulated learning (SSRL) skills (Bransen et al., 2022). Both skillsets help to better prepare medical trainees for the multifaceted challenges inherent in clinical practice. SRL skills are imperative in empowering learners to…

Surveys are typical for student evaluation of teaching (SET). Survey research consistently confirms the negative impacts of careless responses on research validity, including low data quality and invalid research inferences. SET literature seldom addresses if careless responses are present and how to improve. To improve evaluation practices and…

Bounded continuous data are encountered in many applications of item response theory, including the measurement of mood, personality, and response times and in the analyses of summed item scores. Although different item response theory models exist to analyze such bounded continuous data, most models assume the data to be in an open interval and…

Trend scoring constructed response items (i.e. rescoring Time A responses at Time B) gives rise to two-way data that follow a product multinomial distribution rather than the multinomial distribution that is usually assumed. Recent work has shown that the difference in sampling model can have profound negative effects on statistics usually used to…

This article analyzes whether response patterns in surveys differ between the general population, regular immigrants, and recent refugees. Analyses show that the address quality of refugees contacted in the first wave of a panel study is worse than that of the general population, but of a similar quality to that of other recent immigrants. Once…

Interactive tasks designed to elicit real-life problem-solving behavior are rapidly becoming more widely used in educational assessment. Incorrect responses to such tasks can occur for a variety of different reasons such as low proficiency levels, low metacognitive strategies, or motivational issues. We demonstrate how behavioral patterns…

A question asking for respondents' sex is one of the standard sociodemographic characteristics collected in a survey. Until now, it typically consisted of a simple question (e.g., "Are you…?") with two answer categories ("male" and "female"). In 2019, Germany implemented the additional sex designation divers for…

The purpose of this study is to demonstrate clustering methods within a functional data analysis (FDA) framework for identifying subgroups of individuals that may be exhibiting categories of misfit. Person response functions (PRFs) estimated within a FDA framework (FDA-PRFs) provide graphical displays that can aid in the identification of persons…