In the process of school self-evaluation (SSE), teachers are often asked to provide information on the school's functioning by completing questionnaires. However, respondents can leave one item or a series of items unanswered, after which they continue completing the questionnaire. This phenomenon of item nonresponse is not necessarily problematic…

There is a need to investigate the diagnostic utility of autism diagnostic questionnaires in school-age children, who are increasingly being referred for autism assessment. Aside from the standardization sample, little research has been conducted on the Autism Spectrum Rating Scales, particularly regarding teacher reports. This study investigated…

The rapid advancements in generative artificial intelligence have opened new avenues for enhancing various aspects of research, including the design and evaluation of survey questionnaires. However, the recent pioneering applications have not considered questionnaire pretesting. This article explores the use of GPT models as a useful tool for…

Questionnaires in educational research assessing students' attitudes and beliefs are low-stakes for the students. As a consequence, students might not always consistently respond to a questionnaire scale but instead provide more random response patterns with no clear link to items' contents. We study inter-individual differences in students'…

Although self-report surveys are widely used for data collection, data quality can vary across populations because certain groups are more likely to engage in insufficient effort responding (IER). Our study examined how different levels of the educational system--student groups, schools, and cultural contexts--affect data quality due to IER, using…

Mostly in web surveys, attention checks have been proposed to identify inattentive respondents in self-administered surveys as previous research has argued that low-quality answers may introduce severe biases in data analyses. The increasing popularity of mixing survey modes for conducting probability-based surveys amplifies the need for…

A process is proposed to create the one-dimensional expected item characteristic curve (ICC) and test characteristic curve (TCC) for each trait in multidimensional forced-choice questionnaires based on the Rank-2PL (two-parameter logistic) item response theory models for forced-choice items with two or three statements. Some examples of ICC and…

Online surveys are widely used in social science research as well as in empirical studies of open, online, and distance education. However, students' responses are likely to be at odds with their actual behavior. In this context, we examined the discrepancies between self-reported use and actual use (i.e., learning analytics data) among 20,646…

Questionnaires are by far the most common tool for measuring noncognitive constructs in psychology and educational sciences. Response bias may pose an additional source of variation between respondents that threatens validity of conclusions drawn from questionnaire data. We present a mixture modeling approach that leverages response time data from…

Questionnaire design choices, such as handling 'do not know' answers or using check all that apply questions may impact the level of item- and unit nonresponse. We used experimental data from an online panel and the questions of ESS to examine how applying forced answering, offering DK/NA options and specific question formats (CATA vs. forced…

Recent years have seen a growing number of studies investigating the accuracy of nonprobability online panels; however, response quality in nonprobability online panels has not yet received much attention. To fill this gap, we investigate response quality in a comprehensive study of seven nonprobability online panels and three probability-based…

Previous research on grid questions has mostly focused on their comparability with the item-by-item method and the use of shading to help respondents navigate through a grid. This study extends prior work by examining whether lexical similarity among grid items affects how respondents answer the questions in an experiment where we manipulated…

This study examined the impact of two questionnaire characteristics, scale position and questionnaire length, on the prevalence of random responders in the TIMSS 2015 eighth-grade student questionnaire. While there was no support for an absolute effect of questionnaire length, we did find a positive effect for scale position, with an increase of…

Questionnaire designers are encouraged to write questions as complete sentences. In self-administered surveys, incomplete question stems may reduce visual clutter but may also increase burden when respondents need to scan the response options to fully complete the question. We experimentally examine the effects of three categories of incomplete…

This paper presents the item and test information functions of the Rank two-parameter logistic models (Rank-2PLM) for items with two (pair) and three (triplet) statements in forced-choice questionnaires. The Rank-2PLM model for pairs is the MUPP-2PLM (Multi-Unidimensional Pairwise Preference) and, for triplets, is the Triplet-2PLM. Fisher's…