Standard setting theory has largely developed with reference to a typical situation, determining a level or levels of performance for one exam for one context. However, standard setting is now being used with international reference frameworks, where some parameters and assumptions of classical standard setting do not hold. We consider the…

This study investigated the extent to which the performance data format impacted data use in Angoff standard setting exercises. Judges from two standard settings (a total of five panels) were randomly assigned to one of two groups. The full-data group received two types of data: (1) the proportion of examinees selecting each option and (2) plots…

Surveys play a prominent role in assessment and institutional research, and the NSSE College Student Report is one of the most popular surveys of enrolled undergraduates. Recent studies have raised questions about the validity of the NSSE survey. Although these studies have themselves been criticized, documenting the validity of an instrument…

This response to the comments contains three main sections, each addressing a subset of the comments. In the first section, I will respond to the comments by Brennan, Haertel, and Moss. All of these comments suggest ways in which my presentation could be extended or improved; I generally agree with their suggestions, so my response to their…

This paper examines the tension in the process of designing student surveys between the methodological requirements of good survey design and the institutional needs for survey data. Building on the commonly used argumentative approach to construct validity, I build an interpretive argument for student opinion surveys that allows assessment of the…

This study attempts to clarify conceptual and operational inconsistencies in the literature around "Ghiselli's hobo syndrome." I propose that defining characteristics of hobo syndrome should include both the exhibition of frequent job movement behavior and positive attitudes about such behavior. This definition effectively differentiates…

Contemporary literature lacks a comprehensive set of constructs that provides for understanding both the organizational and job factors that influence employee work passion. Through a detailed analysis of literature, this research identifies eight constructs that form the basis for the appraisal of an employee's work experience with the use of…

Three approaches of validation enquiry were applied on the data obtained from a proficiency test carried out with 3,398 PhD candidates as a partial requirement for entering PhD program in Iran. The data obtained from the reading section were subjected to an exploratory factor analysis (EFA). The EFA yielded nine factors in the reading section.…

The purpose of this study was to develop three school-based assessments and determine the content validity for each assessment. The School Activities and Participation Analysis-Elementary (SAPA-E) measures functional movement performance in children with physical disability attending the elementary school, and the School Activities and…

Adopting a constructivist perspective of adolescent development, we argue adolescents' perceptions of "being known" reflect teachers' authentic recognition of adolescents' multiple emerging identities. As such, adolescent perceptions of being known are a distinct factor associated with high school students' engagement in school. A mixed…

This study examined the criterion validity of the Child Behavior Checklist (CBCL) and Teacher's Report Form (TRF) problem scales and items in demographically-matched Singapore samples of referred and non-referred children (840 in each sample for the CBCL and 447 in each sample for the TRF). Internal consistency estimates for both the CBCL and TRF…

Using generalizability (G-) theory, this study examined the accuracy and validity of the writing scores assigned to secondary school ESL students in the provincial English examinations in Canada. The major research question that guided this study was: Are there any differences between the accuracy and construct validity of the analytic scores…

Cognitive and linguistic components related to spelling and writing in English as a second language (ESL) and native-English speaking (EL1) third graders were examined. ESL and EL1 children performed similarly on rapid naming, phonological awareness (PA), verbal short-term and working memory, reading fluency, single-word spelling, text spelling,…

Valid and reliable standardized assessment of nursing competencies is needed to monitor the quality of vocational education and training (VET) in nursing and evaluate learning outcomes for care work trainees with increasingly heterogeneous learning backgrounds. To date, however, the modeling of professional competencies has not yet evolved into…

Systematic reviews have traditionally focused on internal validity, while external validity often has been overlooked. In this study, we systematically reviewed determinants of external validity in the accumulated randomized controlled trials of social skills group interventions for children and adolescents with autism spectrum disorder. We…