While the field advocates for a diversified and comprehensive professional role (National Association of School Psychologists, 2020), school psychologists have long spent most of their time in assessment-related activities (Farmer et al., 2021), averaging about eight cognitive evaluations monthly (Benson et al., 2020). Assessment practices have…

In Part 1 of this series, the authors briefly reviewed some challenges inherent in using standardized tools with students who are not well represented in norming data. To help readers clearly conceptualize the framework steps, the authors present two case studies that showcase how a nonstandardized approach to assessment can be individualized to…

Digitally-based assessments create opportunities for collecting moment to moment information about how students are responding to assessment items. This information, called log or process data, has long been regarded as a vast and valuable source of data about student performance. Despite repeated assurances of its vastness and value, process data…

Automatic item generation may supply many items instantly and efficiently to assessment and learning environments. Yet, the evaluation of item quality persists to be a bottleneck for deploying generated items in learning and assessment settings. In this study, we investigated the utility of using large-language models, specifically Llama 3-8B, for…

Recent research has demonstrated an intersectional approach to the study of differential item functioning (DIF). This approach expands DIF to account for the interactions between what have traditionally been treated as separate grouping variables. In this paper, we compare traditional and intersectional DIF analyses using data from a state testing…

Technical documentation for educational tests focuses primarily on properties of individual scores at single points in time. Reliability, standard errors of measurement, item parameter estimates, fit statistics, and linking constants are standard technical features that external stakeholders use to evaluate items and individual scale scores.…

The purposes of this research are: 1) to compare two equalizing tests conducted with Hebara and Stocking Lord method; 2) to describe the characteristics of each equalizing test method using windows' IRTEQ program. This research employs a participatory approach as the data are collected through questionnaires based on the National Examination…

In this article we present a general approach not relying on item response theory models (non-IRT) to detect differential item functioning (DIF) in dichotomous items with presence of guessing. The proposed nonlinear regression (NLR) procedure for DIF detection is an extension of method based on logistic regression. As a non-IRT approach, NLR can…

An important assumption of item response theory is item parameter invariance. Sometimes, however, item parameters are not invariant across different test administrations due to factors other than sampling error; this phenomenon is termed item parameter drift. Several methods have been developed to detect drifted items. However, most of the…

The Domain Five Observation Instrument (DFOI) is a competency-based observation instrument recommended for sport leaders or researchers who wish to evaluate coaches' instructional behaviors. The DFOI includes 10 behavior categories and four timed categories that encompass 34 observable instructional benchmarks outlined in domain five of the…

This report reviews the major international and regional large-scale educational assessments, including international surveys, school-based surveys and household-based surveys. The report compares and contrasts the cognitive and contextual data collection instruments and implementation methods used by the different assessments in order to identify…

Evidence of comparability is generally needed whenever there are variations in the conditions of an assessment administration, including variations introduced by the administration of an assessment on multiple digital devices (e.g., tablet, laptop, desktop). This article is meant to provide a comprehensive examination of issues relevant to the…

This case study describes a method of collecting data on students' experiences of developing digital literacy (information and communications technology) skills as part of their course at the United Kingdom's Open University. An online reflective quiz was integrated into three health and social care modules, offering students the opportunity both…

This manuscript chronicles the process and products of a redesign for evaluation of the graduate college experience (GCE) which was initiated by a university graduate college, based on its observed need to reconsider and update its measures and methods for assessing graduate students' experiences. We examined the existing instrumentation and…

"The First-Year Seminar: Designing, Implementing, and Assessing Courses to Support Student Learning and Success," a five-volume series, is designed to assist educators who are interested in launching a first-year seminar or revamping an existing program. Each volume examines a different aspect of first-year seminar design or…