When cut scores for classifications occur on the total score scale, popular methods for estimating classification accuracy (CA) and classification consistency (CC) require assumptions about a parametric form of the test scores or about a parametric response model, such as item response theory (IRT). This article develops an approach to estimate CA…

This article presents a comparative judgment approach for holistically scored constructed response tasks. In this approach, the grader rank orders (rather than rate) the quality of a small set of responses. A prior automated evaluation of responses guides both set formation and scaling of rankings. Sets are formed to have similar prior scores and…

The primary objective of this study was to examine the effect of missing data on goodness of fit statistics in confirmatory factor analysis (CFA). For this aim, four missing data handling methods; listwise deletion, full information maximum likelihood, regression imputation and expectation maximization (EM) imputation were examined in terms of…

Purpose: Agreement across methods for identifying students as inadequate responders or as learning disabled is often poor. We report (1) an empirical examination of final status (postintervention benchmarks) and dual-discrepancy growth methods based on growth during the intervention and final status for assessing response to intervention and (2) a…

The 4-H youth development organization is a complex public--private partnership between the U.S. Department of Agriculture's National Institute of Food and Agriculture, the nation's Cooperative Extension system and National 4-H Council, a private, nonprofit partner. The current article is focused on a partnership approach to the…

Nonprofit organizations are under great pressure to use evaluations to show that their programs "work" and that they are "effective." However, empirical evidence indicates that nonprofits struggle to perform useful evaluations, especially when conducted under accountability pressures. An increasing body of evidence highlights…

This study proposes an advanced method to factor in the contributions of individual group members engaged in an integrated group project using peer assessment procedures. Conway et al. proposed the Individual Weight Factor (IWF) method for peer assessment which has been extensively developed over the years. However, most methods associated with…

Within the realm of school-based interventions, implementation integrity is important for practical, legal, and ethical purposes. Unfortunately, evidence suggests that proper monitoring of implementation integrity is often absent from both research and practice. School psychology practitioners and researchers have reported that a major barrier to…

The paper proposes a recommender system approach to cover online judge's domains. Online judges are e-learning tools that support the automatic evaluation of programming tasks done by individual users, and for this reason they are usually used for training students in programming contest and for supporting basic programming teachings. The…

This article provides a brief overview and comparison of three conditional growth percentile methods; student growth percentiles, percentile rank residuals, and a nonparametric matching method. These approaches seek to describe student growth in terms of the relative percentile ranking of a student in relationship to students that had the same…

Best practices of academic advising assessment involve identification of student learning outcomes, the development and use of multiple measures of student learning, and sound professional judgment to understand the information gathered and to improve student learning. However, the assessment results often come from minimal, narrow, and…

Despite our best efforts as evaluators, program implementation failures abound. A wide variety of valuable methodologies have been adopted to explain and evaluate the "why" of these failures. Yet, typically these methodologies have been employed concurrently (e.g., project monitoring) or to the post-hoc assessment of program activities.…

The essay considers two analogies that help to reveal the limitations of value-added modeling: the first, a comparison with batting averages, shows that the model's reliability is quite limited even though year-to-year correlation figures may seem impressive; the second, a comparison between medical malpractice and so-called educational…

As a language skill, writing has had, still has and will continue to have an important role in shaping the scientific structure of human life in that it is the medium through which scientific content is stored, retained, and transmitted. It has therefore been a major concern for writing teachers and researchers to find a reliable method for…

Teacher performance, as measured by evaluations, is increasingly being considered by states in employment decisions, including the awarding of tenure--and the removal of that status, according to a 50-state policy review by the Education Commission of the States (ECS). This report, one of three based on ECS' 2014 review, identifies the states…