We revisit the merits of the retrospective pretest-posttest (RPP) design for repeated-measures research. The underutilized RPP method asks respondents to rate survey items twice during the same posttest measurement occasion from two specific frames of reference: "now" and "then." Individuals first report their current attitudes…

Extension educators are increasingly using online resources in their program design and delivery. Usability testing is essential for ensuring that these resources are relevant and useful to learners. On the basis of our experiences with iteratively developing products using a testing service called UserTesting, we promote the use of fee-based…

We explore the conditions under which short, comparative interrupted time-series (CITS) designs represent valid alternatives to randomized experiments in educational evaluations. To do so, we conduct three within-study comparisons, each of which uses a unique data set to test the validity of the CITS design by comparing its causal estimates to…

This paper aims to extend understanding of the business and societal impact of academic research. From a business school perspective, it has taken stock of the role of academic research and relevance in business and society. The proposed conceptual framework highlights the forces influencing the pursuit of academic rigour and relevance in…

Finding ways to improve student success in calculus is a critically important step on the path to supporting students who are pursuing degrees in STEM fields. Far too many students fail calculus 1 and are pushed to drop their majors in technical fields. One way of addressing this issue is by following a program that was pioneered at University of…

This article is based on the author's experiences as a second-year teacher at a small private school teaching all sections of Geometry and Algebra 2. Working with the same students in both courses, the author was aware of each one's levels of retention and execution of Algebra 1 concepts before the beginning of the year. The author often noticed…

The purpose of this study was to develop a scoring rubric to assess students' manipulation skills and identify students' learning difficulties in conducting organic chemistry experiments. In constructing the scoring rubric, we first analyzed the skills needed in the experiment, then divided the skills into subskills, and finally…

The current study was designed to extend the current literature to study scale drift in CAT as part of improving quality control and calibration process for ACCUPLACER, a battery of large-scale adaptive placement tests. The study aims to evaluate item parameter drift using empirical data that span four years from the ACCUPLACER Arithmetic…

Automated scoring models for the "e-rater"® scoring engine were built and evaluated for the "GRE"® argument and issue-writing tasks. Prompt-specific, generic, and generic with prompt-specific intercept scoring models were built and evaluation statistics such as weighted kappas, Pearson correlations, standardized difference in…

This paper addresses the use of computer-based testing in distance education, based on the experience of Universitas Terbuka (UT), Indonesia. Computer-based testing has been developed at UT for reasons of meeting the specific needs of distance students as the following: (1) students' inability to sit for the scheduled test; (2) conflicting test…

The purpose of the present study was to examine a common practice in some areas of program evaluation, the retrospective pretest, and to present recommendations regarding its use. The authors review literature to emphasize first, that bias is likely in both prospective and retrospective ratings, and second, that under some circumstances,…

Describes procedure to measure a new course's ability to meet standards in three areas: student acceptance, student gains, and student comprehension. It has been used over three years to test the acceptability of 31 courses which contain more than 200 videotape presentations, and thousands of pages of print materials. (Author/JEG)