The demand for diagnostic feedback has triggered extensive research on cognitive diagnostic models (CDMs), such as the deterministic input, noisy output "and" gate (DINA) model. This study explored two Q-matrix specifications with the DINA model in a statewide large-scale mathematics assessment. The first Q-matrix was developed based on…

The recent integration of visual capabilities into Large Language Models (LLMs) has the potential to play a pivotal role in science and technology education, where visual elements such as diagrams, charts, and tables are commonly used to improve the learning experience. This study investigates the performance of ChatGPT-4 Vision, OpenAI's most…

Testing organization needs large numbers of high-quality items due to the proliferation of alternative test administration methods and modern test designs. But the current demand for items far exceeds the supply. Test items, as they are currently written, evoke a process that is both time-consuming and expensive because each item is written,…

Rhode Island is committed to ensuring that all educators receive fair, accurate, and meaningful educator evaluations that provide information that can help all teachers improve and refine their practice. Currently, districts in Rhode Island may submit a district-designed model for approval that complies with the Educator Evaluation System…

According to the model of constructive alignment, learners adjust their learning strategies to the announced assessment (backwash effect). Hence, when teaching for understanding, the assessment method should be aligned with this teaching goal to ensure that learners engage in corresponding learning strategies. A quasi-experimental field study with…

Assessment engineering is a new way to design and implement scalable, sustainable and ideally lower-cost solutions to the complexities of designing and developing tests. It represents a merger of sorts between cognitive task modeling and engineering design principles--a merger that requires some new thinking about the nature of score scales, item…

Rhode Island is committed to ensuring that all educators receive fair, accurate, and meaningful educator evaluations that provide information that can help all teachers improve and refine their practice. This commitment is an outgrowth of the state's recognition of the influence teachers have on student growth and achievement. Currently, districts…

Measurement and evaluation of students' achievement are an important aspect to make sure that students really understand the course content and monitor students' achievement level. Performance is not only reflected from the numbers of high achievers of the students, but also on quality of the grade obtained; does the grade "A" truly…

Content balancing is one of the most important components in the computerized adaptive testing (CAT) especially in the K to 12 large scale tests that complex constraint structure is required to cover a broad spectrum of content. The purpose of this study is to compare the weighted penalty model (WPM) and the weighted deviation method (WDM) under…

This research addresses high school students' understandings of the nature of models, and their interaction with model-based software in three science domains, namely, biology, physics, and chemistry. Data from 736 high school students' understandings of models were collected using the Students' Understanding of Models in Science (SUMS) survey as…

There is much debate on the merits and pitfalls of standardized tests for college admission, with questions regarding the format (multiple-choice versus constructed response), cognitive complexity, and content of these assessments (achievement versus aptitude) at the forefront of the discussion. This study addressed these questions by…

In this article, I specify a conceptual framework for test validity research on content assessments taken by English language learners (ELLs) in U.S. schools in grades K-12. This framework is modeled after one previously delineated by Willingham et al. (1988), which was developed to guide research on students with disabilities. In this framework…

Describes the Texas test development process in detail, showing how each test development step is linked to the "Standards for Educational and Psychological Testing." The routine use of this process provides evidence of the content and curricular validity of the Texas Assessment of Academic Skills. (SLD)

Traditionally, the fixed-length linear paper-and-pencil (P&P) mode of administration has been the standard method of test delivery. With the advancement of technology, however, the popularity of administering tests using adaptive methods like computerized adaptive testing (CAT) and multistage testing (MST) has grown in the field of measurement…

Modified Parallel Analysis (MPA) is a heuristic method for assessing "approximate unidimensionality" of item pools. It compares the second eigenvalue of the observed correlation matrix with the corresponding eigenvalue extracted from a "parallel" matrix generated by a unidimensional and locally independent model. Revised…