Effectively evaluating the credibility and accuracy of multiple sources is critical for college readiness. We developed 24 source evaluation tasks spanning four predicted difficulty levels of a hypothesized learning progression (LP) and piloted these tasks to evaluate the utility of an LP-based approach to designing formative literacy assessments.…

The study examines reliability and validity evidence of observational systems for evaluating teacher effectiveness and fostering professional development conversations in schools. Specifically, this study compared the Framework for Teaching's (FFT) validity evidence using the traditional scoring approach with a new composite scoring approach that…

The State of Washington requires school districts to file court petitions on students with excessive unexcused absences. The "Washington Assessment of Risks and Needs of Students" (WARNS), a self-report screening instrument developed for use by high school and juvenile court personnel in such situations, purports to measure six facets of…

To meet recent accountability mandates, school districts are implementing assessment frameworks to document teachers' effectiveness. Observational assessments play a key role in this process, albeit without compelling evidence of their psychometric rigor. Using a sample of kindergarten teachers, we employed Generalizability theory to investigate…

Elementary school students often exhibit a variety of conceptions associated with algebraic thinking that their teachers fail to recognize or understand. It is crucial that elementary school teachers possess knowledge of the variety of student conceptions and also have abilities to address varying states of conceptions. Otherwise, students who are…

We describe the conceptualization, design, development, validation, and testing of a summative instrument that measures high school students' ability to analyze and evaluate data, construct scientific explanations, and formulate scientific arguments in biology and chemistry disciplinary contexts. Data from 1,405 students were analyzed to evaluate…

Tablets can be used to facilitate systematic testing of academic skills. Yet, when using validated paper tests on tablet, comparability between the mediums must be established. Comparability between a tablet and a paper version of a basic math skills test (HRT: Heidelberger Rechen Test 1-4) was investigated. Five samples with second and third…

Writing from source text is critical for developing college-and-career readiness because it is required in advanced academic environments and many vocations. Scenario-based assessment (SBA) represents one approach to measuring this ability. In such assessment, the scenario presents an issue that the student is to read and write about. Before…

Undergraduate grade point average (GPA) is a commonly employed measure in educational research, serving as a criterion or as a predictor depending on the research question. Over the decades, researchers have used a variety of reliability coefficients to estimate the reliability of undergraduate GPA, which suggests that there has been no consensus…

In most U.S. schools, teachers are evaluated using observation of teaching practice (OTP). This study investigates rater effects on OTP ratings among 421 principals in an authentic teacher evaluation system. Many-facet Rasch analysis (MFR) using a block of shared ratings revealed that principals generally (a) differentiated between more and less…

This study investigates the relationships among factor correlations, inter-item correlations, and the reliability estimates of subscores, providing a guideline with respect to psychometric properties of useful subscores. In addition, it compares subscore estimation methods with respect to reliability and distinctness. The subscore estimation…

Teacher assessment literacy has become a central priority across many educational systems in North America and elsewhere in response to growing accountability demands. Although many scholars have aimed to measure teacher assessment literacy, recent research has identified that current assessment literacy instruments do not fully reflect current…

Students' opinions continue to be a significant factor in the evaluation of teaching in higher education institutions. The purpose of this study was to psychometrically assess short students evaluation of teaching (SET) forms using the UAE University form as a model. The study evaluated the form validity, reliability, the overall question, and…

Assessment systems that track children's progress across time need to be sensitive to the variegated nature of development. Although instruments are commonly designed to assess behaviors within a specific age range, some children advance slower or faster than others and, as a result, often show behaviors from a younger or older age group. This…

To date, assessment validity research on non-native English speaking students in the United States has focused exclusively on those who are presently English language learners (ELLs). However, little, if any, research has been conducted on two other sizable groups of language minority students: (a) bilingual or multilingual students who were…