This article reviews the evidence on the extent to which experts' perceptions of item difficulties, captured using comparative judgement, can predict empirical item difficulties. This evidence is drawn from existing published studies on this topic and also from statistical analysis of data held by Cambridge Assessment. Having reviewed the…

Research objective is comparing the objective methods often used in literature for determination of differential item functioning (DIF) and the subjective method based on the opinions of the experts which are not used so often in literature. Mantel-Haenszel (MH), Logistic Regression (LR) and SIBTEST are chosen as objective methods. While the data…

While there have been concerted efforts to reform undergraduate biology toward teaching students to organize their conceptual knowledge like experts, there are few tools that attempt to measure this. We previously developed the Biology Card Sorting Task (BCST), designed to probe how individuals organize their conceptual biological knowledge.…

We explore how crowdworkers can be trained to tackle complex crowdsourcing tasks. We are particularly interested in training novice workers to perform well on solving tasks in situations where the space of strategies is large and workers need to discover and try different strategies to be successful. In a first experiment, we perform a comparison…

Calls for undergraduate biology reform share similar goals: to produce people who can organize, use, connect, and communicate about biological knowledge. Achieving these goals requires students to gain disciplinary expertise. Experts organize, access, and apply disciplinary knowledge differently than novices, and expertise is measurable. By asking…

The study addresses professional vision, including the abilities of selective attention and knowledge-based reasoning. This article focuses on the latter ability. Groups with different sport-specific and pedagogical expertise (n = 60) were compared according to their observation and interpretation of sport activities in a four-field design. The…

Background: Teachers' important diagnostic abilities include noticing and interpreting students' behaviors and learning processes. By focusing on noticing, I refer to the theoretical framework of professional vision. Professional vision includes the ability to notice what is occurring in complex classroom situations (selective attention) and the…

Visual expertise has been explored in numerous professions, but research on teachers' vision remains limited. Teachers' visual expertise is an important professional skill, particularly the ability to simultaneously perceive and interpret classroom situations for effective classroom management. This skill is complex and relies on an awareness of…

Comparisons of literary experts and novices indicate that experts engage in interpretive processes to "get the point" during their reading of literary texts but novices do not. In two studies the reading and interpretive processes of literary novices (undergraduates with no formal training in literature study) were elicited through…

This study aimed to investigate and compare in-service and pre-service teachers' self confidence on technological pedagogical content knowledge (TPACK) in relation to their teaching experience, expertise, technology usage, and gender. To achieve this goal, survey method was conducted as part of a quantitative method design. Participants of the…

Education reform is increasingly portrayed as a means to improve a nation's global competitiveness as measured by its performance in international league tables of pupil achievement. This has created a demand for comparative research which identifies "what works" in high-performing school systems. A diverse array of consultancies, thinks…

The authors investigated the differences in using problem decomposition and problem recomposition between dyads of engineering experts, engineering seniors, and engineering freshmen. Participants worked in dyads to complete an engineering design challenge within 1 hour. The entire design process was video and audio recorded. After the design…

Classroom observation research and research on teacher expertise are similar in their reliance on observational data with high-inference procedure to assess the quality of instruction. Expertise research usually uses low-inference measures like eye tracking to identify qualitative difference between expert and novice behaviors and cognition. In…

Two studies were conducted to investigate effects of domain knowledge on metacognitive monitoring across the life span in materials of different complexity. Participants from 4 age groups (3rd-grade children, adolescents, younger and older adults) were compared using an expert-novice paradigm. In Study 1, soccer experts' and novices'…

As the United States experiences greater income inequality, more and more students experience an early science achievement gap. This study tested several competing theoretical models of early science achievement with a longitudinal sample of 14,624 children who were followed from kindergarten entry to the end of 1st grade. To understand why and…