Thinking skills play a critical role in determining language performance. Recent advancement in cognitive diagnostic modelling (CDM) provides a powerful tool for obtaining fine-grained information regarding these thinking skills during reading. Studies are scant, however, exploring the relations between thinking skills and language performance,…

Increasing children's access to media has attached greater significance to media literacy education, the content and methods of which have changed with media development. This study aimed to investigate the effects of the Community of Inquiry (CoI) method on media literacy competencies in elementary students. To this end, 95 female sixth-grade…

Research into metacognition has found it to facilitate self-regulation and correlate to learners' L2 writing level. Following Lee & Mak's (2018) framework of Metacognitive Instruction (MI) for L2 writing classrooms, this study applies Dynamic Assessment (DA) to writing MI (MI-DA) in a rural middle school EFL class in China. A one-semester…

There is growing interest in using comparative judgement to assess student work as an alternative to traditional marking. Comparative judgement requires no rubrics and is instead grounded in experts making pairwise judgements about the relative 'quality' of students' work according to a high level criterion. The resulting decision data are fitted…

Analytic rubrics are promoted as important tools to assess learner performance and to improve learning outcomes. Rubrics, however, are not appropriate for every classroom assessment, particularly given the time and effort required to construct well-designed rubrics. In classroom assessment, instructors must balance the beneficial consequences of…

Classroom assessment and large-scale assessment have, for the most part, existed in mutual isolation. Some experts have felt this is for the best and others have been concerned that the schism limits the potential contribution of both forms of assessment. Margaret Heritage has long been a champion of best practices in classroom assessment. Neal…

The present research paper introduces a translation evaluation method called Calibrated Parsing Items Evaluation (CPIE hereafter). This evaluation method maximizes translators' performance through identifying the parsing items with an optimal p-docimology and d-index (item discrimination). This method checks all the possible parses (annotations)…

The present study investigated the effects of text-based and audio-based dynamic glosses on L2 vocabulary learning within a sociocultural approach. Dynamic glosses were operationalised as a set of incrementally ordered prompts, provided during text-based and audio-based interactions that guided the participants to identify the meaning of unknown…

This study analyzes the effect of spatial competition on public schools' efficiency in Chile, an extreme case of market-oriented reforms in the educational sector. To address this issue, we use a measure of competition that captures three major characteristics of market competition in a spatial context: the number of competitors, based on distance…

In Dynamic Assessment (DA), the observation that individuals respond differently to support, or mediation, is important for diagnoses of development. The concept of learning potential refers to openness to mediation, i.e., the extent of change to performance when mediation is available, which may suggest learners will need less overall instruction…

How can we test whether state-of-the-art generative models, such as Blender and GPT-3, are good AI teachers, capable of replying to a student in an educational dialogue? Designing an AI teacher test is challenging: although evaluation methods are much-needed, there is no off-the-shelf solution to measuring pedagogical ability. This paper reports…

We describe a combination of methods for assessing the effectiveness of complex interventions, especially where substantial heterogeneity with regard to the population, intervention, comparison, outcomes, and study design of interest is expected. We applied these methods in a recent systematic review of the effectiveness of reinforced home-based…

Given the widespread use of non-experimental (NE) methods for assessing program impacts, there is a strong need to know whether NE approaches yield causally valid results in field settings. In within-study comparison (WSC) designs, the researcher compares treatment effects from an NE with those obtained from a randomized experiment that shares the…

Pressure on evaluators has been investigated recently by surveys in the USA, the UK, Germany, and Switzerland. This study compares the results of those studies regarding pressure on evaluators in different countries. The findings suggest that independence of evaluations does not exist for many respondents. Moreover, the person who commissioned the…

This article provides an overview of the Hofstee standard-setting method and illustrates several situations where the Hofstee method will produce undefined cut scores. The situations where the cut scores will be undefined involve cases where the line segment derived from the Hofstee ratings does not intersect the score distribution curve based on…