This study aims to report the development an instrument that is standardized (reviewed by validity, reliability, and difficulty index) to detect science misconception in an elementary school teacher. This study used a 4-D model; defining, designing, developing, and disseminating. First, it was prepared with 47 opened-ended questions, and then it…

This article presents an empirical evaluation of the validity and reliability of a peer-assessment activity to improve academic writing competences. Specifically, we explored a large group of psychology undergraduate students with different initial writing skills. Participants (n = 365) produced two different essays, which were evaluated by their…

The elicited imitation task (EIT), in which language learners listen to a series of spoken sentences and repeat each one verbatim, is a commonly used measure of language proficiency in second language acquisition research. The "TOEFL® Essentials"™ test includes an EIT as a holistic measure of speaking proficiency, referred to as the…

There is a dearth of research on hope in studies of second or foreign (L2) language learning. Therefore, the present research contributes conceptually to a deep understanding of hope for learning English as a foreign language and the ways it may be developed. To do so, an exploratory mixed-methods design was employed. Using in-depth interviews,…

Whenever multiple observers provide ratings, even of the same performance, inter-rater variation is prevalent. The resulting "idiosyncratic rater variance" is considered to be unusable error of measurement in psychometric models and is a threat to the defensibility of our assessments. Prior studies of inter-rater variation in clinical…

The purpose of this study was to assess the basic psychometric properties of a progress-monitoring tool designed to measure narrative discourse skills in school-age children with language impairments (LI). A sample of 109 children with LI between the ages of 5 years 7 months and 9 years 9 months completed the "Test of Narrative Language"…

In this study, generalizability theory was used to examine the extent to which (a) time-sampling methodology, (b) number of simultaneous behavior targets, and (c) individual raters influenced variance in ratings of academic engagement for an elementary-aged student. Ten graduate-student raters, with an average of 7.20 hr of previous training in…

Previous research shows inconsistency in clinician-assigned diagnoses of Autism Spectrum Disorder (ASD). We conducted an exploratory study that examined the concordance of diagnoses between a multidisciplinary assessment team and a range of independent clinicians throughout Australia. Nine video-taped Autism Diagnostic Observation Schedule (ADOS)…

Purpose. The purpose of this study was to identify technology-related strategies used by educational leaders to increase prosocial behavior in K-12 schools. Information and communication technology (ICT) is developing at a rapid rate and is becoming more ubiquitous among students. Discovering and understanding common technology-related strategies…

The work presented in this case study results from a study conducted in 2012-2014 examining a newly created teacher evaluation system to determine the inter-rater reliability of the classroom observation instrument. The teacher evaluation system was the result of a partnership between the school district and the university in the same city…

The purpose of this meta-analytic review was to investigate the relation between motivation and reading achievement among students in kindergarten through 12th grade. A comprehensive search of peer-reviewed published research resulted in 132 articles with 185 independent samples and 1,154 reported effect sizes (Pearson's r). Results of our…

Little research has been done on the effects of peer raters' quality characteristics on peer rating qualities. This study aims to address this gap and investigate the effects of key variables related to peer raters' qualities, including content knowledge, previous rating experience, training on rating tasks, and rating motivation. In an experiment…

In the present study, I examined the effects of rater characteristics, in particular, raters' familiarity with a foreign accent, on the assessment of second language (L2) pronunciation. Forty-three native English-speaking teachers were divided into three groups according to their reported types of familiarity with Korean accents: heritage,…

Millions of customer services representatives are assessed each year by subject matter experts (e.g., recruiters, team leaders) in Asian contact centres to ensure good spoken communication skills when serving customers on the phones. In other workplace contexts, language experts are employed to do this work but in Asian contact centres, a…

Recent education reforms including the "Next Generation Science Standards" emphasize integrating engineering into K-12 science instruction. Although prior studies have characterized engineering design (ED) in K-12 settings, no validated protocols attempt to characterize ED integration in secondary science classrooms. The present study…