In this article, Regine Haardörfer outlines five general steps taken by good data analysts and how they need to be theory-driven data-informed. She uses these to discuss some issues and propose approaches to promote better data analysis and reporting. The proposed steps to rigorous data analysis are to: (1) create an a priori data analysis plan;…

Rater effects, or raters' tendencies to assign ratings to performances that are different from the ratings that the performances warranted, are well documented in rater-mediated assessments across a variety of disciplines. In many real-data studies of rater effects, researchers have reported that raters exhibit more than one effect, such as a…

We examine common and unique metadata requirements and their levels of description, determined by the data deposit forms of 20 repositories in three disciplines--archaeology, quantitative social science, and zoology. The results reveal that requirements relating to Creator, Description, Contributor, Date, Relation, and Location are common, whereas…

The aim of the present study was to investigate the properties of paper-and-pencil data collection instruments developed to measure Computational Thinking (CT) based on several variables. Thus, keywords were identified and used in searches conducted in various databases. The outcomes of the search were analyzed based on the inclusion/exclusion…

Modern environmental science research increasingly requires computational ability to apply statistics to environmental science problems, but graduate students in these scientific fields typically lack these integral skills. Many scientific graduate degree programs expect students to acquire these computational skills in an applied statistics…

Education and education research are experiencing increased digitization and datafication, partly thanks to the rise in popularity of massively open online courses (MOOCs). The infrastructures that collect, store and analyse the resulting big data have received critical scrutiny from sociological, epistemological, ethical and analytical…

The Java software jBar consists of a graphical user interface that allows the user to customize and assemble an included script for R. The scripted R pipeline calculates means and standard errors/deviations for replicates of numerical bivariate data and generates presentations in the form of bar graphs. A two-sided Student's t test is carried out…

Dual-frame, random-digit-dialing (DFRDD) surveys, using a frame of landline telephone numbers and a frame of cell phone numbers, have become the norm for telephone surveys. Two alternative frame constructions sample exclusively from a cell phone frame, referred to as the 'single-frame cell,' and sample from the listed landline frame and the cell…

Data mining methods have drawn considerable attention across diverse scientific fields. However, few applications could be found in the areas of psychological and educational measurement, and particularly pertinent to this article, in test security research. In this study, various data mining methods for detecting cheating behaviors on large-scale…

Collecting data for cross-cultural research is time consuming and expensive. We report on techniques for efficient and effective cross-cultural data collection, drawn from 18 cross-cultural studies we conducted in 22 countries. We discuss research design, site selection and collaboration building, ethical review, cross-site protocol design, data…

Peer assessment has proven to be a useful strategy for increasing the timeliness and quantity of formative feedback, as well as for promoting metacognitive thinking among students. Previous research has determined that reviews that contain suggestions can motivate students to revise and improve their work. This paper describes a method for…

We are still designing educational experiences for the "average" student, and have room to improve. Learning analytics provides a way forward. This commentary describes how learning analytics-based applications are well positioned to meaningfully personalize the learning experience in diverse ways. In so doing, learning analytics has the…

This study explores how involving education leaders in the process of research can enhance the practical value of research for them. This paper describes: (1) a design process used for engaging US state science leaders in participatory research; and (2) shifts in understanding that state leaders reported based on their participation. The authors…

Effective collaboration between student peers is not spontaneous. A system that can measure collaboration in real-time may be useful, as it could alert an instructor to pairs that need help in collaborating effectively. We tested whether superficial measures of speech and user interface actions would suffice for measuring collaboration. Pairs of…

Administrators of distance learning courses and programs are faced with various data needs related to a range of issues; from enrollment tracking to faculty development to cost and revenue analysis. This paper will explore a proposed common data set for online program administrators and strategies for implementation. The framework offered for…