The replication crisis in social and behavioral sciences has raised concerns about the reliability and validity of empirical studies. While research in the literature has explored contributing factors to this crisis, the issues related to analytical tools have received less attention. This study focuses on a widely used analytical tool -…

Structural equation modeling (SEM) has been deemed as a proper method when variables contain measurement errors. In contrast, path analysis with composite-scores is preferred for prediction and diagnosis of individuals. While path analysis with composite-scores has been criticized for yielding biased parameter estimates, recent literature pointed…

Likert-type item is the most popular response format for collecting data in social, educational, and psychological studies through scales or questionnaires. However, there is no consensus on whether parametric or non-parametric tests should be preferred when analyzing Likert-type data. This study examined the statistical power of parametric and…

Multivariate analysis of variance (MANOVA) is a statistical method used to examine group differences on multiple outcomes. This article reports results of a review of MANOVA in gifted education journals between 2011 and 2017 (N = 56). Findings suggest a number of conceptual and procedural misunderstandings about the nature of MANOVA and its…

Background: Most of the studies in academic journals use p values to represent statistical significance. However, this is not a good indicator of practical significance. Although confidence intervals provide information about the precision of point estimation, they are, unfortunately, rarely used. The infrequent use of confidence intervals might…

The educational literature, the popular press, and educated laypeople have all echoed a conclusion from the book "Academically Adrift" by Richard Arum and Josipa Roksa (which has now become received wisdom), namely, that 45% of college students showed no significant gains in critical thinking skills. Similar results were reported by…

Multilevel modeling has grown in use over the years as a way to deal with the nonindependent nature of observations found in clustered data. However, other alternatives to multilevel modeling are available that can account for observations nested within clusters, including the use of Taylor series linearization for variance estimation, the design…

Since 1992, the UK Government has published so-called "school league tables" summarising the average General Certificate of Secondary Education (GCSE) "attainment" and "progress" made by pupils in each state-funded secondary school in England. While the headline measure of school attainment has remained the percentage…

This paper introduces a dataset and associated analysis of the scores of National Football League (NFL) games over the 2012, 2013, and first five weeks of the 2014 season. In the face of current media attention to "lopsided" scores in Thursday night games in the early part of the 2014 season, t-test results indicate no statistically…

Business analytics courses, such as marketing research, data mining, forecasting, and advanced financial modeling, have substantial predictive modeling components. The predictive modeling in these courses requires students to estimate and test many linear regressions. As a result, false positive variable selection ("type I errors") is…

In this study, I investigated 2 procedures intended to create test-taker groups of equal ability by poststratifying on a composite variable created from demographic information. In one procedure, the stratifying variable was the composite variable that best predicted the test score. In the other procedure, the stratifying variable was the…

Person-fit assessment may help the researcher to obtain additional information regarding the answering behavior of persons. Although several researchers examined person fit, there is a lack of research on person-fit assessment for mixed-format tests. In this article, the lz statistic and the ?2 statistic, both of which have been used for tests…

In order to circumvent the influence of correlation in paired-samples and repeated measures experimental designs, researchers typically perform a one-sample Student "t" test on difference scores. That procedure entails some loss of power, because it employs N - 1 degrees of freedom instead of the 2N - 2 degrees of freedom of the…

I discuss three common practices that obfuscate or invalidate the statistical analysis of randomized controlled interventions in applied linguistics. These are (a) checking whether randomization produced groups that are balanced on a number of possibly relevant covariates, (b) using repeated measures ANOVA to analyze pretest-posttest designs, and…

The paper shows how the methodology for a pedagogical experiment can be improved through including the pre-research stage. If the experiment has the form of a test procedure, an improvement of methodology can be achieved using for example the methods of statistical and didactic analysis of tests which are traditionally used in other areas, i.e.…