The 0.05 boundary within Null Hypothesis Statistical Testing (NHST) "has made a lot of people very angry and been widely regarded as a bad move" (to quote Douglas Adams). Here, we move past meta-scientific arguments and ask an empirical question: What is the psychological standing of the 0.05 boundary for statistical significance? We…

Using data from multiple informants has long been considered best practice in education. However, multiple informants often disagree on similar constructs, complicating decision-making. Polynomial regression and response-surface analysis (PRA) is often used to test the congruence effect between multiple informants on an outcome. However, PRA…

Sampling distributions are fundamental for statistical inference, yet their abstract nature poses challenges for students. This research investigates the development of high school students' conceptions of sampling distribution through informal significance tests with the aid of digital technology. The study focuses on how technological tools…

The early detection of item drift is an important issue for frequently administered testing programs because items are reused over time. Unfortunately, operational data tend to be very sparse and do not lend themselves to frequent monitoring analyses, particularly for on-demand testing. Building on existing residual analyses, the authors propose…

This brief paper introduces a new approach to assessing the trustworthiness of research comparisons when expressed numerically. The 'number needed to disturb' a research finding would be the number of counterfactual values that can be added to the smallest arm of any comparison before the difference or 'effect' size disappears, minus the number of…

A multiple testing method for examining factorial invariance for latent constructs evaluated by multiple indicators in distinct populations is outlined. The procedure is based on the false discovery rate concept and multiple individual restriction tests and resolves general limitations of a popular factorial invariance testing approach. The…

This study was aimed to find out from the teachers in Nigeria, their perception of the functionalism and their participation in the internal testing programme of secondary schools special reference to a state in southern Nigeria. A sample of 1,000 teachers was randomly stratified from the entire teachers' population of 6,000 in government-own…

There have been many discussions of how Type I errors should be controlled when many hypotheses are tested (e.g., all possible comparisons of means, correlations, proportions, the coefficients in hierarchical models, etc.). By and large, researchers have adopted familywise (FWER) control, though this practice certainly is not universal. Familywise…

In the context of existing "quantitative"/"qualitative" schisms, this paper briefly reminds readers of the current practice of testing for statistical significance in social science research. This practice is based on a widespread confusion between two conditional probabilities. A worked example and other elements of logical argument demonstrate…

In a recent study we found that testing as a final activity in a skills course increases the learning outcome compared to spending an equal amount of time practicing. Whether this testing effect measured as skills performance can be demonstrated on long-term basis is not known. The research question was: does testing as a final activity in a…

We discuss the identification, estimation, and testing of structural equation models that have causal indicators. We first provide 2 rules of identification that are particularly helpful in models with causal indicators--the 2C emitted paths rule and the exogenous X rule. We demonstrate how these rules can help us distinguish identified from…

Assessing the comparability of different groups is an issue facing many researchers and evaluators in a variety of settings. Commonly, null hypothesis significance testing (NHST) is incorrectly used to demonstrate comparability when a non-significant result is found. This is problematic because a failure to find a difference between groups is not…

A theory-driven approach to evaluation (TDE) emphasizes the development and empirical testing of conceptual models to understand the processes and mechanisms through which programs achieve their intended goals. However, most reported applications of TDE are limited to large-scale experimental/quasi-experimental program evaluation designs. Very few…

This study investigated the influence of host community on industrial relations practices and policies using Agbara community and Power Holding Company of Nigeria PLC as a case. The study adopted both the qualitative and quantitative methods. A total of 120 samples were drawn from the population using the simple random sampling technique in which…

Storytelling may be a powerful instructional approach for engaging learners and facilitating e-learning. However, relatively little is known about how to apply story within the context of systematic instructional design processes and claims for the effectiveness of storytelling in training and education have been primarily anecdotal and…