Background: Most of the studies in academic journals use p values to represent statistical significance. However, this is not a good indicator of practical significance. Although confidence intervals provide information about the precision of point estimation, they are, unfortunately, rarely used. The infrequent use of confidence intervals might…

We surveyed researchers in the health and exercise sciences to explore different areas and magnitudes of bias in researchers' decision making. Participants were presented with scenarios (testing a central hypothesis with p = 0.06 or p = 0.04) in a random order and surveyed about what they would do in each scenario. Participants showed significant…

The empirical study we present here deals with a pedagogical issue that has not been thoroughly explored up until now in our field. Previous empirical studies in other sectors have identified the opinions of researchers about this topic, showing that completely unacceptable interpretations have been made of significance tests and other statistical…

There have been many discussions of how Type I errors should be controlled when many hypotheses are tested (e.g., all possible comparisons of means, correlations, proportions, the coefficients in hierarchical models, etc.). By and large, researchers have adopted familywise (FWER) control, though this practice certainly is not universal. Familywise…

This paper includes a nontechnical description of methods for calculating effect sizes in intellectual and developmental disability studies. Different hypothetical studies are used to illustrate how null hypothesis significance testing (NHST) and effect size findings can result in quite different outcomes and therefore conflicting results. Whereas…

Agricultural education teachers' attitudes about the perceptions of diversity inclusion in their programs are variables that may have a strong influence on the number of students that enroll in agricultural education. This study examines the effect of teachers' demographic and personal characteristics on their perceptions of the benefits of…

This study reports the effectiveness of a year-long, small-group, tertiary (Tier 3) intervention that examined 2 empirically derived but conceptually different treatments and a comparison condition. The researchers had randomly assigned all students to treatment or comparison conditions. The participants were seventh- and eighth-grade students…

Assessing the comparability of different groups is an issue facing many researchers and evaluators in a variety of settings. Commonly, null hypothesis significance testing (NHST) is incorrectly used to demonstrate comparability when a non-significant result is found. This is problematic because a failure to find a difference between groups is not…

This methodological brief introduces the readers to the propensity score matching method, which can be used for enhancing the validity of causal inferences in research situations involving nonexperimental design or observational research, or in situations where the benefits of an experimental design are not fully realized because of reasons beyond…

The purpose of this study was to determine if professional development using school contextual data would provide deeper understanding and attention to the important issue of student bullying. This was primarily a quantitative study. Phase 1 involved using data from the pre-existing student and teacher needs assessments to create the school…

Background: A small but growing body of research evidence suggests that place-based police interventions generate significant crime control gains. While place-based policing strategies have been adopted by a majority of U.S. police departments, very few agencies make a priori commitments to rigorous evaluations. Objective: Recent methodological…