Action research often necessitates the use of intact groups for the comparison of educational treatments or programs. This paper considers several analytical methods that might be used for such situations when pretest scores indicate that these intact groups differ significantly initially. The methods considered include gain score analysis of variance (ANOVA), analysis of covariance (ANCOVA) (using both raw scores and estimated true scores), value-added analysis, and within group dependent t-tests, all on a common set of real data from nonequivalent intact groups. Seemingly contradictory results were obtained for this data with gain score ANOVA and with ANCOVA. Comparable results should be expected to occur routinely with data from nonequivalent groups. In view of these results, it is recommended that statistical comparisons across nonequivalent groups be avoided. However, within group comparisons may aid somewhat in such evaluations of alternative educational programs. (Author/RC)