The empirical validity of generalizability theory was investigated by applying two three-facet designs to data obtained in 1988 from administration of the Scientific Thinking and Research Skill Test (STRST). The decision validity of the STRST was also examined. Subjects were 125 fifth-grade and 125 sixth-grade students who were administered the STRST in a test and retest. The STRST contains 13 items on scientific skills domain and 13 items within the logical thinking domain. Applying generalizability theory to the data resulted in the observed score variance being partitioned in two ways, identifying different sources of error and their relative magnitudes. Test domain was one of the large variance components of the total score variance, a finding suggesting that test scores should be interpreted from the separate domain scores rather than from the total score. The interaction effect for persons and items within test domains suggests the possibility of biased item sampling. Findings from the generalizability study further imply that the number of test items should be increased to more than 20 in each domain to attain satisfactory generalization of the STRST into its universe. Recommendations for improvement of the STRST are presented. Nine tables present study data, and two figures illustrate the research designs. (SLD)