Empirical studies in the economics of education, the measurement of skill gaps, the impacts of interventions on skill formation, and the value-added literature rely on psychometrically validated test scores. Test scores are taken as measures of an invariant scale of human capital compared over time and people. We examine if conventional skill measures are comparable on mastery of specific task knowledge. An unusually rich dataset from an early childhood intervention is used to test the assumption of scale invariance. We reject the scale invariance hypothesis for multiple skills and cast doubt on the uncritical use of test scores in research. [Additional support for this report was provided by the Institute for New Economic Thinking.]