This article questions whether popular approaches to the assessment of technology integration with classroom instruction are valid. The article explores Messick's (1993) conceptualisation of consequential validity, in an attempt to understand what validity must be evident when the integration of technology with classroom instruction is assessed. This article also compares and contrasts assessment tools and systems that are currently used, searching for evidence of valid assessment. Individually these tools sample a very limited collection of teacher knowledge and skill. Against a backdrop of rapidly expanding expectations and rapid change, the tests are probably unable to capture what teachers need to do, and more importantly what they actually are doing. The article concludes with recommendations to strengthen the validity of assessments of technology integration. (Contains 2 tables and 1 figure.)