This article presents the results of a pilot longitudinal study that attempted to develop a method to take subjective, qualitative observations about the English language speaking skills of Japanese English language learners and transform them into objective, quantitative measures. The following considerations must be addressed in the course of constructing this measure: What are the appropriate expectations of proficiency of a given student? Which skills should be mastered, to what level, and in what order? Are the instruments used valid measures, encompassing all the proper variables? Are the raters evaluating the students consistently? How can results be compared from speech to speech, class to class, year to year? Three types of tests are used: monologue speaking test (presentation); a dialogue speaking test (interview); and a multilogue speaking test (discussion and debate). (KFT)