The role of item response theory (IRT) in determining the validity of second language tests is examined in the case of one specific test, the listening subtest of the Occupational English Test (OET), used in Australia to measure the language skills of non-native English-speaking health professionals. First, the listening subtest is described. Then the debate over the appropriateness of IRT use in language testing research is discussed in some detail, with reference to a number of separate studies. Finally, a study of the use of IRT in validating the OET is reported. The study involved analysis, using the Partial Credit model, of data from 196 candidates taking the test in 1987. It investigated whether it is possible to construct a single measurement dimension of listening ability from data from the subtest's two parts, and if the answer is yes, whether the skills tested in the two parts are essentially the same. Results indicate that the test is indeed unidimensional, and support the use of IRT for such analysis. It is also concluded that the kinds of listening tasks in the two subtest parts represent significantly different tasks in terms of level of ability required to deal successfully with them. (MSE)