Programme evaluation is essential for quality assurance in education. In this paper, we describe our evaluation strategy for the first year of a new medical programme. Although we used multiple methods in the evaluation, the use of the focus group method was core. This paper reports our experiences of focus groups for this purpose. We describe the implementation of the strategy including the development of topic guides, interviewer training, respondent recruitment, record keeping, template production, transcription, analysis and reporting of results. Using interviewer notes and individual interviews with focus group interviewers and participants, we evaluated focus groups as a method in educational programme evaluation. Focus groups are the most important evaluation method in helping us measure the extent to which we have met the goals of the first year of our programme. Benefits include generating detailed information in a relatively short time. Focus groups are largely valued by participants and appear to be contributing to the development of an "evaluation culture" in our new medical school. They are also building our capacity and capability in qualitative research methods. Challenges include the human resource intensive nature of scheduling, transcribing, analysing and report writing. (Contains 2 tables.)