Research on the evaluation of the professional knowledge of mathematics teachers (comprising for example mathematical content knowledge, mathematics pedagogical content knowledge and general pedagogical knowledge) has become prominent in the last decade; however, the development of video-based assessment approaches is a more recent topic. This paper follows the call for more situated and performance-related ways to assess teacher competence. We discuss the theoretical and methodological challenges connected to the development of such instruments and exemplify these by an instrument developed within the follow-up study of the international "Teacher Education and Development Study in Mathematics (TEDS-M)", called TEDS-FU. Drawing on the novice-expert framework from cognitive psychology allows analysing the structure and development of mathematics teachers' professional competence. More recent concepts on teacher noticing of classroom situations and students' activities are incorporated into this video-based evaluation instrument, which is described in detail in this paper, by assessing perceptual, interpretative and decision-making skills. Reliability and validity concerns remain an issue of such assessments for which solutions are proposed. Overall, the paper shows that a more comprehensive evaluation of teachers' competence comprising cognitive-affective and situated facets is possible and has been achieved.