A model and method for developing a valid and reliable paper-and-pencil test for evaluating the technical knowledge of baseball coaches in the National Coaching Certification Program (NCCP) in Canada are discussed. Development of a general theoretical and operational model for the evaluation of NCCP coaches in all sports are also studied. The operational model for test construction includes: (1) selecting a conceptual framework and conducting content analysis with an expert panel; (2) determining uses and details of test administration; (3) identifying the target population; (4) developing items and indicators, including rating scale test specification tables; (5) modifying selected items and test assembly, dimensions, scoring keys, and test procedures; and (6) testing and qualifying to determine reliability and validity. Thirty test items and constructs and 13 demographic and coaching-related variables were selected for the Baseball Coaching Technical Questionnaire (BCTQ). Content validity was assessed via content analysis, item specification, and use of an expert panel. Factor analysis determined the construct validity, and discriminant analysis assessed the criterion-related validity, using 134 certified and non-certified coaches. Discriminant analysis showed that 64.93% of the coaches were correctly classified by the test, with the highest criterion referencing validity for non-certified coaches (72% correctly classified) and Level-II coaches (64.3% correctly classified). Although the BCTQ showed a more than adequate degree of reliability and validity, it should be administered to a larger sample of coaches from other Canadian communities to confirm its national reliability. Five flowcharts and five data tables are provided. (SLD)