This paper describes the conceptualization and development of an evaluation system that can be used to monitor and evaluate online instructional efforts. The evaluation system addresses concerns of both program administrators and course instructors. Computer technology is used to provide partial automation to reduce respondent burden and to efficiently use evaluation resources. The activities of the project reflect major efforts to obtain information, design components of the evaluation system, and test and revise the evaluation system. The specific activities described in the paper are to: (1) develop a conceptual model for online evaluation; (2) identify specific vital signs for an online program and determine appropriate measures; (3) automate the collection and analysis of evaluation data using knowledge engineering; (4) create an electronic performance support system to assist in program evaluation; and (5) conduct a field test of the evaluation system. (Contains 31 references.) (MES)