In 1984 the State of New York mandated participation in the Comprehensive School Improvement Program (CSIP) for elementary schools that ranked in the bottom 9 percent on statewide student competency exams. To discern the overall effectiveness of the program and identify why it appeared to succeed in some schools and fail in others, the changes in 20 elementary schools required to implement CSIP were compared with those in 23 schools that developed their own improvement programs. Twenty-six teachers, all members of the participating schools' CSIP teams, and 6 building principals were interviewed and asked to complete a written survey instrument. CSIP, as a statewide policy and program, seemed to have minimal positive effect. The research revealed five qualitative factors that seemed to distinguish between improving and nonimproving schools: (1) reduction of teacher isolation; (2) consensual decision-making; (3) broad-based involvement; (4) building-wide intervention strategies; and (5) formal monitoring programs. Implications of the findings, such as the idea that real school improvement cannot be legislated, are discussed. Recommendations for further research are made. (MLF)