In the present climate of public accountability, there is increasing demand to show "what works" and what return is gained for the public from investments to improve communities. This increasing demand for accountability is being met with growing confidence in the field of philanthropy during the past 10 years that the impact or effectiveness of community initiatives can be measured. A review of recent evaluation reports and literature reveals three different meanings of effectiveness in use: increased understanding, accountability, and demonstrated causal linkages. Drawing on a general analysis of the concepts of accountability, effectiveness, and causality, each meaning of effectiveness is examined in light of its implications for evaluation design. The article closes with a guiding schema for organizations planning evaluations to help determine which kind of effectiveness is important in light of organizational values and aims. (Contains 4 notes and 1 table.)