ERIC Number: EJ1293869
Record Type: Journal
Publication Date: 2020
Pages: 32
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0161-4681
EISSN: N/A
Available Date: N/A
Chapter 7: Developing and Using Indicators for Continuous Improvement
Ming, Norma; Kennedy, Alec
Teachers College Record, v122 n14 2020
Background/Context: Amidst the complex and fast-paced demands in schools and classrooms, identifying what most deserves educators' attention can pose a challenge. Indicators help focus attention by highlighting key features that signal important outcomes or opportunities to take action or learn. Purpose: Informed by multiple literatures, we offer a conceptual framework that elaborates on the connection between an indicator's development and use. Specifically, the framework proposes that how to develop indicators depends on the answers to several questions about how they will be used: the intended use (why), timing (when), content (on what), and threshold (how much). Research Design: To explore this conceptual framework, we present two cases on the construction and use of indicators for continuous improvement in the San Francisco Unified School District. In the first case, we applied statistical process control (SPC) theory to create control charts that visualize district-level outliers on student achievement outcomes to examine potential causes of variation among schools. In the second case, we combined SPC with more basic statistical methods to develop indicators highlighting areas of potential promise across multiple measures and student subgroups within a school. In our exploration of the two case studies, we present the statistical methods applied to construct the indicator, an example of the visualizations created, and descriptions of how central office leaders interpreted and used the results. We document how the intended use informed the indicators' design, and the ramifications of those decisions for educators' subsequent use of the indicators for improvement. Conclusions and Recommendations: In learning from these cases, we highlight the importance of constructing indicators from the perspective of their intended use and linked with specific conditions and potential actions. This requires including more useful process data in the development of those indicators, as well as more follow-up analysis of the effectiveness of the actions taken based on those indicators, contrasted against alternative actions. We also underscore the importance of developing indicators that distinguish between the need for systemic versus targeted improvement efforts, based on SPC. We believe that combining approaches across disciplines and methodologies holds great promise for the development and use of indicators for continuous improvement in education.
Descriptors: Educational Indicators, Educational Improvement, Test Construction, Academic Achievement, Outcomes of Education, Data Use
Teachers College, Columbia University. P.O. Box 103, 525 West 120th Street, New York, NY 10027. Tel: 212-678-3774; Fax: 212-678-6619; e-mail: tcr@tc.edu; Web site: http://www.tcrecord.org
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: California (San Francisco)
Grant or Contract Numbers: N/A
Author Affiliations: N/A