ERIC Number: ED643333
Record Type: Non-Journal
Publication Date: 2016
Pages: 24
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
NURTURES Evaluation Report: Five Year Summary: 2011-2016
Gale Mentzer; Peter Paprzcki
Grantee Submission
The NURTURES project, consisting of three phases, was implemented per its original timeline. Hiring, planning, piloting, and implementation were all completed in a timely manner with regards to the original timeline. Elements of the project were modified for improvement based upon reflection and formative evaluation findings. During the Planning Phase (Year 1), all staff were hired and plans for recruiting, the Summer Institute, academic year follow up, community events, research and data collection, and family packs were completed. The Planning Phase concluded having met its objectives. The groundwork for a successful Math Science Partnership (MSP) was laid for the Year 2 Training Phase. Hallmarks of the success of this Phase include: (1) Adherence to timeline; (2) Frequent meetings that built teamwork and group identification; (3) In-depth planning for the Pilot Phase; and (4) Adherence to staff and teacher qualifications when hiring as mapped out in the proposal. The Pilot Phase (Year 2) allowed the NURTURES team to test essential elements of the project prior to scaling up. During Year 2 the NURTURES partners met for a retreat that brought together the key players and familiarized them with the complementary learning paradigm as well as specific elements of the project. Piloting the Summer Institute provided an opportunity for the project leaders to try out various methodologies, schedules, and activities that were considered for inclusion in the actual Summer Institute that was to be offered the following year. Eight Master Teachers attended the Year 2 Summer Institute that included ideas for the actual Institute. Feedback from the Master Teachers was incorporated into the Summer Institute model. SciFUN Community Events were also piloted in Year two. Families were enthusiastic about both the events and the Family Packs. Process evaluation was an integral part of the Implementation Phase (Years 3-5). Findings verified fidelity of implementation of the various components of this comprehensive project and provided internal validity evidence. Corrections or modifications to implementation were made due to early detection of variance from implementation plans. For example, facilitators from SciFUN event hosts did not engage with families as designed during Year 3. Project leaders used process evaluation findings to better train facilitators with the result of close adherence to the event intentions. The effect of participating in NURTURES on teachers was cataloged using several established instruments. Measures of teacher attitudes about teaching science showed that teacher comfort teaching science increased after participation as did their tendency to equate effective science instruction with cognitive learning theory. On the other hand, teacher's found that teaching inquiry based science to be more challenging after participating in NURTURES. This is most likely due to their realization that to teach inquiry based science is more work, at least initially, than business as usual. Classroom observations of teachers teaching science were conducted pre and post Summer Institute participation. Observations were scored using the Electronic Quality of Inquiry Protocol created through the project "Inquiry in Motion" at Clemson University. This tool was based upon the Next Generation Science Standards. A chi square goodness of fit test compared pre- and post- frequencies and found a statistically significant change ([Alpha] < 0.001) suggesting that teachers were incorporating significantly more scientific inquiry into their practice than they did prior to participation in NURTURES. This supports teacher FOI of inquiry-based instructional strategies and verifies the obtainment of the improved science teaching objective. Improved family science was evaluated through observations of families engaged in the SciFUN community events. Summative data shows that the majority of family interaction was either balanced between the child and parent or the parent slightly dominated the interaction. This is to be expected when working with young children as they may need more guidance for some activities. Discourse between parent and child included question/short answer, why questions, and open-ended questions. Parent made frequent use of the SciFUN Event Guides to get ideas of questions to ask their children. Baseline equivalence was established based on kindergarten STAR Early Literacy data. Improved student learning was measured using a standardized test of early literacy, mathematics, and reading (STAR K-121). A hierarchical linear design was employed to examine the differences in achievement scores between students of NURTURES teachers and students of other teachers in the Toledo Public Schools. Participants consisted of 2899 students for the early literacy study, 2002 students for the mathematics study, and 1810 students for the reading study. Control students consisted of 2515 students for the early literacy study, 3028 students for the mathematics study, and 2448 students for the reading study, who had never had a NURTURES teacher within the same time frame. Results showed statistically significantly higher scores for NURTURES students on all three scales. The summative examination of measures of outcomes compared to intentions showed that NURTURES achieved all of its outcomes and its overall goal of improving student learning. While the original intention was to improve student science learning specifically, there were no standardized, rigorous measures of science achievement available for the young target population. As a result, mathematics, reading, and early childhood literacy were substituted under the assumption that reading and mathematics achievement in particular are highly correlated with science achievement. Findings from the five year evaluation indicate that NURTURES is a successful intervention for improving science teaching in the early childhood classroom as well as for increasing family science participation and the quality of that participation. In conclusion, with baseline equivalence established, learning from a NURTURES teacher was associated with student net gains on Renaissance STAR Assessments of 11.24 points to a student's Early Literacy spring score (effect size 0.09), 21.75 points to a student's Mathematics spring score (effect size 0.18) , and 47.85 points to a student's Reading spring score (effect size of 0.29). [This report was prepared by Acumen Research and Evaluation, LLC on behalf of the NURTURES program at the University of Toledo.]
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF)
Authoring Institution: N/A
Grant or Contract Numbers: 1102808
Author Affiliations: N/A

Peer reviewed
