NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: ED659345
Record Type: Non-Journal
Publication Date: 2023-Sep-28
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Encouraging Families to Visit a Literacy Website: A Randomized Study of the Impact of Email and Text Message Communications
Candace Hester; Andrew Jaciw; Anja Kurki; Jenna Zacamy; Ashley Pierson; Garrett Lai; Amy Feygin
Society for Research on Educational Effectiveness
Background: In 2019, Arkansas had the nation's fourth-lowest reading scores (National Assessment of Educational Progress, 2019). Grounded in evidence that parents and guardians can play a pivotal role in developing their children's reading proficiency (Carnegie Council on Advancing Adolescent Literacy, 2010), the Arkansas Department of Education (ADE) created the Reading Initiative for Student Excellence (R.I.S.E.) website to provide family-based literacy activities and promote reading proficiency. Brief email and text messages (known as "nudges") sent to parents/guardians can encourage behavior leading to positive outcomes for students (York et al., 2019; Heppen et al., 2020; Kraft & Dougherty, 2013). However, not all nudges are effective (e.g., Bettinger et al., 2022) and features of nudges such as personalization matter (Head et al., 2013). This study explores the impact of different nudge features on parent/guardian visits to the R.I.S.E. website. Research Questions: To study the effectiveness of using emails and text messages to increase parent/guardian visits to the R.I.S.E. website, ADE partnered with the authors to address four research questions about households with children in kindergarten-grade 6 in Arkansas public schools: (1) What percentage of households had a working email address or cell phone number in November 2021?; (2) What percentage of households opened an email or were delivered a text message in any of the three communication rounds?; (3) Did nudge features impact the average percentage of households visiting the website?; and (4) Did time spent on the website vary by nudge features? Setting: This study occurred among families with children in kindergarten-grade 6 students in Arkansas public schools. Population: In November 2021, ADE worked with an email/text message vendor to send two test messages to all eligible households (180,531 households in 700 public schools) to establish whether households had a working email address or cell phone number. At this stage, 11,784 households opted out of further contact. The remaining 168,747 households in 700 schools were randomly assigned to study conditions (the analytic sample). Demographic data at the household level was not available. Intervention: ADE developed email and text messages with literacy information and a school-specific link to the R.I.S.E. website. In January 2022, households in the analytic sample were sent three nudges, whose features varied by random assignment (see appendix). Communications were sent in English or Spanish, depending on household preference. Findings include results for both languages. Research Design: The study team randomly assigned 700 schools to one of eight conditions, which varied three nudge features: communication mode (email only or email and text message), graphic (no graphic or with graphic), and sender (generic or known sender [school principal]; figure 1). This allowed for estimating the impact of different nudge features. Data/Analysis: This study used ADE data on the number of households and school characteristics; email/text message data on working email, working cell phone, and click rates; and Google Analytics data on time spent on website. The authors calculated percentages of households with working contact information and who opened emails/were delivered text messages to answer the first two research questions. For research question 3, the authors estimated the impact of being assigned to receive nudges on school average click rates (an intent-to-treat analysis) using regression models controlling for school characteristics (see appendix. For research question 4, the authors calculated average time spent on the website. Findings: Nearly all households had a working email address or cell phone number (97.8%). More households had a working cell phone number (89.9%) than email address (73.1%), indicating households are more likely to receive communications through text messages (figure 2). Households with children in rural schools or schools with higher percentages of economically disadvantaged students were less likely to have a working email address, while having a working cell phone number did not vary greatly by school locale or demographic composition (figure 3). Almost one-fifth of households opened at least one email (18.9%), and 78.0% of households assigned to receive a text were delivered at least one text message. Email open rates were lower among schools with higher percentages of economically disadvantaged students; open rates or text delivery rates did not vary by other school characteristics. Click rates varied among nudge features from 0.9 to 9.2% (figure 4). Adding text messages and sending communications from a known sender had a positive impact on click rate (an increase of 6.5 and 1.5 percentage points, respectively), while adding a graphic had no impact (figure 5). Impacts were similar for schools with different characteristics (figure 6). Lastly, adding a text message did not seem to influence time spent on the website. Households assigned to both an email and a text message spent a similarly small amount of time (35.5 seconds on average) on the website as households assigned to email only (37.2 seconds). Conclusions Overall, the study found a low click rate (9.2%), but given the low cost of implementation, email and text messages could be a cost-effective communication strategy. These messages could be used as a complement to other approaches such as partnering with local organizations. To send information to families, ADE could consider sending both email and text messages and sending these from the school principal--these low-cost features increased the click rate by 8 percentage points, and were associated with increases among all types of schools. Sending both emails and text messages can help ensure equitable communication with families, as findings indicate more households had a working cell phone number than a working email address, particularly among schools with higher percentages of economically disadvantaged students. Finally, additional research is needed to better understand how households interact with R.I.S.E. resources and what they find useful, as this study found parents/guardians spent little time on the site. This type of information could inform decisions about future investments in education resources targeted toward families. Limitations include that findings do not generalize to other contexts or time periods; authors had no information on how families engaged with R.I.S.E. resources or on family/student demographic characteristics; and there is a possibility of contamination between experimental conditions if households forwarded messages.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: Arkansas
Grant or Contract Numbers: N/A
Author Affiliations: N/A