麻豆精品

Get stories like this delivered straight to your inbox. Sign up for 麻豆精品 Newsletter

Like an online retailer trying to woo a customer back by offering a 10% discount on the boots they’ve been eyeing, education researcher Angela Duckworth wanted to understand how to incentivize teachers to log in regularly to an online math platform that aims to help them improve their students鈥� academic performance.

鈥淭oday is perfect for checking your Pace Report!鈥�

鈥淜eep Zearning!鈥�

鈥淏y opening this email, you鈥檝e earned another 100 digital raffle tickets in the Zearn Math Giveaway!鈥�

In partnership with Zearn Math, a nonprofit online math instruction platform used by roughly 25% of U.S. elementary school students, Duckworth and a team of researchers from the University of Pennsylvania鈥檚 Behavior Change for Good Initiative launched a megastudy that peppered 140,000 teachers with different types of email prompts to log into the platform鈥檚 dashboard each week and check their students鈥� progress.

Behavioral scientists like Duckworth, who popularized the 鈥� about a decade ago, spend a lot of time trying to pinpoint what, exactly, it is that prompts an individual to sign a form, become an organ donor or click an ad that promises a secure and safe retirement now.

鈥淚n the case of education there’s the idea of nudging the students directly,鈥� Duckworth said. 鈥淏ut there’s also the idea that’s less commonly studied, which is, what do you do to nudge the teachers, who are not in complete charge, but have a lot of authority about what is going to happen in the classroom that day? It was clear to us that if we could get the students onto the Zearn platform that their learning would progress. But are they actually going to log in?鈥�

To that end, the team developed 15 different types of intervention emails featuring things like planning prompts, teaching tips, learning goals, digital swag and celebrity endorsements. The goal was to change behavior without mandates, bans or substantial financial incentives 鈥� though teachers were enrolled in a giveaway and earned digital raffle tickets every time they opened an email, increasing their chances of winning such prizes as autographed children鈥檚 books, stickers and gift cards. 

The researchers then compared the average number of lessons the teachers鈥� students completed on the Zearn Math platform over four weeks to a control group using Zearn that received only a simple weekly email.

So did it work? Did the emails prompt teachers to log in more regularly? And if so, did the number of lessons their students completed increase? To some degree, yes, it did work. But not at all to the extent that Duckworth and researchers had anticipated. 

The best-performing intervention, which encouraged teachers to log into Zearn Math for an updated report on how their students were doing that week, produced a 5% increase in students鈥� math progress. Emails that referenced data specific to a teacher鈥檚 students 鈥� versus those without that information 鈥� boosted students鈥� progress by 2.3%. And teachers who received any of the behaviorally informed email nudge saw their students鈥� math progress increase by an overall average of 1.9%

Duckworth was sure that the emails featuring famed astrophysicist Neil deGrasse Tyson and literary rockstar Judy Blume would move the needle more than anything else. But teachers were virtually unaffected. 

鈥淲e had sexier treatment conditions,鈥� she said. 鈥淏ut no, it turns out, a simple message that says, 鈥楬ey, your students鈥� data are here, remember to log in,鈥� that is what worked the best.鈥�

Notably, the intervention effects were consistent across school socioeconomic status and school type, both public and private. Moreover, they persisted for eight weeks after the email intervention period ended. Collectively, the reminders resulted in students completing an estimated 80,424 additional lessons during the four weeks their teachers received emails, and an estimated 156,117 additional lessons during the following eight weeks.

Yet the limited impact of the email reminders surprised virtually everyone involved with the study: Students whose teachers received any type of behaviorally-informed email reminder only marginally outperformed students whose teachers received a simple email reminder. In fact, the effect was at least 30 times smaller than forecasted by the behavioral scientists who designed interventions, by Zearn Math staff and by a sample of elementary school teachers. 

鈥淚t鈥檚 a sober reminder that big effects are very rare,鈥� said Duckworth. 鈥淚n general, we鈥檙e finding in our megastudies and what鈥檚 emerging across the social sciences is that intervention effects tend to be very small.鈥�

鈥淥ne of the things that this megastudy has reinforced is a kind of humility about how complicated human beings are and how challenging it is to durably change behavior. A kid is a complicated organism. Teachers are complicated. Schools are complicated,鈥� she continued. 鈥淚t would be naive to think that you could radically change behavior with these like light touch interventions.鈥�

The findings not only underscore the difficulty of changing behavior, but also the need, Duckworth said, for large-scale, rigorous, empirical research on how to drive impact in math, which is a high-priority subject for education policy experts at the moment. 

Indeed, the findings come at an inflection point for math in the U.S. 

The most recent release of the National Assessment of Educational Progress showed that, nationally, average mathematics scores in 2024 were lower by 3 points among fourth-grade students and lower by 8 points among eighth-grade students compared to their scores in 2019 鈥� the most significant drop since 1990. School districts have struggled to rebound after significant academic setbacks incurred by the COVID-19 pandemic. For math in particular, by the spring of 2022, the average public school student in grades three to eight had lost the equivalent of a half-year of learning.

Compared to students in other developed countries, Americans have ranked in the bottom 25% of students globally on standardized tests of mathematics for decades. U.S. students saw a 13-point drop in their 2022 math results when compared to the 2018 exam 鈥� “among the lowest ever measured by PISA in mathematics” for the U.S., according to the Organisation for Economic Co-operation and Development, which administers the exam. 

As a result, a contentious debate has erupted surrounding whether educators are effectively teaching the subject 鈥� and whether they themselves are being effectively taught how to teach it. 

鈥淭here was a dawning realization that there’s a real urgency around math achievement in the United States,鈥� Duckworth said when her team decided to design the megastudy. 鈥淭his very light touch nudge was helpful, but it does underscore how hard behavior is to change. And if there are bigger levers to influence teacher behavior, I think we would have found a bigger downstream effect on student achievement.鈥�

Get stories like these delivered straight to your inbox. Sign up for 麻豆精品 Newsletter