Continually using data to scale up programs that work and moving away from programs that don’t – making the most of limited resources and improving the delivery of services to residents.
Used randomized control trials to test the best way to encourage social distancing during the early days of COVID-19.
To reimagine public safety services, Portland Street Response used geographical data to determine whether a situation warrants a response from police or mental and/or physical support services teams – which helped determine which areas were in most need of social service resources.
Building the City’s Evaluation Muscle
Maser coordinates citywide behavioral science and evaluation efforts, and has spent the last half-decade working with staff across the City to strengthen Portland’s evaluation muscles with support from The Behavioral Insights Team (BIT), an expert partner of What Works Cities.
As of early 2021, staff across the City have run 19 different randomized control trials. The basic approach of these trials is this: create statistically similar test groups, including one that experiences the intervention and one that does not. Then compare the results of the groups to see if the behavior of intervention group participants changed in desired ways.
One example: Testing the best way to encourage social distancing during the early days of COVID-19. In April 2020, when it became clear the City needed to deliver effective public health messages to residents, Maser and staff at the Bureau of Emergency Management jumped at the opportunity to work with BIT. They got to work designing and testing posters to encourage grocery shoppers to stay six-feet away from store staff and customers to prevent the spread of the virus.
Eight variations of a poster were tested via an online randomized controlled trial. Among other things, the results showed that using a message of duty to protect others and showing an image of grocery store staff increased the number of people who understood and remembered the need to stay six feet apart.
“It was incredibly helpful to be able to test a bunch of ideas so quickly and move forward with confidence,” Maser says. The City distributed copies of the poster to grocery stores and heard grateful feedback from store owners who were worried about their staff’s safety. BIT helped design the poster based on COVID-related messaging trials done elsewhere, and then shared Portland’s findings with governments across the U.S. and abroad.
The City’s commitment to evaluations has inspired other cities, and they’ve been quick to join BIT-led multi-city trial cohorts, including efforts to decrease 911 dispatcher burnout and increase voter turnouts.