Skip to main content

Search

Carlsbad, California, USA

Data and a Cross-Sector Approach Lead to Street Safety in Carlsbad.

Project Type:
Community Engagement, Finance, Health and Wellbeing, Infrastructure, Public Safety, Transportation

At a Glance

Using road collision heatmaps and other data to inform interventions, the City saw a 19% decrease in all injury collisions.

Monitored progress and changed course when needed to achieve traffic goals using Performance & Analytics strategies.

City’s staff telecommuting policy reduced employee commute time by 47,000 hours and saved the City between $300,000 – $400,000. It has also improved traffic conditions for all city residents and eliminated 424 metric tons of greenhouse gas emissions.

Budget and finance processes require data and alignment with the city’s 5-Year Strategic Plan, to ensure funds are efficiently and effectively allocated to address the most important priorities of the community.

The number of collisions involving bikes and e-bikes was already surging in Carlsbad when, in August 2022, two bikers were killed during a 10-day period. The City had issued an ordinance on e-bike safety a few months before, but the tragedies and an alarming 233% increase in collisions involving bike and e-bikes between 2019 and 2022 promoted greater action. City Manager Scott Chadwick declared a 6-month local emergency, which was ratified by the City Council at its next meeting. The emergency allowed the City flexibility to move quickly and focus resources on encouraging everyone to be safer on the road.

Some residents worried that the emergency declaration would lead to less access for bikes or more traffic. But Chadwick was able to reassure them. “We’re going to let the data guide us,” he said. And that is exactly what they did.

Immediately after declaring the emergency, the City began gathering and analyzing additional data on collisions. They created heatmaps to identify the most dangerous intersections, did a 5-year trend analysis, and set up tracking for the future. Armed with data and streamlined procurement processes as part of the emergency declaration, the City was painting key intersections and bike lanes with high-visibility green paint within two weeks. In fact, they moved so fast that they exhausted the supply of green paint in the region.

Within 30 days of the emergency declaration, the City had a full plan in place for improving street safety. The Safer Streets Together Plan seeks to change public behaviors and attitudes by focusing on education, engineering and enforcement. “It wasn’t just, ‘Here’s an emergency.’ The public saw real things happening in the first weeks and months and that’s how this has changed things so quickly,” Chief Innovation Officer David Graham said.

Six months after declaring an emergency, injury collisions across all transportation modes were down by 19% compared to the same time period in the previous year, and injury collisions related to bikes and e-bikes had decreased 13%. Graham points to qualitative measures of success as well – street safety yard signs and car window clings on display throughout the community, residents saying they are wearing helmets and slowing down, and the city’s partnerships with schools and bike organizations.

Because of the positive trends and evidence of behavior change, in March 2023 the City Council voted to extend the emergency declaration for a few more months. City staff hope that a year’s worth of data and analysis will help build a sustainable approach to traffic safety and that the early positive trends will become permanent.

The traffic safety emergency is not the first emergency that Carlsbad has tackled with data-driven decision making. It took a similar and equally successful approach during COVID. Hopefully the City won’t have cause for testing its emergency response again anytime soon, but having a well-honed system for collecting and analyzing data, and for innovating and tracking outcomes means that no matter what the future holds, Carlsbad will be well prepared to handle it.

“To see transformation in government you have to invest in areas that aren’t readily apparent like data and analytics, process improvement and operational excellence. When we work together with our community to discover shared insights around issues like traffic safety, we can create impactful change.”

David Graham, Chief Innovation Officer

“It’s not easy to change the way people behave on the road. Often, you’re trying to change habits people have had for years or decades, for better or worse. By taking a balanced approach to traffic safety and digging into the data, we can see what’s working and what isn’t working, and learn how to be more effective as we move forward.”

Scott Chadwick, City Manager

Join Our Certified Cities!

Mogi das Cruzes, Brazil

Where Urban Planning is for the Children of Yesterday, Today and Tomorrow.

Project Type:
Community Engagement, Health and Wellbeing, Infrastructure, Parks and Recreation, Youth Development

At a Glance

Created Participa Mogi – an online platform for citizen participation. In its first year, the City received over 1,300 public contributions.

When employment data showed that 25% of Mogi workers were commuting outside of the City, Mogi added 7,000 new jobs in 2022 and increased revenue by 33% ($700 million BRL).

To combat hunger, Mogi das Cruzes created the Social Market Program, which connects farmers to over 1,200 families in vulnerable situations to receive free food.

Deployed innovative Qualitative Data Practices to better understand residents’ needs.

Mogi das Cruzes has found a winning combination. The City of more than 470,000 in São Paulo State is setting itself up for success by using data and community engagement as the foundation of its strategic plan.

Part of this plan includes giving the mic to Mogi’s youngest residents. Mogi das Cruzes wants to be a child-centered city. City officials are asking children for ideas for improving their neighborhoods so that the next generation engages with their government and community. In December 2022, Mogi announced its first “Our Neighborhood Detectives.” Between the ages of 9 and 12, these 24 children will participate in discussions and make suggestions about how to improve the quality of life and urban landscape for all children and adults living in Mogi.

“These boys and girls will represent the children in their neighborhoods and help us create a better city. Their input is an important complement to our data that will improve our decision-making and inspire civic engagement in young people.”

Caio Cunha, Mayor
Image Courtesy of Warley Kenji.

The Neighborhood Detectives project is part of the Mogi Cidade da Criança (City of Children) program, which uses an innovative community engagement and design approach to inform the City’s investments in the wellbeing of its children. Another project within Mogi Cidade da Criança is monitoring air quality for its impact on children’s health. The air quality data is used to make decisions and create action plans, such as enhancing green spaces and encouraging active mobility to reduce emissions.

In addition to engaging its children in planning, the City seeks feedback from residents through regional meetings and neighborhood visits. By thoughtfully and rigorously soliciting resident input, the City is able to use this qualitative data to deepen community impact and better serve residents’ needs. For instance, the City asked for resident feedback about public transportation by conducting surveys in-person on the bus. Bus users were interviewed and the resulting data was used to optimize bus routes.

Mogi das Cruzes also created the Participia Mogi platform for residents to provide input on planning and budget priorities online. The City is opening its internal data up to residents as well, by hosting Open Data Days and making geospatial data available on the GeoMogi website.

“Staff perceive the need to use data in their day-to-day and Certification is helping with this culture shift. Once you have this cultural shift, you can’t go back.”

Caio Cunha, Mayor
Image Courtesy of Warley Kenji.

Additionally, under the guidance of Mayor Cunha, the formula of data and resident input is being used to create a long-term, 40-year plan for the City. The ultimate success of the plan requires institutionalizing recent progress on collecting, managing and analyzing the City’s data. The Mayor’s hope is that residents will expect future administrations to continue the shared vision within a long-term plan: “We wanted this to be a plan for everyone, so we are building a sustainable program that represents the needs of the City as a whole, for today and for the next generation of Mogi residents.”

Join Our Certified Cities!

Buenos Aires,
Argentina

Public Policies Based on Data Lead to Significant Decrease in Infant Mortality

Project Type:
Health and Wellbeing, High-Performing Government, Youth Development

At a Glance

39% decline in the infant mortality rate between 2016 and 2022.

15 minutes: the maximum time it takes for a resident to reach a community healthcare center.

300 individual metrics are being tracked to support the quality and reliability of 115 public services.

100% of Buenos Aires community health centers now operate with electronic medical charts.

The City now has a thorough data strategy, clear evidence-based policies, 30 executive dashboards, and more than 4,300 indicators after creating the Undersecretariat for Evidence-Based Public Policies and the General Directorate of Monitoring and Evaluation.

Reducing the Infant Mortality Rate

Improving the infant mortality rate in Buenos Aires, which was 7.2 deaths per 1,000 live births in 2016, required a multi-pronged strategy, especially because the hospitals and doctors offices are run by public and private entities as well as social security. Additionally, the strategy was not just medical—it required the coordinated approach of the Ministry of Health and the Ministry of Human Development and Habitat to carry out policies that considered both medical and social implications.

Effective solutions to complex and urgent problems require more than passion and good ideas. City leaders need the right data to illuminate the depth and breadth of an issue; that is what sets the stage for smart public policy.

Horacio Rodríguez Larreta, Mayor

The City, which had already worked to build a data-driven culture, took another step forward by implementing electronic medical records in all health and community action centers (CeSACs), collecting healthcare data from across Buenos Aires to better identify at-risk pregnant women and develop integrated interventions to both strengthen health services and create targeted solutions. Specific goals were established:

  • Make healthcare more accessible so that every resident has a community healthcare center less than 15 minutes from their home.
  • All women would receive at least five checkups over the course of a pregnancy and seven pediatric consultations during the baby’s first year.
  • Promote the healthy development of vulnerable children between 45 days and 3 years old through 76 early childhood centers.

With these clear, measurable targets and the increase in higher quality data, all of the goals had been reached by 2022. Additionally, the City reached their goals with an emphasis on transparency: Buenos Aires’ General Directorate of Statistics and Censuses allowed residents to have transparent and reliable access to data as well as a way to monitor and evaluate progress on the measures the City was taking to improve healthcare.

How else has Buenos Aires become a more data-driven government? 

  • Digitized and streamlined their procurement system and made data on city contracts open and accessible.
  • BOTI, the first city-developed chatbot for WhatsApp in the world, had 59 million conversations with residents in 2022. 
  • Developed “green streets” to create more natural space for pedestrian enjoyment and capture stormwater.
  • Created a 3D model of the City to aid neighborhood development and make it easier to see regulations and codes. 

The Ministry of Health constructed seven new health care centers and renovated 10 others. Pregnant women were given priority when making appointments online for primary care visits. Targeted campaigns involving workshops, at-work training and seminars—on subjects including sleeping and eating habits—had reached 7,000 families considered to be highly vulnerable. And the overall impact was clear: The City of Buenos Aires reduced its infant mortality rate by 39% from 7.2 deaths per 1,000 live births in 2016, to 4.4 deaths per 1,000 live births in 2022. 

Buenos Aires’ progress on maternal care and infant health is just one example of how the City’s commitment to improve data quality, quantity and practices is bearing fruit. But a 39% decrease in infant mortality rate is more than a success story for the City of Buenos Aires—it’s a number that represents the prevention of heartbreaking losses in scores of families—and incalculable joy as families watch their children grow up.  

“We have a limited time to transform reality—and data-driven governance can accelerate positive change. By having shared standards and rules for data management, we create a common understanding and language, powering day-to-day change.”

Melisa Breda, Undersecretary of Evidence-Based Public Policy

Tulsa, Oklahoma, USA

Tulsa Scales Up Data-First Innovation.

Project Type:
Communications, Cross-Sector, Economic Development, Education, Energy, Equity, Finance, Health & Wellness, High-Performing Government, Housing, Public Safety

At a Glance

Created a cross-departmental team that identifies the most effective methods for achieving the city’s top goals and leads the city’s data-driven transformation.

Found patterns in 911 repeat call data that signaled the need for a new referral program to deliver specialized healthcare and social services for residents. Within the first three months of launching the program, there was a 70% reduction in calls from its top 911 utilizers.

Partnered city agencies and civic tech nonprofits to develop a text reminder system that reduced missed fines and warrants that have helped the City’s Court see an annual 187,000 increase in revenue.

Using Data to Power Innovation

G.T. Bynum has leadership in his veins. One of the youngest people ever elected mayor of Tulsa, Oklahoma, he’s the fourth person in his extended family to serve in the role since the turn of the last century. But he is the city’s first mayor to place data-driven decision making at the top of a change agenda. Since becoming mayor in December 2016, his administration has marked a turning point in how Tulsa uses data to power innovation and improve the quality of life in Tulsa.

Mayor Bynum didn’t waste any time after being elected. The idea of improving city services and using data to make key decisions was at the core of his mayoral campaign. One of his first moves as Mayor was the creation of the Office of Performance Strategy and Innovation (OPSI). The office works to align the city’s top goals with effective strategies. It quickly became key to the city’s data-driven transformation, says James Wagner, who led OPSI at its inception and is now the city’s director of finance and CFO.

Ben Harris, OPSI’s Data Analytics Manager, convened a team of employees from 16 departments to lead the city’s data governance and strategic planning efforts. The Data Governance Committee, which sets the standard and strategy for data quality, integrity, and use for the city government, has helped integrate the use of data citywide through the creation of a Central Data Repository where any employee or resident can request data.

“Through this cross-departmental team, we encourage transparency, access to data, and a feedback loop; ultimately it creates a trust relationship between departments,” Harris said.

“In addition to teamwork, technology played a huge role in orchestrating communication, automating data movement, securing data, and making it accessible.”

Data Analytics Manager Ben Harris

OPSI and the Committee also facilitate regular sessions with department leaders to focus on the value of performance metrics. These meetings aren’t just about tracking progress reviewing data — they’ve created a new space within the city to cultivate innovation.

“Mayor Bynum and other city leaders have consistently looked to OPSI to drive data-driven innovation work in Tulsa. This matters because we’re making real changes that improve city services and save taxpayers money.”

Chief Financial Officer James Wagner

A Caring Fire Department

For years, the number of calls to the Tulsa Fire Department was increasing, putting stress on their resources and capacity. By analyzing the data, the fire department discovered the source of the increased calls was not an increase in fires, but instead an increasing aging population who needed lift assists. Lift assists are calls to the 911 system for a non-emergency fall — the help the resident is requesting is to literally be picked up off of the ground. The city discovered a repeat lift assist pattern, with some residents requesting a lift assist as many as nine times a day.

Under the direction of Chief Michael Baker, the Fire Department developed and launched the Tulsa Community Assistance Referral and Educational Services (CARES) program, which was designed to connect high-utilizers of the emergency system to healthcare and social service providers. Visits to the highest utilizers became proactive, with the CARES team working on simple fixes such as installing low-cost solutions like handrails and opening up a dialogue with the resident’s primary care doctor. Within the first three months of the pilot, the fire department saw a 70 percent reduction in calls from its top 911 utilizers.

With preliminary results in hand, Baker presented his findings through the TulStat forum.

“TulStat,” based on the successful “LouieStat” program out of Louisville, Kentucky, has created a forum for change in Tulsa. City leaders gather to discuss priority problems, define success, innovate solutions, and develop methods for measuring progress. They identify specific, quantifiable goals, such as average time for reviewing building permit applications (previously 5 weeks, now 92 percent completed in 5 days) or responding to a 911 call, and troubleshoot obstacles to achieving them.

While CARES was developed before Bynum’s administration founded TulStat, having a space to build off of the pilot’s success was critical in connecting more residents to much-needed services. The program has served 204 clients; in 2020, four Tulsans have “graduated” the program and have the needed support services in place for them to live safely in their homes.

In the future, CARES hopes to work with OPSI to expand their data capacity to learn how to predict who is at risk for becoming a repeat caller to the 911 system and intervening early to distribute tools and services. Aligning community resources to provide innovative, proactive care will not only save the city’s Medicare and Medicaid partners money, it could save a resident’s life.

Breaking the Cycle

Working with What Works Cities and the Behavioural Insights Team, OPSI also helped the Tulsa Municipal Court solve a problem that had burdened the court and vulnerable residents for years.

Previously, when the court issued a resident a fine in a criminal case, but that resident wasn’t able to pay that fine on time, the court would offer an extension in the form of a “Time to Pay Order.” Some found themselves with a fine due more than 12 months in the future — enough time for them to save money for the payment, but also plenty of time to forget when it was due. As of early 2018, more than 70 percent of those orders resulted in a failure-to-pay warrant. For many, a warrant can exacerbate the cycle of poverty: a driver’s license might be suspended and additional fines can accrue, pulling someone further into the criminal justice system.

To combat the problem, OPSI partnered with the Court and Code for Tulsa to figure out how to reduce the number of warrants issued. Within a month, a text message pilot project was underway, designed around a simple hypothesis: Many people missed their Time to Pay Order deadline because they forgot the due date or lost paperwork. Together, OPSI, the Court, and Code for Tulsa developed a system to text simple, personalized reminders to a randomly selected pool of Time to Pay Order recipients. The test group received a text message reminder once a month leading up to their deadline.

Image Courtesy of the City of Tulsa.

The results were remarkable. During the six-month pilot, 63 percent of those who received a reminder paid all of their outstanding fees, compared to 48 percent of residents who did not receive reminders. Armed with data showing this 15 percent point increase, the Court system adopted the new reminder system. It now estimates an additional 320 people are paying their fees on time each year, avoiding warrants and additional problems because of the system. The Court benefited as well, seeing an annual $187,000 increase in revenue and a morale boost among employees who helped implement the solution.

“I’ve never been so excited about a job,” said Jamie King, a cost administrator at the court.

At the City’s Core

OPSI’s successful partnerships with city departments go beyond the fire department and courts. Three years in, OPSI has implemented practices and programs that have positioned Tulsa as a leader in data and innovation. In 2017, the office launched Urban Data Pioneers, an award-winning program consisting of teams of residents and city employees who analyze data to help the city solve key challenges and present policy recommendations.

With OPSI’s clear-cut ability to drive innovation, Mayor Bynum decided to integrate the office into the city’s key funding decisions. When Wagner became Director of Finance and CFO in early 2019, he brought OPSI with him to the Finance Department. This has changed the way Tulsa funds innovation. In essence, a data-driven approach has been institutionalized and scaled. Today, the city bases funding on data that proves programs work. OPSI vets data.

“We had the opportunity to take the approach and plug it into the finance department,” Mayor Bynum said. “It helps make it have much more of a citywide cultural impact.”

Join Our Certified Cities!

Tempe, Arizona, USA

Data-Rich Sewage in Tempe, AZ.

Project Type:
Communications, Cross-Sector, Health & Wellness, High-Performing Government, Infrastructure

WWC - Gold Certification Badge for year 2021

At a Glance

Compiled data from the city’s sewage system monthly to track community drug use patterns and understand the depth of opioid use.

Using this data, monitored areas with high opioid use and deployed emergency response resources and abuse prevention interventions to hotspots accordingly.

Utilized similar wastewater testing data and tracking methods to monitor COVID-19 levels and identify outbreaks.

The Desert City’s Approach to Data

Like so many other cities in the country, Tempe, Arizona has been deeply affected by the opioid abuse crisis. The desert city of nearly 200,000 is part of Maricopa County, Arizona’s most populous area. This county also has the highest number of opioid-related deaths in the state.

As this public health emergency became more devastating in Tempe and around the country, Tempe’s leaders realized they needed to step up in two ways. First, they needed to be transparent about the severity of the problem facing the community. Second, they needed to create innovative solutions to help stem the opioid epidemic.

In 2018, the Strategic Management and Diversity Office, in partnership with ASU’s Biodesign Institute, submitted a proposal to the Tempe City Council Innovation Fund. The proposal focused on using wastewater to track the presence of opioid metabolites at the community level. City leaders supported the idea with innovation funds and began a partnership with ASU to tap this unlikely resource for data and to better inform decisions. Today, Tempe is on the cutting-edge of opioid abuse prevention work in the United States and has expanded this partnership to gather data on the presence of COVID-19 in the community.

“Cities may not want to call attention to opioid overdoses or abuse in their community,” said Wydale Holmes, a strategic management analyst in the city’s Strategic Management & Diversity Office.

“In Tempe, we’re courageously saying, ‘Yes, we have that, but we’re also doing something about it.’

It turns out that sewage offers an abundance of public health-related data — including a community’s drug consumption patterns. Tempe leverages its wastewater to identify areas of the city with elevated levels of opioid compounds — and then deploys emergency response resources and abuse prevention interventions to hotspots accordingly. All of this aligns to one of Tempe’s performance measures: ending opioid-related abuse and misuse by 2025, as measured by the percentage of 911 calls likely related to the drugs.

“Tempe is committed to data-informed community solutions. This first of its kind city model using wastewater-based epidemiology data informs strategic policy and operational decisions to advance community health.”

Director of the Strategic Management and Diversity Office Rosa Inchausti

It’s great to have data, but if you’re not doing anything with it or connecting it to resources and strategies for change, then it’s just information.”

Strategic Management Analyst Wydale Holmes

A New Diagnostic Matrix

Testing wastewater for real-time information about key markers of public health — everything from viruses to food contaminants to drug compounds — has been around for decades. But the approach is relatively uncommon in the United States. And no other city is trying to map the needs of residents around the opioid abuse epidemic in this way, said Dr. Rolf Halden, a professor at ASU’s Biodesign Center for Environmental Health Engineering.

“The United States is behind Europe. Every community that has done this work has not abandoned it, which is a testament to how well it works and how successful it is,” said Dr. Halden, who leads the wastewater data collection project in partnership with the City of Tempe.

Dr. Rolf Halden and his team analyzing the wastewater samples.
Image courtesy of the City of Tempe.

Here’s how the wastewater analytics project helps the city identify opioid abuse hotspots and deploy resources strategically. Dr. Halden’s team takes raw sewage samples directly from five collection areas of the city for seven consecutive days each month. The scientists then test for four different types of opioids: fentanyl, heroin, oxycodone, and codeine.

After processing the samples, the ASU team hands off data to Tempe’s Enterprise GIS and Analytics team. The city can see where elevated levels of the four opioids are, and whether the opioids were metabolized or improperly disposed of. But the data contain no personally identifiable information — there is no way to tie data to specific addresses, neighborhoods or businesses.

Created by William Mancini for Fighting Opioid Misuse by Monitoring Community Health and provided courtesy of the City of Tempe.

All data are then published on the public Tempe Opioid Wastewater Collection Dashboard, created and maintained by Dr. Stephanie Deitrick, Tempe’s Enterprise GIS Manager. Through this dashboard and the Opioid Abuse Probable EMS Calls Dashboard, the information is analyzed by a multidisciplinary team, including Tempe’s Fire Medical and Rescue Department, to determine needed interventions. For example, if the data show a rise in opioid use among people under 18 in one area, the city might ramp up in-school outreach efforts. If one area suddenly becomes a major hotspot, the Tempe Fire Medical and Rescue Department can decide which emergency medical services and overdose prevention resources to move or increase to that area.

After implementing interventions, city officials can then track their potential effectiveness by monitoring changes in wastewater data alongside the EMS calls data. It’s a data feedback loop enabling the city to target its efforts — and, hopefully, prevent abuse and deaths.

“Dashboards allow people to quickly see overall trends within the data and to understand who is being impacted and where. Providing context is key when providing data to inform decision-making.”

Enterprise GIS Manager Dr. Stephanie Deitrick

Building a Data-Driven Culture

From its outset, the wastewater analytics project was directly tied to Tempe’s performance measure of ending opioid-related abuse and misuse. The Mayor, City Council, and City Manager set the expectation that reducing calls for opioid misuse or abuse was important to the executive leadership, and that both the goal and the performance measures supporting it needed to be shared internally with the City Council and administrative staff, community partners and externally to residents.

“Whatever we do, we always approach our employees and explain it to them and take the time to have the conversations about what we’re doing and why,” Tempe City Manager Andrew Ching said. “Every job has a purpose, and that job and that purpose exist within the framework of our strategic priorities and performance measures.”

City leaders also worked to communicate their efforts around the opioid abuse epidemic to the general public. The City of Tempe held an Opioid Town Hall in February 2019 to detail the wastewater project partnership and the types of support that Tempe provides to its residents struggling with opioids, their families and caregivers.

Anyone can visit the wastewater data dashboard and the Opioid Abuse Probable EMS Call Dashboard, designed and conceptualized by Dr. Deitrick and her team. The latter dashboard, which launched in 2018, gives Tempe Fire Medical and Rescue Department and the public a window into opioid abuse in the community. It details when calls related to abuse occurred, along with patients’ ages and genders, and the number of times Narcan/Naloxone overdose reversal medication was used during opioid-abuse related emergency calls.

Together, both dashboards inform Tempe officials’ efforts to end opioid misuse and abuse in the city, and help first responders and public health agencies on the ground see the impact of the targeted outreach and other interventions. It’s too early to tell how quickly the city will advance toward its goal — it was officially set in December 2019, when the baseline percentage of opioid-related EMS calls was 3.74 — but the right approach to data is in place to drive progress.

As the COVID-19 pandemic took hold around the world and in Tempe, city leaders have once again partnered with ASU wastewater researchers and are using this data in the same manner as the opioid data. The city is following the data to find areas of greatest need and is directing resources to help.

Join Our Certified Cities!

Seattle, Washington, USA

Seattle: Transforming a City in Fast-Moving Transition.

Project Type:
Communications, Equity, Finance, Health & Wellness, High-Performing Government, Infrastructure

At a Glance

Initiated a data-driven approach to homelessness intervention that reoriented providers’ thinking — and their service delivery models — around the goal of ensuring any experience of homelessness in Seattle is rare, brief, and one-time.

Used a results-based contracting approach to monitor contract progress and encouraged contract managers and providers to meet regularly to review performance data.

Developed a dashboard focused on homelessness-related data from twelve departments to have better situational awareness of the homelessness crisis, in addition to how human services programs are performing.

Seattle is More than a Cup of Coffee

Fast-paced economic development is bringing plenty of high-tech jobs to Seattle and leading to spikes in household incomes, but progress isn’t being felt by everyone. It’s also contributing to a severe shortage of affordable housing and a homelessness crisis that led the City to declare a state of emergency in November 2015. This wasn’t for a lack of funding directed toward the city’s most vulnerable residents; Seattle’s budget for homeless services grew from $29 million in 2005 to $50 million in 2016 while homelessness continued to rise. Struggling to keep up, the City had to take a hard look at how it was tackling the crisis.

In response, the City launched its Pathways Home plan to shift its focus away from emergency, short-term interventions toward longer-term solutions, using data-driven decision-making to guide the way. As the City says, “Every dollar spent on emergency beds is a dollar not spent on strategies that allow people to exit homelessness.” A critical aspect of the plan was to rethink relationships with outside providers that contract with the City’s Human Services Department (HSD) to provide homelessness services, beginning with a pilot of $8.5 million worth of contracts. The pilot was carried out as part of Seattle’s engagement with What Works Cities partner the Government Performance Lab at the Harvard Kennedy School.

George, Maria, and their young son are among families that nonprofits contracted with the City have helped to move into permanent housing.

Through the pilot, providers were no longer tracking indicators like how many beds were filled or meals were distributed, but rather metrics such as how many people moved into permanent housing or became homeless again after being served, and how long they experienced homelessness. The approach reoriented providers’ thinking — and their service delivery models — around the goal of ensuring any experience of homelessness in Seattle is rare, brief, and one time. By using a results-based contracting approach, that’s what the City began holding providers accountable for too; contract managers and providers began meeting regularly to review performance data, enabling the City to troubleshoot problems in real time and spread the most effective practices.

“It’s not just about more money, although more resources is important. It’s also about thinking and how we do our work differently. How do we use data in a way that is not just compliance-driven, but helps us figure out what is working for people we’re trying to support out of crisis?”

Human Services Department Director Catherine Lester

Seattle has just expanded the pilot to $34 million in contracts awarded to bidders following the issuance of the City’s first competitive RFP for homelessness services in ten years. By keeping providers on target with performance benchmarks, the City aimed to double the number of people being moved into permanent housing by the end of 2018. Seattle is also expanding the performance-based model even further — across the entirety of HSD, which invests $105 million in contracts annually. Simultaneously, the City is developing a dashboard that will bring together homelessness-related data from twelve departments to have better situational awareness of the homelessness crisis, in addition to how human services programs are performing. Soon, real-time data will be available to staff, enabling a more coordinated, citywide approach to tackling the problem, tracking vendor performance, and more.

These efforts are part of a larger culture of data use throughout City Hall. Seattle was one of the first cities in the country to pursue open data and has a robust approach to engaging residents that goes beyond simply publishing data sets on its open data portal. The City is also advancing skills it developed with What Works Cities partner the Center for Government Excellence at Johns Hopkins University by rolling out performance management citywide. To help facilitate that process, the City’s Office of Performance is conducting twelve-week engagements with departments on a rolling basis to train staff. “Getting people the right resources — that’s what’s critical to getting the job done,” says former Organizational Performance Director Tyler Running Deer, who also worked extensively to help departments link their performance and budgeting goals. Seattle is also sharing progress toward citywide goals via its performance portal, one of several public-facing ways residents are kept informed.

After data showed that use of a former bikeshare program wasn’t offsetting its cost or meeting users’ needs, the City piloted a dockless model.

By tracking data and seeing what works, Seattle is learning important lessons about when and how to allocate funding, manage programs, or sometimes, when to shut them down. In one recent example, the City rolled out a bike share program, but data showed use wasn’t offsetting the cost and the service wasn’t meeting users’ needs. Bike docks were taking up valuable parking space, much to the dismay of local business owners. A highly-used station was located on a hilltop, so users weren’t returning the bikes, leaving the task to a truck that drove large numbers of bikes back to the dock at the end of each day. Now the City is piloting a dockless model with three different companies that are funding the program through their own revenue, and had to provide a data-collection plan before receiving permitting. Users can take the new bikes on the routes they truly travel and park them in more convenient locations. Without the temptation to concentrate docks in the highest-income areas, the hope is that bike access will also become more equitable. So far, the results seem promising, but for this city, there’s more than time that will tell — there’s also data.

Read more about Seattle’s data journey here.

Join Our Certified Cities!

San Jose, California, USA

Harnessing Silicon Valley’s Genius in San Jose.

Project Type:
Community Engagement, Cross-Sector, Equity, Health & Wellness, High-Performing Government, Infrastructure, Technology

At a Glance

Implemented public-private partnerships to identify barriers to break down the digital divide and make internet access more equitable in one of the nation’s largest cross-sector digital inclusion efforts to date.

Set up grant-based contracts for local community-based organizations to help them deliver broadband to 50,000 unconnected households by the year 2030.

Launched an app for residents to both submit service requests and receive service information from the city, which fields 165,000 service requests a year.

The Silicon Valley

San José’s status as the capital of booming Silicon Valley is hard to miss. Construction cranes dot the skyline, powering the city’s steady downtown growth. Major tech companies like Zoom and Cisco call the city home, Google will soon open a huge new campus on 60 acres downtown, and giants like Apple and Facebook are headquartered in suburbs just to the west. With great pride in its diverse heritage, this is a city oriented toward the future, a stance reflected in San José City Hall. The postmodern structure’s glass-paneled rotunda confidently embraces the Valley’s nearly constant sunshine.

With five major roadways criss-crossing the city, a gleaming new BART line, and a major public transportation hub set to open in 2021, strengthening connections to Oakland and other East Bay locales, San José is poised to become a true hub of the Bay Area. The region’s tech talent and entrepreneurial spirit is alive in City Hall, with leaders and staff of all stripes having spent time in the local private sector. They bring big aspirations of impact to this diverse city of more than one million people, of whom 40 percent were born outside of the United States, and over 10 percent live under the federal poverty line.

“Like any city, we have our share of challenges,” Mayor Sam Liccardo says. “But there’s a lot about San José that can be a model for others in the country. If we can get things right, it can be the next great American city, the next great model of a multicultural, diverse city.”

Underlying San José’s aspirations is a foundational belief in balancing innovation with equity and inclusion. It’s a new take on the Silicon Valley-esque mindset of growth at all costs. And it’s at the core of its “Smart City Vision” to deploy data-driven decision making and technology to continuously improve how City Hall serves residents.

Using Data to Bridge the Digital Divide

Universal broadband access is part of the city’s current “Smart City Roadmap.” The fact that people in Silicon Valley’s largest city lack broadband access was unacceptable to city leaders, so in 2016, working from the premise that internet access is a basic human right in the 21st century, Mayor Liccardo launched the Digital Inclusion Fund, pledging to close the digital divide.

Led by the Mayor’s Office of Technology and Innovation and the City’s Office of Civic Innovation and Digital Strategy, this public-private partnership between local government teams and external partners is believed to be the nation’s largest cross-sector digital inclusion effort to date.

The first step was to learn who lacked access. Working with external partners such as Stanford University, the City’s digital inclusion team used a variety of data sources to identify over 95,000 San Jose households without access to broadband. After creating a heat map of the digital divide down to the neighborhood level, the team canvassed over 600 residents and conducted street surveys and interviews in multiple languages to identify primary barriers to access.

Digital exclusion heat maps developed by the city to identify “digital deserts” and further identify which populations have the least access to a broadband connection. Image courtesy of the City of San José.

“We knew we needed to bring ‘hyper local’ solutions to San José’s digitally underserved communities. We integrated several data sets to develop a geography-based ‘Digital Exclusion HeatMap’ that allows the City and our partners to know exactly where to expand existing programs and develop new solutions — which library, community center, or park, for example, would be most effective in providing free Wi-Fi, device lending, and digital literacy training to our underserved communities in East San José.”

Civic Innovation Director Dolan Beckel

With this essential data in hand, the team identified three critical components for digital inclusion: (1) an affordable broadband connection at home, (2) a working device, and (3) digital literacy skills. The City then made the case to external private-sector funders, including major telecom companies and others in the private sector, to help fund the initiative.

A streetlight in San Jose outfitted with small cell technology. As of June 25, 2020, over 250 small cell sites were on-air and operating across the city, with another 1,479 sites under construction. Image courtesy of the City of San Jose.

Funding for San José’s broadband strategy is bolstered by the deployment of “small cell” technology — basically, 5G-compatible antennae that can be installed on rooftops, streetlights, and other locations. Beckel’s team negotiated innovative outcomes-driven contracts with telecom companies Verizon, AT&T, and Mobilitie on behalf of Sprint: pricing was structured so that the cost per broadband-enabling small cell site built by the telecom giant was tied to the number of sites built. To support residents in need of the other two components of true digital inclusion — working devices and digital literacy — the team set up grant-based contracts with the California Emerging Technology Fund (CETF) and local community-based organizations.

The City is now on its way to improving broadband for all San José residents while simultaneously delivering broadband to 50,000 unconnected households by the year 2030. Since the implementation of the digital inclusion program began, 23 grants totaling $1,000,000 have been issued to San José community-based organizations with the goal of achieving 4,000 “adoptions” in the next year (i.e., connecting previously unconnected households to broadband internet access, ensuring household members have the appropriate devices, and providing digital literacy training). Average connectivity speeds across the city have improved fivefold to 30Mbps per second, and permits approved for construction of small cell sites have skyrocketed — up from five permits total in 2017 to more than 70 permits each week as of early 2020.

The vital importance of closing the digital divide and building out a city-wide digital infrastructure that connects all its residents — and ensures equity in digital access — was underscored by the COVID-19 pandemic. San José was one of the first places in the country under a stay-at-home order, which immediately presented challenges for work and education for the thousands of school-aged children in the city.

San Jose’s #SiliconValleyStrong landing page, which was designed specifically for the broader community in Santa Clara County to give and get assistance during the COVID-19 pandemic.

In addition to its ongoing digital divide work, to respond to the immediate needs at hand, the City quickly partnered with on-demand tech companies like DoorDash to support meal delivery to vulnerable residents. San José also steered critical regional leadership by launching Silicon Valley Strong, a multi-city initiative where residents can give help or get help with COVID-19 related issues. To date, more than 3,000 volunteers have signed up through the online platform, millions of free meals have been distributed throughout the metro area, more than 200 internet-enabled devices have been collected and distributed, and $27 million in donations have been raised.

Fighting Blight While Boosting Resident Engagement: There’s an App for That

San José’s city government has also improved its own digital infrastructure in recent years. In 2019, it launched My San José, a dashboard residents can access either through a mobile app or web browser to both submit service requests and receive service information from the city. Since My San José’s launch, the City has fielded 165,000 service requests a year.

The vision behind this tool, now called San José 311, is to “use data to make it easier for the community and local government to work together to keep San José safe, clean, and engaged,” according to Deputy City Manager Kip Harkness.

San José 311 landing page, where residents can live chat with a city customer service representative, request city services, and view various service data reports.

To that end, the dashboard focuses on five types of service requests: abandoned vehicles, graffiti, illegal dumping, potholes, and streetlight outages. Before, if residents wanted to request services related to these things, they would have to find the right phone number to reach the right call center, and rely on the call center to email the request to the right departments.

Now, residents submit their request through San José 311 and automatically receive information about expected turnaround time and the status of their request. Great customer service is built into the platform where residents receive confirmation that an issue has been resolved and an opportunity to provide feedback to the city. The City can now collect and analyze a robust set of data on specific service request areas and neighborhood needs, and strategically deploy staff and resources to boost efficiency and productivity. Real results have come from the new far more user-friendly system, including:

— Abandoned vehicles. The average initial response time for inoperable vehicle removal dropped from 15 days to four days over the last 12 months. Average time to complete a service request dropped to about nine days, from 27 days. A giant backlog of over 4,000 service requests was prioritized so that important requests were not left waiting, and has now been whittled down so that few high-priority requests remain untackled.

— Illegal dumping. Response times to cleanup requests used to take up to six months — they’ve dropped to less than a week thanks to work done over the past two years. With service requests far easier to make (they’d been handled over the phone historically), they have doubled within a year of the dashboard’s launch. The City staffed up, using service request data to justify an increased number of workers.

These kinds of improvements deliver tangible benefits to residents. Various studies have shown a high degree of correlation between neighborhood cleanliness and crime. The City is determined to use data-driven tools to remove early signs of blight — thereby preventing the need for more police services down the road. And there’s another benefit: Residents involved in service requests report being more engaged with the city, Harkness says.

“Responsiveness matters to those feelings of being heard and being engaged.”

Deputy City Manager Kip Harkness

Further progress is on the way, including language translation in Spanish and Vietnamese to increase community engagement, and piloting a chatbot to help reduce calls into the call center. Additionally, the City plans to add new services to the platform, starting with automating manual processes for recycling. The city is only “scratching the surface on the data we’re getting from San José 311,” Harkness adds.

Charting a ‘Smarter, Leaner’ Future

San Jose’s data-driven, iterative approach to innovation — which embraces “failing fast” to drive continuous improvement — is familiar to anyone who has spent time around tech startups. But the City’s mindset isn’t just a reflection of its Valley surroundings. It’s taken root out of necessity.

Through a long, slow recovery from the Great Recession of 2008, the City has had to use data to drive efficiency. Budget cuts reduced the city’s workforce by 16 percent from its pre-recession peak. Compared to similar-sized cities, San José has an extraordinarily lean staff: 6,700 employees for just over one million residents.

“We’re the leanest big city in the country,” City Manager Dave Sykes says. “We cannot just throw resources at a problem to solve it. We need to be making decisions that are informed. We have to be smarter, leaner about how we do our work.”

“We have the capacity to use Silicon Valley’s genius to make this a valley of opportunity — that’s really important to us,” Beckel says. “We have a core of people who push hard to find different ways to do things better. This is a city and team laser-focused on addressing what matters to people in this city.”

“We are starting to measure what it is that the community wants and support the priorities of elected officials with data. The open data portal is the perfect place around which to coalesce those conversations.” says Chief Data Officer Joseph D’Angelo

A visual representation of the city’s approach to prioritizing work in service of working “leaner and smarter.” Image courtesy of What Works Cities, February 2020.

And, once an initiative gets underway, city officials look for evidence of success quickly. If something “fails fast,” that’s OK — it informs how the city uses resources going forward.

When discussing new projects, Mayor Liccardo frequently challenges his staff to always view a project from the question, “How does this benefit someone in our community who is at greatest risk?” At the end of the day, the city staff’s focus and purpose are here to solve real world problems for the community, particularly those with the most need.

Join Our Certified Cities!

San Francisco, California, USA

San Francisco: Building Stronger Neighborhoods and a Data-Fluent City Hall.

Project Type:
Community Engagement, Communications, Environment, Health & Wellness, High-Performing Government

At a Glance

Since becoming one of the first local governments in the country to pass an open data policy in 2009, the City has continued to build on its commitment to transparency and putting data at the core of decision-making.

Surveyed 500 residents in five neighborhoods to determine and address the top concerns that have the greatest impact on the residents’ quality of life.

Implemented data skillbuilding courses for City staff resulting in each saving an average of 1.4 hours weekly — translating into $1.7 million in savings annually for the City.

San Francisco’s Fix-It Efforts

Residents of San Francisco’s Glen Park neighborhood gathered one evening with city staff outside the local BART train station in preparation for a Fix-It walk. The group would spend the next two hours walking around, noting problems such as street light outages or traffic congestion and developing a plan for repairs.

Judy, a Glen Park resident for 30 years, says she joined the Fix-It effort so that she could “get involved before the changes are implemented, rather than complaining about them after the fact.” That’s exactly what Fix-It Director Sandra Zuniga wants. “Doing fixes the community doesn’t want me to do is a waste of my time,” she explains.

Fix-It Director Sandra Zuniga speaks with a resident before a Fix-It walk in San Francisco’s Glen Park neighborhood.

The late Mayor Ed Lee launched the Fix-It initiative in May 2016 as part of his Safe and Clean Neighborhoods Promise to improve the quality of life in San Francisco. Making the city a better place to live and providing more efficient government services are part of his vision for San Francisco’s open data strategy as well. Since passing an open data policy in 2009 — becoming one of the first local governments in the country to do so — the City has continued to build on its commitment to transparency and putting data at the core of decision-making.

Fix-It is a great example of how data can ensure that cities are targeting the neediest communities. Initially, Fix-It was rolled out in five pilot neighborhoods where the City received a lot of complaints. Before Mayor Lee decided to expand the program into 20 more neighborhoods, the team surveyed some 500 residents in five languages in five neighborhoods to determine the top concerns that have the greatest impact on residents’ quality of life. The team then worked with the Mayor’s Office of Civic Innovation to map data from 311 and the San Francisco Police Department to visualize where those three concerns were most concentrated, leading to the identification of new target neighborhoods.

A handout distributed to attendees of the Glen Park Fix-It walk maps problem areas they identified during a community meeting held a few weeks prior.

In City Hall, Chief Data Officer Joy Bonaguro directs efforts to ensure that the broadest, best use of data is embedded in the City’s culture through DataSF, a team responsible for maintaining the City’s open data portal and supporting staff with data. DataSF offers a four-month engagement to departments that identify a challenge ripe for data science. Receiving the assistance is a two-way street, however, so each department must remain committed to a service change, if that’s where the data leads. The first cohort has been tackling issues that include keeping WIC-eligible women and their infants enrolled in a nutrition program and increasing eviction prevention.

Since 2014, Data Academy, a partnership between DataSF and the Controller’s Performance Unit, has grown from a handful of workshops on data visualization to nearly 20 courses on behavioral economics, information design, lean process mapping, and more. The goal, according to Bonaguro, is to “empower staff with the data skills that help them thrive.” In turn, they’re helping the City thrive as well. By March 2017, more than 1,700 city staff had attended training, and by taking what they’ve learned back to their teams, they are each saving an average of 1.4 hours weekly — translating into $1.7 million in savings annually for the City.

The City’s Performance Director, Peg Stevenson, notes that increasing staff capacity has prompted employees to help each other to problem-solve. There’s another ripple effect as well: policymakers are more frequently asking for data, and there are clear benefits for residents, too.

As the City continues to apply data to efforts promoting “real-time democracy on the ground,” as Bonaguro describes Fix-It, residents like Judy are showing up and demanding it.

She may just be starting to see the presence of Fix-It in her neighborhood, but Judy already seems to have a hunch that the effect of small fixes can really add up.

“It’s these little things that make your life good or bad,” she says.

Read more about San Francisco’s data journey here.

“Data helps people better understand what they should be expecting from the government.”

Deputy Chief of Staff of the Mayor Kate Howard

Join Our Certified Cities!

Salinas, California, USA

Salinas Data-Driven Youth Violence Prevention Strategy.

Project Type:
Community Engagement, Communications, Cross-Sector, Health and Wellness, Public Safety. Youth Development

WWC - Silver Certification Badge for year 2021

At a Glance

Implemented a strategic, data-driven plan that directed increased funding to a street outreach program that decreased the youth assault victim rate from 22% in 2007 to less than 10% in 2019. That success came from implementing a comprehensive approach to violence reduction.

Conducted a three-year data-driven effort that helped provide a clearer picture of how the city’s police department could better match calls for service with staffing and police officer beats.

Used online capacity assessments to help City staff concentrate outreach efforts efficiently and boost the effectiveness of external partnerships.

Making Safe Choices

In 2010, the Obama administration kicked off a pilot program to address violence in some of America’s toughest places. On the shortlist, alongside large urban centers like Chicago, Detroit, and Boston, was the midsize Central California city of Salinas.

The problems Salinas faced were not well known on the national stage, but residents were all too familiar with the gang violence plaguing their city. In just the first three months of the year prior, Salinas had seen a dozen homicides — and victims of violence were largely people under the age of 25. The inclusion of Salinas in the federal pilot highlighted both the severity of the problem and the potential to solve it.

“The others were bigger cities with bigger problems, but our crime rate was just as high,” says Jose Arreola, community safety administrator for the City of Salinas and director of the Community Alliance for Safety and Peace (CASP). Led by the City’s community safety division, CASP launched in 2008 to convene public and private stakeholders around the problem of youth violence. The program caught the eye of Justice Department officials who saw its potential.

As a result of the federal support, CASP was able to implement a strategic, data-driven plan that resulted in the rate of youth assault victims dropping from 22% in 2007 to less than 10% in 2019. That success came from implementing a comprehensive approach to violence reduction that involved increased funding to a street outreach program based on an assessment of gaps in current services provided by the City and nonprofits. The youth victim rate continued to drop in 2020, but increased slightly in 2021. Arreola attributes the increase to disruption of in-person services during the COVID-19 pandemic, which closed schools and paused street outreach efforts.

The successful violence prevention strategy is a major example of how Salinas’ commitment to foundational data-driven practices — especially performance & analytics and evaluations — has translated into meaningful results.

“The data has shown us that we’re moving in the right direction in terms of strategy and tactics.”

Community Safety Administrator Jose Arreola

‘Relationships Are Key’

What worked in Salinas is two-fold. Externally, CASP deploys a street outreach team that takes time to build trust and relationships with gang-affiliated youth. The aim is to reach individuals who are at risk before they become a victim of violence. Internally, the program convenes agencies and nonprofits and engages with them regularly, so that CASP staff can fast-track youth to the right types of services.

Four years ago, high school administrators identified the need to have on-site mental health services for students. Through CASP, the school district was able to connect with the City’s behavioral health department to place counselors at each school. They are still there today, a testament to the lasting effects of the program’s connections.

“The whole strategy of CASP is that relationships are key,” Arreola says.

CASP staff and partners at a 2016 meeting. Image courtesy of the City of Salinas.

His team relies on data to track the program’s progress. They work in partnership with the Monterey County Health Department to process data (from the police department and other agencies) and determine what is statistically significant. Homicide data, for example, is not useful in determining what drives violence, since it tends to be erratic. Heat maps that identify areas where violent assaults happen have been more helpful in guiding where the program directs its efforts, and in showing that it works. Arreola says his team is particularly proud of how the maps show hotspots diminishing over time.

“That represented thousands of youth not victimized by violence and thousands of families not living in fear,” he said.

CASP also uses data in the form of capacity assessments to gauge the engagement levels of existing partners, which helps City staff concentrate outreach efforts efficiently and boost the effectiveness of CASP’s external partners. Assessments have also helped identify new partners. For example, mindful that hospital-based violence intervention programs have proven to be valuable parts of other cities’ comprehensive strategies, the City launched a pilot out of the Natividad Medical Center’s Level II Trauma Center. The program, which offers trauma-informed care to violently injured patients, is now permanent.

Another reason CASP has been so effective is that it bridges a common gap among cities with gang violence: It’s not easy to connect young people who have dropped out of school with services. They often do not get any help until they make contact with the justice system after being arrested.

Having the data to show that CASP works helps sustain the effort. The initiative is now supported by a variety of funders including foundations and government grants.

“Funding for something like this can be a tough sell. Ideologically, a lot of people believe kids involved in this lifestyle are making a negative choice and don’t deserve help. That’s a hard narrative to push back against.”

Community Safety Administrator Jose Arreola

An Evolving Police Department

The progress Salinas has made in reducing gang and youth violence has coincided with a period of transformation within the City’s police department. One goal was to make the department’s officer corps look more like the people they interact with.

As the city’s population shifted from 47% Hispanic in 2016 to 73% Hispanic in 2021, the police department shifted its hiring practices to prioritize adding Spanish-speaking staff. Over that period, it has hired an additional 26 officers who are Hispanic, for a total of 73. The department also more than doubled the number of female officers over that period.

A Salinas police officer at work. Image courtesy of the City of Salinas.

“We can truly say that we are the community and the community is the police department,” Police Chief Roberto Filice says.

The department also embarked on an efficiency study in 2017, a three-year data-driven effort that helped provide a clearer picture of how it can better match calls for service with staffing and police officer beats. As a result, it may implement a new beat system for the first time in three decades to spread the workload more evenly across staff.

The department also piloted a fourth shift this year that helped align staffing with times of day when calls for service are more frequent. The fourth shift, which overlapped with existing morning and afternoon shifts, helped the department maintain its response time — four minutes and 28 seconds — even as it saw an increase in calls and operated with an overall reduced staff. In the last four years, 15 positions have been cut from the force due to budget reductions: the equivalent of an entire shift.

Chief Filice says he won’t be able to continue the fourth shift or make the beat changes immediately due to the staffing shortage, but he is confident that the numbers demonstrate the value in innovating and realigning the department to fit the community’s evolving needs.

“I am a data-driven person. I love using technology to make us more efficient.”

Salinas Police Chief Roberto Filice

Join Our Certified Cities!

Portland, Oregon, USA

Keep Portland Data-Driven.

Project Type:
Communications, Health and Wellness, High-Performing Government, Public Safety

WWC - Silver Certification Badge for year 2021

At a Glance

Continually using data to scale up programs that work and moving away from programs that don’t – making the most of limited resources and improving the delivery of services to residents.

Used randomized control trials to test the best way to encourage social distancing during the early days of COVID-19.

To reimagine public safety services, Portland Street Response used geographical data to determine whether a situation warrants a response from police or mental and/or physical support services teams – which helped determine which areas were in most need of social service resources.

Celebrating Oregon

Portland, Oregon has long been celebrated for thriving cultural scenes that produce outstanding artisanal foods, craft beer, and music. Talk to a good governance wonk about the City, though, and something else might come up: randomized control trials. One of the most rigorous tools for evaluating whether something actually works as intended, the trials were all over the news in recent years as pharmaceutical companies developed COVID-19 vaccines. But governments are increasingly using this valuable evidence-building innovation tool as well. Portland has been a trend-setter.

Over the past few years, the City has emerged as a public-sector leader in data-driven evaluations, leveraging randomized control trials to support social distancing public health messaging, police recruitment, and disaster preparedness efforts. By learning what works, city leaders have been able to scale up programs and messages with confidence, making the most of limited resources and improving the delivery of services to residents. And if a trial reveals an intervention doesn’t have impact? That’s valuable too, allowing staff to pivot away from less effective ends to different approaches, and avoid wasting time and city resources.

Building the City’s Evaluation Muscle

“The reality is that sometimes what we think will work doesn’t work,” says Lindsey Maser, who works in the City’s Bureau of Planning and Sustainability. “Rigorous, data-driven evaluation methods allow us to understand what works, for whom, and by how much.”

Bureau of Planning and Sustainability Lead Lindsey Maser

Maser coordinates citywide behavioral science and evaluation efforts, and has spent the last half-decade working with staff across the City to strengthen Portland’s evaluation muscles with support from The Behavioral Insights Team (BIT), an expert partner of What Works Cities.

As of early 2021, staff across the City have run 19 different randomized control trials. The basic approach of these trials is this: create statistically similar test groups, including one that experiences the intervention and one that does not. Then compare the results of the groups to see if the behavior of intervention group participants changed in desired ways.

One example: Testing the best way to encourage social distancing during the early days of COVID-19. In April 2020, when it became clear the City needed to deliver effective public health messages to residents, Maser and staff at the Bureau of Emergency Management jumped at the opportunity to work with BIT. They got to work designing and testing posters to encourage grocery shoppers to stay six-feet away from store staff and customers to prevent the spread of the virus.

The eight poster designs tested, with the winning design indicated by the red circle. More information via the City of Portland’s website.

Eight variations of a poster were tested via an online randomized controlled trial. Among other things, the results showed that using a message of duty to protect others and showing an image of grocery store staff increased the number of people who understood and remembered the need to stay six feet apart.

“It was incredibly helpful to be able to test a bunch of ideas so quickly and move forward with confidence,” Maser says. The City distributed copies of the poster to grocery stores and heard grateful feedback from store owners who were worried about their staff’s safety. BIT helped design the poster based on COVID-related messaging trials done elsewhere, and then shared Portland’s findings with governments across the U.S. and abroad.

The full version of the winning poster from Portland. Image courtesy of the City of Portland.

“Portland takes every opportunity to pursue meaningful evaluations. Its staff knows how to scope an opportunity, and they have the skills to follow through due to their range of evaluation experiences across departments.”

BIT North America Principal Advisor Carolina Toth

The City’s commitment to evaluations has inspired other cities, and they’ve been quick to join BIT-led multi-city trial cohorts, including efforts to decrease 911 dispatcher burnout and increase voter turnouts.

A New Evaluation Normal

Maser is proud of the City’s evaluation efforts and how they’ve generated data to show the best path forward. But she also notes that randomized control trials can be time-consuming and require new ways of working. Building a strong muscle for evaluation work often requires cross-departmental collaboration and buy-in. That can be challenging, especially in the last large U.S. city operating under a commission form of government, in which the executive function is split amongst members of council. This structure has a tendency to breed silos, but Portland has risen to meet this challenge, showing other cities that there’s always a way.

And then there’s the reality that not all trials offer decisive results for the City.

“We’ve tested things we thought would have a positive impact, and then saw no effect,” Maser said. “While it can be disappointing, we always learn something that helps us keep improving.” In some trials, teams have realized they had to do something bigger to have an impact.

“The scale of the intervention needs to meet the scale of the problem. Having clear data gives staff evidence to advocate for more resources.”

Bureau of Planning and Sustainability Lead Lindsey Maser

Portland’s culture of data-driven evaluation has spread across the City. After working with BIT on multiple randomized control trials, city staff have started running their own. A staff member with the Environmental Services department tested different approaches to increase signups for a water bill discount program, and the Portland Bureau of Emergency Management ran a trial aiming to increase city employee preparedness for earthquakes. All this work is made visible to city staff via an internal e-newsletter and the public via the City’s website.

One high-visibility data-driven pilot launched in early 2021 is Portland Street Response, part of city efforts to reimagine public safety services. Designed in partnership with Portland State University, the program aims to assist people experiencing homelessness or low-acuity behavioral health issues. If 911 dispatchers determine a situation does not warrant a response from police, firefighters, or an ambulance service, a street response team featuring a therapist, paramedic, and two community health workers is sent instead.

The Portland Street Response Team does a “team huddle” on the pilot’s launch day (February 21, 2021), just before heading out to answer their first call. Image courtesy of the City of Portland.

The pilot’s geographical focus is the product of an analysis showing that 60 percent of resident complaints received via 911 calls occurred in the greater Lents neighborhood. It’s a highly diverse half-square-mile area — 150 languages are spoken by Lents residents — that lacks social service resources. The pilot kicked off in February 2021 and expanded its boundaries in April. With additional street response teams added as time goes on, the City expects to be handling 30,000 service calls annually by 2022, with the twin goals of addressing the root causes of homelessness and reducing demand for police and fire department services. The City tracked data to support its evaluation of the pilot’s impact after six months and one year.

For Mayor Ted Wheeler, a data-driven mindset has become the norm in the City and the benefits are clear.

“I’m seeing and hearing more and more from staff who are embracing the benefit of evaluations. I don’t know of a single project in the City that does not use some kind of reliable statistic, survey, or data. Ultimately, that leads to more effective city communications, policies, and programs for residents.”

Portland Mayor Ted Wheeler

Join Our Certified Cities!

Disclaimer