Scaling to share unprecedented volume of election donation data, quickly and cost-effectively

Photo by Jonathan Simcoe on Unsplash

Campaign contributions have grown exponentially in the United States. In 1980, there were around 500,000 contributions made; in 2020 alone, the Federal Election Commission (FEC) expects 500 million contributions. Meanwhile, the evolution of technology has changed the way Americans contribute to political campaigns, making it easier to make many small contributions. To meet unprecedented demand for data transparency, the FEC turned to the cloud.

Campaign finance transparency

The FEC aims to protect the integrity of campaign finance process, including for the president, House of Representatives, and Senate, by reviewing campaign contributions, enforcing our campaign finance laws, and reporting and sharing information with the public. This independent agency, created in 1974 as part of the Federal Election Campaign Act, enforces campaign finance law and enables disclosure of political campaign financial information. This work is important to protecting the integrity of the campaign finance system and national elections.

Scaling to meet unprecedented demand

The FEC has used Amazon Web Services (AWS) since 2014, when they migrated their mission-critical public facing website application. But since 2014, the FEC has seen huge growth in the volume of data submitted. In 2024, the FEC expects to review and process one billion individual campaign contributions. This increase in the volume of contributions and data has created some challenges for FEC.

“Based on the website’s success, we wanted to look at [our] longer plan to migrate more enterprise applications to the cloud,” said Wei Luo, deputy chief information officer at the FEC. With the 2020 election on the horizon and the important role of FEC, they decided to turn to AWS. Their goal: to improve the website query performance, improve their data ingestion process, and make it easier to access and analyze data internally. What started as a website project in 2014 has now expanded into a digital transformation.

“As much as we prepare, it’s hard to prepare for what we don’t know,” said Wei.

AWS helped establish their Landing Zone in the cloud and helped them build the cloud security operations. They are continuing to work through migration readiness planning activities to prepare them for a broader cloud migration. Even with the increase in processed transactions and the additional website capabilities enabled by the cloud, moving to AWS has saved FEC money, and most importantly, FEC can scale up the infrastructure to meet user demands quickly.

Managing multiple databases

With the data coming in from political campaigns, the FEC maintains three main databases: one primary database, one public-facing database, and one that does process receiving. This complicated system of databases requires data transfer every night. The public-facing database is available on the FEC website. Anyone can go to FEC.gov and view political contributions to any federal election—whether it’s your favorite celebrity, neighbor, or coworkers going all the way back to 1976.

FEC moved their website database to Amazon Aurora. The move saved 50 percent of the FEC’s database cost and improves performance time by 50 percent. They also are implementing Amazon Glue to reduce the extract, transfer, load (ETL) data ingestion process—from hours to minutes.

AWS also provided a data lake architecture using cloud data warehouse Amazon Redshift and business intelligence tool Amazon QuickSight, to allow data to be accessed internally.

Said Wei, “When I start a project, I learn a new thing every day. AWS has guided me through the process—from executive sponsorship briefing to now. AWS has been a key part of application so they understand”

Realizing other benefits

While the main benefits of the move to the cloud were scalability and cost savings, Wei also credits the cloud with enabling innovation and experimentation due to its ease of access. “Innovation is not cheap, but the cloud makes it possible,” said Wei. “During these IT modernization engagements, one of my staff moved things into sandbox environment. It shows they are willing to try—and the cloud makes them not afraid to try new ideas.”

Wei recommends that other independent agencies looking to move to the cloud focus on people. “Everything is about people. Get the right people, get the right executive sponsor—you need to get trust. After you make the business case, you need to have people willing to learn take on a challenge,” said Wei. “We’ve been very fortunate. We’ve had great support from leadership and our executive sponsor. Internally, the FEC has a great team that shares the same passion about our mission and partnership with AWS, which is crucial for success.”

Independent agencies turn to the cloud

The AWS work with FEC is an example of how AWS helps independent agencies take advantage of the cloud and achieve their missions. Learn more about how AWS can help government agencies of all missions and sizes by reading about the cloud for federal government.