Ballotpedia and Lambert Labs Optimize Security and Costs on AWS

Ballotpedia is a nonprofit and nonpartisan online political encyclopedia that covers federal, state, and local politics, elections, and public policy in the United States. The website was founded in 2007, and is sponsored by the Lucy Burns Institute, a nonprofit organization based in Middleton, Wisconsin. Ballotpedia wanted a purpose-built tool that could help it automate the identification, ingestion, and curation of information and relieve the burden on staff of performing these tasks. Ballotpedia approached AWS Partner Lambert Labs for help because of its expertise in working with Python and Amazon Web Services (AWS). While working with Ballotpedia on the development of the tool, Lambert Labs identified opportunities to improve security, availability, and scalability for Ballotpedia using AWS Organizations.

Ballotpedia provides a nonpartisan source for political information for US voters. Since its founding in 2007, the organization has gathered, analyzed, and presented information about elections and public policy issues. It relied on AWS to deliver this information to users. However, as it grew, Ballotpedia found that working with an ever-increasing number of data sources presented a challenge.

The organization had grown to the point where it needed to upgrade its infrastructure. “We started with just the very simple website years ago,” says Matt Latourelle, director of technology operations at Ballotpedia. “You can throw a website online, but once you start to get millions of views per month, the complexities and the risks put a load on your technology.”

Ballotpedia wanted to provide even more insights to its users by including information on municipal elections. To do this, it wanted to create a way to automate the collection and processing of data. The organization regularly monitors about 3,000 sites to gather information. This was a largely manual process that required a lot of time and took Ballotpedia staff away from higher-value tasks.

To develop the tool it wanted, Ballotpedia knew it needed outside expertise. After interviewing multiple firms, it chose Lambert Labs, an AWS Partner. “There was a lot of manual work,” says George Lambert, chief executive officer of Lambert Labs. “Manually monitoring 3,000 sites per week, even at just 5 minutes each, that’s 15,000 minutes per week—over 250 hours—just to gather data. Automation made a lot of sense for them.”

As it began working with Ballotpedia to develop the tool, Lambert Labs proved to be a good fit, providing the missing expertise that Ballotpedia needed. “We’d been on AWS for years, running the website there and gradually adding services as we needed them,” says Latourelle. “But we didn’t have the deeper knowledge, the expertise to architect everything we needed. That’s where Lambert Labs fit in.”

The two organizations worked together on developing the automation tool and getting it up and running. During the process, the team at Lambert Labs identified an opportunity to optimize the way Ballotpedia worked on AWS. “They had built up their infrastructure over time and, while it was functional, there were some things that could be improved,” says Lambert. “They had talked about wanting to upgrade security and scalability. We suggested doing an AWS Well-Architected review so we could identify the best use of resources.”

Because of the success of the development project and the good working relationship it had developed with Lambert Labs, Ballotpedia agreed to undertake the AWS Well-Architected review, which helps cloud architects build secure, high-performing, resilient, and efficient infrastructure.

Lambert Labs focused on the pillars of security, reliability, cost optimization, and operational excellence. The review revealed that Ballotpedia was spending significant budget annually on AWS but didn’t have enough insight into how it used the services. It couldn’t tell which users were responsible for which costs or who was using which services, and had few restrictions on permissions. The latter represented a security risk and the former limited insights that would allow the organization to make better use of resources.

Working with Lambert Labs, Ballotpedia used AWS Organizations to create a more efficient environment. It separated workloads and environments and created 10 AWS accounts in AWS Organizations. This meant that users could have their own access with the permissions they needed to do their work, but limited the risk if any account was compromised.

Ballotpedia also gained greater insight into resource usage and permissions, enabling it to optimize costs. Lambert Labs configured the AWS Organizations to use AWS IAM Identity Center—successor to AWS Single Sign-On—which allows companies to securely create, or connect, workforce identities and manage their access centrally across AWS accounts and applications. This has reduced onboarding time for new hires from several hours to just minutes, as well as improving security.

For a nonprofit organization like Ballotpedia, efficient use of resources is important. Lambert Labs has worked with Ballotpedia to educate its eight engineering staff on how to do more with the AWS services they have. “As we built the tool and then revamped our infrastructure, Lambert Labs was sharing knowledge with our engineering team to help us better understand how everything fits together,” says Latourelle. “We’re looking at better ways to deploy our stuff. We were satisfied using AWS, but now we’re getting even more out of it with Lambert Labs.”

Ballotpedia can experience rapid swings in demand in response to political developments and most notably during elections. With the implementation of its automation tool, its optimized infrastructure, and its better understanding of AWS, it’s ready for the 2024 US election campaigns. “We had some hiccups during the 2020 election campaign,” says Latourelle. “Since working with Lambert Labs, we’ve had no downtime and we’re making better use of resources. This will help us do a better job of informing voters.”