The Obama For America (OFA) campaign, headquartered in Chicago, Illinois, had one goal: the re-election of President Barack Obama. To achieve their goal, they required the coordination of hundreds of thousands of staffers and volunteers, donations from supporters to enable their efforts, and a Get Out The Vote campaign.
"We knew we were going to go big, and we knew we were going to be resource-constrained," says Harper Reed, chief technology officer for the Obama campaign. "Whether it was money or people, we knew we weren't going to have everything we wanted."
The OFA campaign organization had a choice: they could invest time, resources, and tens of millions of dollars on hardware, or they could host their operations in the cloud and get their campaign efforts running quickly. With the clock ticking to Election Day, the choice to use the cloud was clear.
The OFA organization knew they needed a technology solution that would enable strong, connected and flexible ground operations. They needed a central data store to avoid duplication, they needed to engage volunteers, donors, and voters alike with technology that was appealing to use and quick to develop and deploy, and they needed a solution that would make their operation scale.
For the 2012 campaign, OFA chose AWS as the foundation of their system design. The campaign needed what AWS offered: on-demand utility pricing, elasticity, security, scalability and reliability. Amazon Web Services (AWS) provided OFA with a foundation on which to build close to 200 applications with a set of APIs that leveraged diverse data sets as one shared data source.
“AWS let us engineer our own approach to problems. The AWS Cloud let us build solutions for an environment that moves so rapidly that you can’t plan for it,” says Michael Slaby, chief integration and innovation officer for the Obama campaign. “It made a big difference to the success of the campaign.”
One such solution was CallTool, an app that enabled thousands of volunteers to make millions of calls to voters during the last four days of the campaign. OFA leveraged virtually every service that AWS has to offer, including the campaign website (www.barackobama.com), donation processing, and data analytics to maximize fundraising efforts. OFA used AWS to enable the following solutions for their campaign:
- Dashboard, a virtual field office app that enabled volunteers to join the campaign effort without having to go to a central office. Dashboard also handled metrics, tracking canvassing, voter registration and the campaign’s phone calls to voters.
- Dreamcatcher, an application that micro-targeted voters based on sentiments expressed on social networking sites.
- CallTool, a remote calling tool that allowed volunteers to do phone canvassing from their homes and matched volunteers with voters who had similar life experiences.
- A voter incident tracking tool that was used to collect data on incidents of voter fraud and abuse, illegal electioneering, and other polling-place issues. The tool would alert local attorneys who volunteered for the campaign, who would then be deployed to assess the situation and collect information.
One decision that served the OFA campaign well was an early focus on loose coupling in their design. The organization adopted a pragmatic, deep redundancy approach across dozens of distinct but interrelated systems. OFA took an aggressive risk management approach and supported automatic partial availability for each of their applications, in addition to standard practices for high availability. For example, an application that created a map for delivering leaflets used an external system to prioritize each volunteer’s route. If that system was offline, rather than having the application show an error, it simply displayed an un-optimized map. Similarly, many of the campaign applications could rely on static versions of their sites hosted on Amazon Simple Storage Service (Amazon S3).
With constraints on time and the need for agility, the OFA organization knew that developer volunteers might want to build their own apps. This would have hindered the campaign’s efforts to coordinate its volunteers and make sure its data was as current and accurate as possible. Therefore, OFA built a series of APIs that exposed internal campaign data in a way that new applications could easily consume. Doing so allowed the campaign to present a diverse, distributed set of data resources as if they were centralized. These APIs supported dozens of applications, all of which operated in a coordinated fashion—and as a result, user-facing applications could be built in almost any language, in any framework, leveraging whatever production experience the volunteer developers brought to the effort.
In the system, Ruby on Rails (RoR), Python/Django, PHP, and a host of other front- and mid-tier technologies intermingled, creating a robust heterogeneous design. Below that, the use of 10 different structured storage systems reflected a focus on bringing tools suited to the data itself. Intermingling technologies included relational database management services like Amazon Relational Database Service (Amazon RDS) for MySQL, PostgreSQL, and Microsoft SQL Server; NoSQL software like MongoDB, Apache Hadoop, Vertica, and LevelDB; and Amazon S3, Amazon DynamoDB, and Amazon SimpleDB.
A number of third-party components were also integrated into Obama for America’s solution, including several Software as a Service (SaaS) components from members of the AWS Partner Network. These components included:
- New Relic and Chartbeat to monitor performance
- Cloudability to monitor costs
- CloudOpt to optimize the critical cross-regional transfer and replication operations
- Mashery to provide API delivery tools
- GitHub and Campfire to coordinate developer efforts
- Puppet and Netflix Asgard to automate infrastructure
- OpenVPN to provide network access
Together with AWS management applications like Amazon CloudWatch, AWS Identity and Access Management (IAM) and AWS CloudFormation, this created a robust toolbox for the OFA team to quickly implement and manage this diverse infrastructure. The campaign’s Narwhal data platform, in particular—which underpinned the OFA data analytics and tracking efforts—was built following core AWS best practices.
- Auto Scaling with Elastic Load Balancing (ELB), elasticity via Puppet, AWS CloudFormation and Netflix Asgard
- Multi-AZ with Amazon RDS
- IAM users for rights-limited provisioning
- Regular SSH and AWS Access Key rotations
- Sophisticated DNS management with Amazon Route53
- Aggressive access management via security groups
- Amazon Virtual Private Cloud (Amazon VPC) and AWS Direct Connect
To further engage its campaign staff and volunteers, OFA held several “Gameday” test events, where internal teams would pair off to simulate failure modes in their testing environments. Teams built experience by rapidly identifying and mitigating issues, consistently surfacing new opportunities for automated mitigation, additional redundancy, and other design improvements. These patterns delivered a system that was efficient, robust and available all day during Election Day, November 6, 2012. The result was an inexpensive, always-on, scalable system that served millions of users.
During the campaign, the AWS solution helped facilitate the fundraising of hundreds of millions of dollars, prioritized millions of phone calls and coordinated thousands of volunteers. The system delivered terabytes of data and protected voters from fraud and electioneering.
Using AWS facilitated collaboration on an enterprise scale, connecting small ground teams of volunteers with central offices and engaging volunteers, donors and voters alike with technology that was appealing to use and quick to develop and deploy. Between on-demand utility pricing and the millions OFA saved in not having to make infrastructure investments, AWS enabled the OFA to reduce their costs and use those resources elsewhere.
“Using AWS gave us the flexibility and the power we needed to run a very strong campaign,” Reed says. “Between connecting our volunteers, enabling our fundraising efforts, and delivering analytics, AWS helped set us up for success.”