Quantcast specializes in real-time advertising, and audience insights and measurement, powered by artificial intelligence (AI) technology. Every day, the company processes up to 40 petabytes of data and provides accurate, direct audience measurement to more than 100 million mobile and web destinations.
For many years, Quantcast hosted its real-time bidding application and other key solutions in several large data centers. As the physical hardware in these data centers aged, Quantcast realized it was time to move its infrastructure into the cloud rather than upgrade its infrastructure. “When we had hardware down, we didn’t always have dedicated technical support people waiting onsite to fix the problem,” says Leah Blank, senior systems engineer for Quantcast. “If a machine went down, sometimes the best we could do would be to mark it decommissioned and work at reduced capacity.” Hardware problems were also a concern from a customer standpoint. “If our real-time bidding systems go down, we can’t support customers and we don’t make any money. Reliability is essential,” Blank says. Additionally, Quantcast sought more flexibility when it came to scaling the compute capacity of its applications. “In the data centers, we had a fixed number of machines, and beyond a certain point we couldn’t scale up quickly,” says Blank.
To lower costs and improve reliability, Quantcast decided to add to its existing Amazon Web Services (AWS) implementation. The company had already made inroads into AWS, including porting the open source Quantcast File System to use Amazon Simple Storage Service (Amazon S3) as a backing store, while adopting several additional AWS services.
Quantcast decided to move some of its critical business applications to AWS, taking advantage of Amazon Elastic Compute Cloud (Amazon EC2) instances. The company recognized it needed to manage the amount of money spent on Amazon EC2 instances and reduce the cost of systems already running on AWS and new systems it wanted to port. In order to address this, Quantcast adopted Amazon EC2 Spot Instances—discounted spare AWS compute instances with prices that fluctuate based on demand.
Quantcast uses AWS to support numerous business systems, including:
- Real-Time Bidder Services: a system that performs real-time bidding on online ads. The system uses AWS Auto Scaling groups for automated scaling, as well as Amazon EC2 Spot Instances. Quantcast currently has the largest deployment of a real-time bidding solution on AWS.
- QuantFlow: a big data computation platform that uses both reserved Amazon EC2 and Amazon EC2 Spot Instances.
- QCLearn: a machine learning system that runs on Amazon EC2 Spot Instances. The system uses Amazon EC2 Spot Fleet instances across two Availability Zones.
By using Amazon EC2 Spot Instances for its critical business systems, Quantcast reduced its operational costs in several ways, including saving up to 60 percent for its QCLearn machine learning system, and spending about 25 percent less on Amazon EC2 Spot Instances than it would have on Amazon EC2 reserved instances for its real-time bidding solution. "As we roll out more infrastructure to AWS, Amazon EC2 Spot Instances are helping us control costs and scale our systems to meet demand,” says Blank.
Quantcast can scale its QCLearn system more easily than it could using its previous data-center model. “Using AWS, we get virtually unlimited scalability built right into the system. For our QCLearn system, we can automatically scale up to 1,000 instances every day to support customer demand and then scale back down later in the day,” says Blank. Previously, acquiring new machines meant a long budgeting and build-out process, or it might require negotiating with other departments for use of their machines. “We have much more flexibility when it comes to being responsive to our customers’ needs—we can scale at will and not worry about being limited by having a fixed number of machines. It’s also much less painful for us to rebuild compute instances, as opposed to rebuilding physical machines.”
Using AWS, Quantcast is delivering highly available services to its customers around the world. “Our real-time bidding service and our other solutions are the core of our business—without those services, we can’t serve ads to our customers,” Blank says. “AWS gives us the reliability we need to consistently provide high uptime, so customers always have bidding capabilities when they need them.”
Learn more about Amazon EC2 Spot Instances.