605 LLC Scales Analytics Applications to Measure 21 Million US Households Using Amazon FSx for Lustre
2021
Independent TV measurement and analytics firm 605 LLC (605) drives insights from historical data to help networks, programmers, brands, advertisers, and agencies understand engagement and measure advertising effectiveness. By 2019, 605 had collected so much data that its database couldn’t efficiently scale to ingest new data, inhibiting the company’s ability to provide near-real-time insights to customers. So 605 decided to optimize its cloud architecture on Amazon Web Services (AWS) to improve efficiency and scalability and reduce costs.
The ability to cost-effectively deliver at scale on AWS is existential for us.”
Dan Berg
Chief Technology Officer, 605 LLC
Dealing with Creeping Storage Costs
Since its 2016 founding, 605 has used AWS as its primary compute and data storage environment. Originally, it stored and analyzed a smaller amount of historical data using Amazon Redshift, a cloud data warehouse, and Amazon Elastic Compute Cloud (Amazon EC2) I3en Instances, which offer low-price solid state drive instance storage. Because 605 adds hundreds of millions of rows of viewership and ad exposure data daily, it needed a more scalable solution.
To improve backend compute performance, 605 migrated its storage to Amazon FSx for Lustre, which provides cost-effective, scalable storage for compute workloads. 605 was also able to save significantly by scaling down from Amazon EC2 I3en Instances to Amazon EC2 R5 Instances, which are well suited for memory-intensive applications like near-real-time big data analytics. The company also built a data lake that holds over 4 PB of data on Amazon Simple Storage Service (Amazon S3), which offers industry-leading scalability, data availability, security, and performance. Further, to reduce deployment times, 605 migrated to Amazon Elastic Kubernetes Service (Amazon EKS), a managed container service. Migrating to this new architecture without downtime took about 2 months. Upon completion, 605 immediately saw cost savings and efficiency gains.
Deploying in Minutes Instead of Hours
Using Amazon FSx for Lustre, 605 adds terabytes of data every few days. “Being able to increase high performance storage capacity by clicking a button in Amazon FSx for Lustre and being able to scale efficiently are huge benefits,” says Jared Williams, director of DevOps at 605. Further, 605 reduced deployment times from 2 hours to minutes using Amazon EC2. And by using Amazon EMR—a solution for rapidly processing, analyzing, and applying machine learning to big data—605 can spin up thousands of Amazon EC2 instances when it needs to reprocess huge datasets.
On AWS, 605 cut costs by 50 percent without decreasing performance. Using Amazon EKS, the company can scale simply and ingest data into its backend storage more quickly, using this historical data to prove to its customers that their advertising is working. “The ability to cost-effectively deliver at scale on AWS is existential for us,” says Dan Berg, chief technology officer of 605. Because AWS manages the infrastructure, 605’s engineers can focus on innovating, and the 605 team can compete with larger companies. Additionally, 605 has gained a more granular look at its cloud environment, which improves security and cost management.
Improving Data Analytics on AWS
On AWS, 605 enhanced performance and scalability while cutting costs, driving better insights for its customers. The company also uses machine learning to better pinpoint data drops and other anomalies in its incoming source data. “A key part of what we offer to customers is sophisticated data analytics and measurement,” says Berg. “Without the ability to scale infrastructure dynamically on AWS, we wouldn’t be able to do that.”
605 LLC Reference Architecture
Click to enlarge.
About 605 LLC
605 LLC is an independent TV measurement firm offering advertising and content measurement, full-funnel attribution, and analytical solutions. Its matchable, deterministic, and reportable multisource viewership data covers over 21 million US households.
Benefits of AWS
- Reduced costs by 50%
- Quickened deployments from 2 hours to minutes
- Adds thousands of Amazon EC2 instances on demand
- Increased team productivity
AWS Services Used
Amazon FSx for Lustre
Amazon FSx for Lustre is a fully managed service that provides cost-effective, high-performance, scalable storage for compute workloads.
Amazon EKS
Amazon Elastic Kubernetes Service (Amazon EKS) is a managed container service to run and scale Kubernetes applications in the cloud or on-premises.
Amazon EC2
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides secure, resizable compute capacity in the cloud.
Amazon S3
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance.
Get Started
Organizations of all sizes across all industries are transforming their businesses and delivering on their missions every day using AWS. Contact our experts and start your own AWS journey today.