GoPro Reduces Compute Footprint by 70% Using Amazon ECS
GoPro makes versatile cameras that help people capture and share their meaningful experiences. Whether handheld or mounted to drones, helmets, or other gear, GoPro hardware makes it easy to take photos and videos indoors and out, in motion or at rest, and even underwater. Customers who subscribe to GoPro Plus—the company’s cloud-based video service—can also upload, edit, and store their videos easily.
By taking advantage of Amazon ECS, we have the power to understand, manipulate, and manage our environment easily."
DevOps Engineering Lead, GoPro
Sharing Exciting Moments
GoPro runs its entire cloud-based IT infrastructure on Amazon Web Services (AWS), serving hundreds of millions of API requests daily and storing multiple petabytes of data. Its architecture uses a loosely coupled microservices approach. For GoPro Plus, functions such as identity management, user-profile updates, and media processing are each handled by independent microservices that communicate with but do not rely on one other.
Initially, the company was running each GoPro Plus microservice using virtual machines on a single Amazon Elastic Compute Cloud (Amazon EC2) Auto Scaling group. It used Iron.io to manage worker queueing, batch jobs, and short-running instances. Although this approach worked reasonably well, it did not utilize server capacity as efficiently as possible. Managing individual Auto Scaling groups was complex for GoPro’s DevOps team, and working with a third-party vendor slowed down the team’s ability to make changes and innovate. The company wanted a solution that would simplify management and deployment, increase development agility, and use resources more efficiently.
Why Amazon Web Services
Simplifying to Save
GoPro decided to adopt a container-based approach to achieve these goals. The company considered several container-orchestration technologies—including the Kubernetes open-source platform—before choosing Amazon Elastic Container Service (Amazon ECS). Seamlessly integrating with other AWS services that GoPro already uses, Amazon ECS provides the detailed security permissions, monitoring, and metrics GoPro requires. The familiar development environment also made for a minimal learning curve.
All GoPro Plus worker processes now run in Docker containers managed by Amazon ECS, and the company uses Amazon Simple Queue Service (Amazon SQS) to handle the process queue. Microservices publish events using Amazon Simple Notification Service (Amazon SNS), which can then be consumed by other microservices as needed. This loose coupling provides additional orchestration flexibility because it removes the failure points that exist when microservices need to communicate with one another directly.
Using Amazon ECS has enabled GoPro to greatly increase resource utilization. “We just set parameters such as CPU and memory requirements for each service, and then leave it up to Amazon ECS to place individual containers on the cluster,” says Zaven Boni, DevOps engineering lead at GoPro. “We have been able to reduce our Amazon EC2 footprint of 60 c4.2xlarge instances by 70 percent, and we are no longer paying third-party licensing fees.”
Infrastructure as Code Drives Agility
Using Docker containers managed by Amazon ECS allows the GoPro DevOps team to take an infrastructure-as-code approach, making management and deployment transparent, seamless, and fast. “Our previous solution was a bit of a black box,” says Boni. “The old model had a lot of homegrown code involved, as well as code from Iron.io. By taking advantage of Amazon ECS, all aspects of deployment, networking, and orchestration are standardized and visible to us as code, so we have the power to understand, manipulate, and manage our environment easily.”
The creation of new server resources is completely automated, so developers can focus on software rather than managing servers. Before Amazon ECS, DevOps engineers would spend weeks building an environment for each project. Now developers can innovate much faster. “Getting things up and running in an automated fashion is huge,” says Isaac Gaskin, DevOps engineer at GoPro. “The infrastructure is embodied in Terraform templates. The developer just fills in the name and port.”
Infrastructure as code with Amazon ECS and Docker means server instances are never out of sync. “Before we used Amazon ECS, people could make changes to individual servers, which became unmanageable,” says Boni. “Now, we know all our servers are identical.”
Infrastructure as code also gives developers more control over the services they run. “Developers now have access to Amazon SQS queues so they can modify components and primers,” says Boni. “Changes go through source control and testing, which means we don’t have to monitor every single change. DevOps is no longer a bottleneck.”
GoPro uses Amazon CloudWatch to monitor services, giving developers another mechanism for ensuring the health of their services. “Developers are making their own Amazon CloudWatch dashboards and using automated alerting for queue size and age so they get notified if there’s an issue.”
In addition, the container-management methodology driven by Amazon ECS helps ensure proper deployment. “In our previous solution, when you deployed a new container, if the name was not unique, the old cache would not be cleared, resulting in old code running without our being aware of it,” says Gaskin. “Using Amazon ECS, we have all our containers neatly tagged and semantically versioned so we have confidence we are running the latest code.”
Just as GoPro Plus makes it simple for users to capture and share videos with the world, it uses Amazon ECS to reduce the complexity of its cloud environment—saving money, empowering developers, and reducing time-to-market.
GoPro builds cameras, software, and accessories that help people around the world share their experiences in immersive and exciting ways.
Benefits of AWS
- Reduced Amazon EC2 footprint by 70%
- Empowered developers to monitor and manage service queues for rapid mitigation
- Adopted infrastructure as code for deployment and management consistency
- Sped innovation by enabling rapid deployment of development infrastructure
- Automated cluster and container management to simplify operations
AWS Services Used
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides secure, resizable compute capacity in the cloud.
Amazon Elastic Container Service (Amazon ECS) is a highly scalable, high-performance container orchestration service that supports Docker containers.
Amazon CloudWatch is a monitoring and observability service built for DevOps engineers, developers, site reliability engineers (SREs), and IT managers.
Amazon Simple Notification Service (SNS) is a highly available, durable, secure, fully managed pub/sub messaging service.
Companies of all sizes across all industries are transforming their businesses every day using AWS. Contact our experts and start your own AWS Cloud journey today.