Saving While Scaling: How VigLink Cut Costs by 15% with AWS

Guest post by Hanna Fritzinger, Sr PMM, VigLink

VigLink is a platform that allows publishers to transform their online content into an ecommerce business. Ever since our company was founded in 2009, we’ve been utilizing many of the offerings provided by AWS. VigLink’s CEO Oliver Roup jokes, “In our first office, I bought us a server because I was convinced we needed at least one server physically in the office. I installed Linux on it and never touched it again.” But as we began to scale and grow, so did the AWS bill. In this post, we review how VigLink cut costs by 15% with its strategic use of AWS.

VigLink identifies commercial products mentioned within a publisher’s content, and automatically transforms those mentions into revenue-generating hyperlinks whose destinations are determined in  real-time, advertiser-bid auctions. While scaling, the number of views and clicks running through our servers increased steadily. We determined that we were facing three main challenges:

1. We did not have consistent policies on how long to retain raw event level and aggregated datasets.
2. We provisioned our capacity based on the maximum number of servers required for peak traffic without accounting for fluctuation.
3. We wanted to avoid hampering creativity and productivity for the engineering team while still pushing a “cost-conscious” mentality.

After we identified these initial challenges, we set a goal: to reduce the bill by 10% within a few months while supporting an expected doubling of our traffic.

We assigned a member of the engineering team to manage this initiative as to not impact the software development of the rest of the team. We allotted two weeks for him to come up with a comprehensive project plan to lower costs. Once the plan was in place, we looped in software engineering. We prioritized the projects with a close eye on ROI. We constantly asked ourselves the question, “Which cost-cutting efforts would require the least amount of effort with the highest savings?”

The first project we undertook was figuring out how much money we were spending on each part of the VigLink product and which AWS services were being used for that product line. Although AWS Cost Explorer gives us the ability to break down costs on a daily basis, we needed a more granular breakdown. We spoke to a number of service providers that could lead this process, but we found them to be too costly. Our engineering team opted to use an open source tool, Netflix’s Ice, which pulls the AWS cost breakdown on an hourly basis. We tagged all usage with custom tags to track the cost spending in different product lines, and we created an internal dashboard to focus on various VigLink products, AWS services, and AWS Regions.

After looking at this data, we identified our three key problems and their solutions.

a. Rising Amazon S3 costs for data storage.
The solution: Implement lifecycle policies for each S3 bucket, and archive some data on less expensive S3 storage classes like SIA and Amazon Glacier. This lowered our spending on data storage by 25%.

b. Not accounting for traffic fluctuation.
The solution: Identify traffic patterns and increase the number of servers when traffic increases, or scale down when traffic decreases. Understanding traffic means using fewer On-Demand Instances and more Spot Instances.

c. Deepening general knowledge of AWS.
The solution: Learn best practices from the AWS account management team, including both one- and three-year reservation options, as well as all upfront or partial upfront payments.

The resulting work allowed us to save while scaling and cut the total cost of AWS by 15%.

Founded in 2009, VigLink is located in heart of San Francisco and is backed by top investors including Google Ventures, Emergence Capital, and RRE. VigLink manages a publisher network of over 2 million sites and apps, as well as an advertiser network of over 60,000 brands and merchants. Take a behind-the-scenes look of VigLink in this one minute video.