More and more businesses are using Amazon EC2 Spot instances to run and scale their infrastructure cost-effectively. Here are some examples of how customers have achieved business agility, cost savings, and scale with EC2 Spot instances.
Big Data & Analytics
NextRoll (formerly AdRoll)
NextRoll is a global leader in ad retargeting, providing cross-platform reach across large display inventory sources and tools to enable personalized ad campaigns. NextRoll chose to run their business with Amazon EC2, S3, and DynamoDB, because of low latency, guaranteed throughput, and the ability to scale quickly. Additionally, they use Spot for variable capacity, allowing them to run workloads more quickly and efficiently at a lower cost.
Learn how web portal and online service provider, AOL, runs Big Data workloads using Amazon EC2 Spot.
BloomReach built a personalized discovery platform with applications for search, content marketing, and merchandizing. They launch up to 2,000 Amazon EMR clusters and run 6,000 Hadoop jobs every day. With Spot and Amazon EMR, they have increased efficiency while reducing costs.
Inneractive has based its entire ad exchange infrastructure on Amazon Web Services (AWS) since 2010. It currently operates three fleets of Amazon Elastic Compute Cloud (Amazon EC2) instances to ensure it is always running as cost-effectively as possible. For base capacity, Aviv and his team have up to 100 Amazon EC2 Reserved instances. Inneractive saves money by paying in advance for instances that exist for one to three years, rather than paying the on-demand price. The company’s next, and largest, fleet of Amazon EC2 instances is made up of around 800 Spot instances. This tier provides the best cost-performance. The prices of Spot instances fluctuate according to demand.
With over 800 million users, ironSource leverages Spot to scale their production environment quickly while saving up to 80%. ironSource powers their big data solution, ironsource Atom, with EC2 Spot so they can reliably handle any amount of data while reducing costs by tens of thousands of dollars.
"Analytics is at the core of Lotame platform which helps our customers to maximize the value of data. We use Spot instances to run large-scale Big Data Analytics workloads. The new EC2 Spot Instances pricing model makes it even easier for us to acquire Spot capacity at predictable prices, giving us more confidence in the savings and stability we can get by moving additional workloads to Spot Instances."
Learn how Mapbox combined the right architecture, Amazon EC2 Spot instances and some creative orchestration, to run their application with extremely low COGS.
Learn how puclic transit app, Moovit, uses Amazon EC2 Spot instances to accelerate data processing and save costs.
Pinsight Media is a mobile data and insights company based in Kansas City, Missouri. Each day, Pinsight gathers and processes more than 80 terabytes of anonymized location signals, packet layer data, and other kinds of mobile-carrier signal data. To keep costs manageable, Pinsight configured its Amazon EMR pipelines to use Amazon EC2 Spot instances.
"As we roll out more infrastructure to AWS, Amazon EC2 Spot instances are helping us control costs and scale our systems to meet demand."
Leah Blank, Senior Systems Engineer - Quantcast
Using Amazon EC2 Spot instances, red violet can access spare compute capacity available in the AWS Cloud at steep discounts. Using Amazon EC2 Spot instances reduces red violet’s compute costs between 50 and 70 percent compared to using on-demand instances. This empowers red violet to increase compute capacity without increasing its budget.
TellApart’s big data platform enables retailers to unlock the power of their customer data. They use Amazon Elastic MapReduce to bring up Hadoop clusters to batch process log data, and have reduced costs by 75% by using Spot Instances.
"We are beating the 3 year reserved hourly rate, plus we are only utilizing nodes when needed. These Spot instances are not on 24/7 so we are saving significant amounts of money by only using machines when we need them. In short, we like Auto Scaling with EMR and Spot instances!"
Brian Filppu, Director Business Intelligence - Zillow
Salesforce DMP leverages the Amazon EMR infrastructure using Amazon EC2 Spot instances to gain access to compute functionality at reduced costs.
CI/CD & Testing
"Spot instances have been fantastic for our need at Basware. They enable us to run a world class CI infrastructure with hundreds of Windows based EC2 instances at a great price. Saving us upwards of 60% against the on demand cost."
Alistair Gilbert, Director of DevOps - Basware
Lyft is a San Francisco-based ridesharing company that is on Fortune magazine’s “Unicorn” list of hot startups, with a valuation of $5.5 billion. By using AWS Spot instances, Lyft saves up to 90% percent a month simply by changing four lines of code.
"We use Spot instances to run our Jenkins code deployments and production web server workloads. The new EC2 Spot Instances pricing model makes it even easier for us to acquire Spot capacity at predictable prices, giving us more confidence in the savings and stability we can get by moving additional workloads to Spot Instances."
Practo is a platform that helps India’s 8,000 doctors connect with three million patients across India. Get insights from Practo on scaling, cost reduction and log management when using containers using AWS services like EC2 Spot, Amazon SQS and third party software.
Israeli online news publisher, Walla, embraced transformation as they migrated to AWS. By leveraging ECS for containers with EC2 Spot Instances, along with a variety of AWS managed services, Walla is able to keep costs and operational overhead low.
Delivery Hero saves 70 percent on infrastructure for containerized Kubernetes workloads. The food delivery company transports 1 million food orders a week in 39 countries. Delivery Hero transitioned its Kubernetes clusters to run only on Amazon EC2 Spot Instances to take advantage of unused Amazon EC2 capacity at a discount.
YipitData is the on-demand, 100+ person alternative data team for hundreds of the world's largest corporations and investment funds.
As YipitData pioneers alternative data for investment research, they use Spot Instances to optimize costs for running web scraping and batch workloads. Since 2015, the company has run over 1,000 instances monthly on Spot, and recently they shifted almost all of their workloads to ECS running on Spot. ECS on Spot Instances helped their infrastructure become more robust and cost-efficient.
“YipitData has been able to save 70% on EC2 instances for the past years, allowing the company to expand faster” – Hugo Lopes Tavares, Staff Engineer
High Performance Computing
Australian Museum Research Institute
The Australian Museum Research Institute set out ot learn more about koala populations, genetics, and diseases. Amazon EC2 Spot gave them access to flexible, low-cost compute power which allowed the bioinformatics team to sequence the koala's genome.
CoreLogic is the largest provider of property information, analytics and property-related risk management services in Australia and New Zealand. CoreLogic replaced their EC2 fleet with spot instances and use a worker pattern to reduce their compute costs by 90%
DNAnexus provides a unified system of data management and sequence analysis for DNA sequencing centers and researchers. DNAnexus uses Amazon Elastic Compute Cloud (Amazon EC2) Spot Instances to conduct all of its DNA analysis, while Amazon EC2 On-Demand Instances handle the company’s interactive services, such as its client front end portal and visualization tools. DNAnexus also relies on Amazon Simple Storage Service (Amazon S3) to meet the company’s extensive storage demand, which will grow from terabytes into petabytes of data.
Fermilab is participating in the CERN project, the largest data-intensive project in the world. After proving the existence of the Higgs Boson (“God particle” which gives humans their mass), CERN can analyze even more data at a higher velocity and get closer to proving the existence of dark matter. They successfully increased computing capacity cost-effectively by a factor of 4, to a total of 58K cores. More than 500 Million events were fully simulated in nearly 10 days using 2.9 million jobs that normally would have taken Fermilab six weeks to complete without the help from AWS.
“We use EC2 Spot instances to perform high-intensity genomic sequencing that requires analyzing large data sets to discover millions of unique patterns to detect early-stage cancer. As our research needs vary week over week, we have spiky computing needs and Spot’s new pricing model gives us the reliability of savings to scale our research with low and predictable prices.”
The Guttman Lab at the California Institute of Technology (Caltech) chose EC2 Spot instances beacuse they needed a cost-effective, flexible, and elastic infrastructure solution for lab researchers to access large HPC clusters on the fly. Since they could launch and de-commission 1,000's of nodes on-demand, lab researchers could easily run multiple projects simultaneously, while reducing the time required to complete genomic sequencing from weeks to days.
"Using Spot provides us with the most cost-effective solution for the analysis of many terabytes of metagenomic data. We further benefit from the flexibility of Amazon Spot, as different analysis stages in the pipeline involve different workloads (data cleaning, DNA comparison, machine learning). Each task is best performed by different EC2 instance types, so in each stage we launch a different cluster. Clusters in large projects such as this typically involve several thousand instances that run for dozens of hours."
Elad Kehat, VP of R&D Software - BiomX
"We use EC2 Spot instances to perform high-intensity genomic sequencing enabling our customers to read and understand genetic variations. We are excited to move our additional genetic pipelines that run for more than 12 hours to Spot Instances as Spot’s new pricing model makes it even easier for us to acquire Spot capacity and gives us the reliability of savings at low and predictable prices."
Metabiota has driven significant cost savings by using Amazon EC2 Spot instances to run simulations at various times of the day. Metabiota has been able to save upwards of 60-70 percent of their compute costs by taking advantage of Amazon EC2 Spot instances.
"Using Amazon EC2 Spot instances has been a source of tremendous savings for us."
Mike Gahan, Senior Data Scientist - Metabiotic
Novartis built a platform for cancer research, estimating that it would take $40 million dollars. With Spot Instances, they conducted 39 years of computational chemistry in 9 hours for $4,232 and discovered 3 compounds to help fight cancer. Learn how from Novartis' Global Head of Scientific Computing.
"By using Amazon EC2 Spot instances, we saved $800,000 last year. Our customers benefit from that cost savings as well, because EC2 Spot instances give them the ability to be more flexible. Whether they need to generate images in milliseconds or perform complex chemistry operations taking many hours, they now have the cost flexibility they need."
Craig Bruce, Head of Infrastructure - OpenEye Scientific
Rock Flow Dynamics
Learn how Rock Flow Dynamics leverages Amazon EC2 Spot instances and Amazon S3 to create cost-effective, scalable clusters that power tNavigator, Rock Flow Dynamics' solution for running dynamic reservoir simulations.
Scribd estimates that it saved 63%, or $10,500, by using Spot Instances for the batch conversion instead of On-Demand instances for one particular job.
"We only had to write a couple of really small scripts. We were able to move from On-Demand to Spot instances in a couple of hours, coffee breaks included."
Jared Friedman, Co-founder - Scribd
Seattle-based aerospace engineering company, TLG Aerospace, saves 75% on STAR-CCM+ computational fluid dynamics (CFD) simulations. EC2 Spot instances have enabled TLG to access more memory and cores at a lower cost, allowing them to scale the number and size of increasingly demanding simulations.
University of California Santa Cruz
"Our collaborators are asking us for the data to be processed as quickly as possible, so they can analyze cancer samples against other cancer samples in the database. Using AWS, we can get the results to them in days instead of months, which could contribute to faster disease diagnoses."
Benedict Paten, Director - Computational Genominics Lab
"Storage technology is amazingly complex and we’re constantly pushing the limits of physics and engineering to deliver next-generation capacities and technical innovation. This successful collaboration with AWS shows the extreme scale, power and agility of cloud-based HPC to help us run complex simulations for future storage architecture analysis and materials science explorations. Using AWS to easily shrink simulation time from 20 days to 8 hours allows Western Digital R&D teams to explore new designs and innovations at a pace un-imaginable just a short time ago."
Steve Phillpott, CIO - Western Digital
EagleView re-architected its image-processing system to take advantage of Amazon EC2 Spot Instances, saving an average of 80% over On-Demand instances. EagleView uses aerial imagery combined with machine learning, computer vision, and data analytics to extract data and provide insights to customers in construction, emergency response, and many other fields. The company built a distributed, event-driven application that uses Amazon EC2 instances as compute resources, Amazon SQS to queue processing jobs, and AWS Lambda as an orchestration layer.
proteanTecs uses the AWS Cloud to run tens of millions of simulations in parallel, reduce costs by up to 60 percent, and concentrate on creating new products. The company, based in Israel, provides software and embedded sensors for predicting failures in electronic systems. proteanTecs runs its HPC workloads on AWS, relying on Amazon EC2 Spot Instances to lower costs.
Image & Media Rendering
Barnstorm VFX is a boutique visual effects house specializing in high quality digital effects, design, and production. They have been the primary visual effects studio for the Amazon Prime Original Series Man in the High Castle (seasons 2 and 3), and have also worked on CBS’ Strange Angel. Barnstorm started using AWS Thinkbox Deadline in 2014, and began rendering on the cloud with Amazon EC2 Spot instances in 2017.
"Currently we use Deadline to manage all of our in-house renders. We utilize Spot instances to be able to render projects our internal farm can't handle such as large scale 3D projects. Using AWS to scale our rendering pipeline has allowed us to complete heavy creative 3D projects like Man in the High Castle and Strange Angel. It has also allowed us to have a smoother iterative process. From an artistic standpoint artists are able to render up to 10x as many iterations when EC2 Spot is implemented."
Erik Nelson, Head of Technology - Barnstorm VFX
FuseFX is an award-winning visual effects (VFX) studio that specializes in content creation for television, film, commercials, games, and special venues. FuseFX uses Thinkbox Deadline and Amazon EC2 Spot Instances to render its scenes, enabling it to meet project deadlines. Thinkbox Deadline is a render management solution that integrates with the AWS Portal, enabling VFX studios to take advantage of the reduced rendering cost available via Amazon EC2 Spot Instances.
"Using Spot Instances gives us limitless capacity. We would not be able to make our delivery schedules without Deadline and Spot, and we no longer need to worry about having enough physical capacity for rendering. We can react on a moment’s notice to our daily rendering needs. This allows us to be agile and efficient."
Jason Fotter, CTO - FuseFX
Milk Visual Effects
Milk is a London-based visual effects (VFX) company with extensive television and film credits, having won an Academy Award for Best Visual Effects Ex Machina as well as multiple BAFTA awards for Doctor Who and other projects. Milk was also tasked with creating massive, and compute-intensive, ocean simulations and stormy seas for the feature film Adrift (released in 2018, starring Shailene Woodley).
"The scope of the VFX work for Adrift was easily 10 times bigger than anything we'd tackled previously, and it wasn’t the only project we were working on. By using Deadline on AWS, our modest-sized team was easily able to generate a ton of work. The seemingly limitless capacity of AWS Thinkbox Deadline and Amazon EC2 Spot Instances made for more fluid iteration and better results."
Dave Goodbourn, Head of Systems - Milk Visual Effects
Nexus Studios is an Oscar- and Emmy-nominated studio that specializes in animation, film, and interactive experiences. With locations in both London and Los Angeles, Nexus creates content that ranges from animated films to virtual reality. Nexus started using AWS Thinkbox Deadline and Amazon EC2 Spot Instances at the beginning of 2018.
"We needed to greatly increase our render capacity quickly and easily without up-front costs associated with purchasing hardware. We looked at other cloud providers, however the tight integration between Deadline and AWS made the choice easy. Using AWS allowed us to render jobs/projects that we otherwise would never have been able to render on our local render farm. We can now render basically any type of job, with any level of complexity needed."
Ryan Cawthorne, Systems Engineer - Nexus Studios
Passion Pictures is an award-winning production, animation, and commercial studio, having won an Academy Award for Best Documentary Feature in 2000 for their work on One Day In September. In 2017 they started using AWS Thinkbox Deadline and Amazon EC2 Spot instances for rendering and haven’t looked back. With offices in London, Barcelona, Paris, New York, and Melbourne, Passion Pictures continues to churn out critically-acclaimed work, including the Emmy Award-winning Netflix series Five Came Back.
"We started using Deadline mid-2017, and starting implementing the use of EC2 Spot in November 2017. We now use EC2 Spot for 90% of our Compute. AWS has helped us move towards a more OpEx business model, providing great flexibility with the machines available as well as the capacity. We are becoming a more agile business, and now we have the ability to grow without massive CapEx."
Jason Nicholas, Head of CG - Passion Pictures
Scripps Networks Interactive is a mass-media company specializing in factual and lifestyle television brands such as HGTV, DIY Network, and Food Network. Scripps Networks Interactive reduced its CGI render time by 95 percent using a solution based on AWS Thinkbox Deadline. The company is using the AWS Portal in Thinkbox Deadline to manage and administer Amazon EC2 Spot Instances as CGI render nodes.
AutoDesk is a software company that provides infrastructure for people who innovate. With 100 million people world-wide using AutoDesk, their customers rely on AutoDesk for compute intensive workloads to render 3D, realistic photos. AutoDesk turned to Spot to control costs and support the education community, so that their customers could save money for image rendering workloads, and even use these services for free.
Learn how social learning platform, Edmodo, optimized it's infrastructure costs with Amazon EC2 Spot instances and On-Demand instances.
Gett is an Israeli-based startup that connects people with taxi drivers and runs its website and mobile app on several hundred Amazon Elastic Compute Cloud (Amazon EC2) instances. Gett chose to reduce some of its costs by taking advantage of Amazon EC2 Spot Instances. The company runs the Amazon Elastic MapReduce (EMR) service on Amazon EC2 Spot Instances to help them process huge amounts of data.
Smadex is the mobile-first programmatic advertising platform of Entravision, an international media company specialized in omnichannel advertising solutions. As a Demand-side-platform, Smadex provides advertisers with access to a wide and high-quality media inventory and ad-exchanges. In order to bid efficiently, their Real-Time Bidding platform decides in less than 100 ms if they want to place a bid and the price of that bid.
"By using Amazon EC2 Spot Instances across multiple AWS regions we are able to handle billions of transactions for ads in less than 100 ms, dynamically scaling our infrastructure based on advertisers demand while saving consistently over 70% of our infrastructure costs. We also use Amazon EC2 Spot Instances to power a set of different Amazon EMR clusters used to process hundreds of TBs of data in real time allowing us to train our decision algorithms and providing our clients a high level of transparency based on our real time analytics dashboards."
Lucas Ceballos, CTO - Smadex
Dynamic AI56 has been using Amazon Web Services (AWS) infrastructure since 2015. Flexible hardware configurations of instances in Amazon Elastic Compute Cloud (Amazon EC2) enable the company to run research and system evolution workloads on-demand. A few ultra high-memory machines are used for coordination, data distribution, and preparation while actual workloads run on a dynamically allocated fleet of up to 400 Spot Instances across multiple US Regions. Dynamic AI56 diversifies Spot Instances across different instance types and different AWS Regions to align with the varying nature of the intense compute workload they run.
“Spot Instances provide the best balance of cost and performance to run what would otherwise be an expensive instance configurations with high CPU, GPU, RAM or all of the above which are required to train its AI/ML models. Dynamic AI56 was able to save 75% Instance cost by using Spot.”
- Ievgen Sliusar, Chief of Infrastructure
Keen Eye develops an image-centric platform for pathologists and translational scientists. Keen Eye needed to migrate its AI-ML platform from an outdated hosting platform to a Health Data certified cloud with performant GPUs, in order to scale up, increase the performance of their algorithms, while optimizing the infrastructure costs. The company starting running Kubernetes managed cluster EKS and Auto-Scaling groups to spin on and off EC2 Spot GPU instances used for the data model inference.
“By using ASG and EC2 Spot GPU instances to train and run our data models on our EKS Kubernetes Cluster, the total cost of our infrastructure was divided by 2.”
– Florian Grignon, Head of Infrastructure
Sinergise is a GIS company building large turn-key geospatial systems in the fields of cloud GIS, agriculture, and real-estate administration. In 2016 Sinergise established satellite imagery processing engine in the cloud, Sentinel Hub, which now powers several hundreds of earth observation applications around the world and processes more than quarter of a billion requests per month, crunching more than quadrillion satellite imagery pixels. The Sinergise team uses the machine learning on AWS to tackle the problem of cloud detection, the most crucial step during the pre-processing of optical satellite images.
“s2cloudless is a machine learning algorithm for computing cloud masks on Sentinel-2 imagery. It has become one of the state-of-the-art algorithms for cloud detection and it has been downloaded over 84,000 times and is used in dozens of Earth Observation applications. The scale and performance of Amazon EC2, coupled with petabytes of data staged for analysis in Amazon Simple Storage Service (Amazon S3), allowed us to scale our compute capacity to run our machine learning algorithms and churn through 13 million scenes at peak processing rate of 780 scenes per second. Overall, it took us only 9.5 hours to process 130 Bn km² of cloud masks. We use EC2 Spot whenever we can to keep our costs down. We were able to save up to 70% of on-demand costs. Spot instances automatically pause and resume our work around interruptions, so our applications can start right where they left off.”
– Grega Milcinski, CEO of Sinergise