Formed in 2009, WeTransfer is a technology start-up located in Amsterdam, the Netherlands. The 14-person company provides a free online service that enables users to transfer files that are too large to attach to a mail message. Customers can use the service to send files, up to 2 GB in total, to as many as 20 people at a time. WeTransfer stores the data on its servers for 14 days. The company also offers a fee-based premium service that allows larger file transfers and longer storage options. WeTransfer calculates that it has handled more than 370 million transfers since 2009 and estimates current service usage at approximately 15 million customers and 30 million transfers per month.
As a fast growing start-up, WeTransfer needed the ability to scale its infrastructure to meet customer demand. However, its original host provider wasn’t as flexible as the company needed it to be. “It would take the service provider almost three weeks to install a single server,” says Dave Forsey, Chief Technology Officer (CTO). “When we approached them about a plan to ensure future scalability, they needed twelve months advanced notice to update our environment. We had no room to grow because we only had a limited amount of rack space assigned to us.” After WeTransfer’s service went down a few times because of data center problems, the company began to look for alternatives.
WeTransfer chose Amazon Web Services (AWS) because the company wanted a fast, flexible solution. "The most attractive element of AWS is the ability to scale almost instantly," comments Forsey. "We knew that we could start new Amazon Elastic Compute Cloud (Amazon EC2) instances that would be ready to use in minutes.
WeTransfer built the file transfer service with Ruby on Rails and PHP. The company runs the service on Linux Amazon Machine Images (AMIs) and Amazon EC2 instances with Elastic Load Balancing. “We consider Amazon Simple Storage Service (Amazon S3), with ready-made shared storage, a huge benefit,” says Forsey. “We originally stored files on multiple servers. This system required an entire database just to track where everything was located, and files were frequently unavailable during routine maintenance. Using Amazon S3 to store files simplified the process and reduced our workload.”
Using Amazon Relational Database Service (Amazon RDS), WeTransfer provisioned a Multi-AZ DB Instance, which automatically created a primary DB Instance that acts as the main database and synchronously replicates the data to a DB Instance in a different Availability Zone. WeTransfer uses the read-replica DB Instance to run statistical queries. The company also took advantage of Amazon DynamoDB to rebuild its statistics system in a reliable NoSQL environment.
“We’re using Amazon Route 53 because the Domain Name System (DNS) web service lets us use the root domain, (e.g., we.tl) instead of forcing a subdomain such as www.we.tl,” says Forsey. “We plan to move all our domains to Route 53 later this year.” WeTransfer uses Amazon CloudWatch for monitoring and set up Amazon Simple Notification Service (Amazon SNS) to deliver updates and notifications for both AWS and the Nagios monitoring system. Figure 1 demonstrates WeTransfer’s environment.
WeTransfer estimates that its servers manage more than 8100 TB of data and the service transfers 8 files per second. Forsey says, “Our usage has almost doubled in the last six months, but thanks to AWS, scaling to meet demand isn’t an issue.” During very busy times, WeTransfer reports using over one hundred Amazon EC2 instances across both the EU and the US. “With Auto Scaling, we automatically increase capacity to over 60 file-processing worker instances during peak usage and usually scale back to around 10 instances during quiet times such as holidays and weekends. We’re also planning to implement Auto Scaling for our web services soon.”
“As a bootstrap start-up, we appreciate that AWS costs are based on demand rather than on a data center's flat fees,” says Forsey. “Additionally, we’re taking advantage of multiple AWS Availability Zones to protect our service from a single location failure. Problems may always happen—it's inevitable—but by ensuring that our service is in multiple locations, we know we will improve stability. We believe that using AWS will improve our uptime and overall reliability.”
To learn more about how AWS can help your web application needs, visit our Web, Mobile, and Social Apps details page: http://aws.amazon.com/web-mobile-social/.