AWS Case Study: LOUD3R

LOUD3R uses Amazon Web Services (AWS) extensively, supporting an architecture that includes everything from clusters of aggregation and semantic technology processes to front-end hosting and application programming interface (API) access. The LOUD3R team takes advantage of Amazon Elastic Compute Cloud (Amazon EC2) with Amazon Elastic Block Storage (Amazon EBS) boot volumes for their Linux/Apache/MySQL and PHP/Python front ends. They use Amazon Simple Storage Service (Amazon S3) and Amazon CloudFront for their content delivery network and storage, Amazon SimpleDB as a scalable solution for their API, Amazon Simple Queue Service (Amazon SQS) for task management, and Elastic Load Balancing (ELB) for distributing load across Apache instances.

LOUD3R lead engineer Billy McClure raves about AWS’ “incredible flexibility,” as the technology he supervises requires the processing and storage of large amounts of data at all times as well as the flexibility to increase processing power as LOUD3R’s client implementations scale. As McClure notes, “We are able to have nearly complete control over the architecture and can make changes when we need them…The ability to spawn instances for a few hours at a time to try out a new technology is invaluable for a small technology startup that needs to stay vigilant when it comes to resources spent on R&D.”

The LOUD3R API sits behind ELB and is hosted on machines distributed across the United States-East availability zone. LOUD3R stores large data sets in CouchDB, indexed by user ID and important attributes corresponding to the data in Amazon SimpleDB. The LOUD3R API runs on TurboGears2, with an in-house Amazon SimpleDB model that builds queries based on GET request parameters, and currently serves content to over 170,000 domains hosted by NameMedia. “AWS was able to provide us with scalability at an incredibly low price,” says McClure.

McClure is also grateful for the frequent releases of new AWS features that lessen the need for IT management, such as Amazon Relational Database Service (RDS), Amazon Simple Queue Service (SQS) and Amazon SimpleDB. “Our flexibility as a company grows even more,” says McClure. “We can divert resources we’d usually spend supporting software tools into our own algorithmic advancements.”

LOUD3R has an Amazon RDS migration planned in the future in addition to an integration of RabbitMQ and Celery to create a distributed task flow across a set of instances managed by Amazon Elastic Load Balancing and Amazon CloudWatch. “I’d personally like to add, as a developer,” says McClure, “that it was an unexpected and well-received action of Amazon Web Services, getting in contact with LOUD3R after eighteen months in which we never really contacted your team. In the last three weeks [AWS has] extended nothing but helpful ideas and desire to see us architecturally succeed using your service. I am beyond grateful for the time set aside to help us.”

To learn more about LOUD3R, visit http://www.loud3r.com/ .

(Published 2010)