Category: Success Stories

Pulse – Using Big Data Analytics to Drive Rich User Features

by Jeff Barr | on | in Amazon DynamoDB, Amazon EC2, Amazon Elastic MapReduce, Big Data, Case Studies, Customer Success, Success Stories | | Comments

Its always exciting to find out that an app that has changed how I consume news and blog content on my mobile devices is using AWS to power some of their most engaging features. Such is the case with Pulse, a visual news reading app for iPhone, iPad and Android. Pulse uses Amazon Elastic MapReduce, our hosted Hadoop product, to analyze data from over 11 million users and to deliver the best news stories from a variety of different content publishers. Born out of a Stanford launchpad class and awarded for its elegant design by Apple at WWDC 2011, the Pulse app blends a strong high-tech backend with great visual appeal to conquer the eyes of mobile news readers everywhere.

Pulse backend team members from left to right: Simon, Lili, Greg, Leonard

The December 2011 update included a new feature called Smart Dock, which uses Hadoop and a tool called mrjob, developed by Yelp, to analyze users reading preferences and continuously recommend other articles or sources they might enjoy.

To understand the level of engineering that goes behind such rich customer features, I spoke to Greg Bayer, Backend Engineering Lead at Pulse:

How big is the big data that Pulse analyzes every day? 

Our application relies on accurately analyzing client event logs (as opposed to web logs) to extract trends and enable other rich features for our users. To give you a sense of the scale at which we run these analyses, we literally go through millions of events per hour, which translates to as many as 250+ Amazon Elastic MapReduce nodes on any given day. Since we are dealing with event logs, generated by our users from the various platforms on which they access our app (Android, iPhone, iPad, etc.), our logs grow in proportion to our user base. For example, the recent influx of new users from Kindle Fire (Android) means we now have a lot more logs coming in from those devices.  Also, since the logs are big, weve found that it is very efficient to write them to disk as fast as possible – directly from devices to Amazon EC2 (see my tandem article on the logging architecture we use and the graph below, which highlights some of our numbers).

For more Pulse numbers, checkout the full infographic.

Powering Rich Features for Our Users

Much of our backend is built on industry standard systems such as Hadoop. The innovation happens in how we leverage these systems to create value. For us, its all about how we can make the app more fun to use and provide rich features that our users will love. For techies, you can read about many of these features in the backend section of the Pulse engineering blog and learn about all the details.

The Right Choice for Big Data

I joined the team here pretty early on as the first backend engineer. I came to Pulse after working at Sandia National Labs, where I built and managed an in-house 70-node Hadoop cluster. This was an investment of over $100,000, operational support, and over 6 months time to get it fully fine-tuned. Needless to say, I was fully aware of the cost and resources needed to run something at the scale that Pulse would need to accommodate.

AWS was and still is the only feasible solution for us. I love the flexibility to quickly stand up a cluster of hundreds of nodes and the added flexibility of choosing the pricing scheme thats needed for a job. If I need a job done faster, I can always spin up a very large cluster and get results in minutes, or take advantage of smaller instances and the spot marketplace for Amazon Elastic MapReduce if Im looking to complete a job thats not time-sensitive. Since an Amazon Elastic MapReduce cluster can simply be turned off when we are done, the cost to run big queries is usually quite reasonable. Consider a cluster of 100 m1.large machines: a set of queries that takes 45 minutes to run on this cluster could cost us approximately $11 – $34 (depending on whether we bid on spot instances or use regular on-demand instances).

Lessons Learned (the bold fomatting below is our doing :) )

It is important to consider the trade-offs and choose the right tool for the job. In our experience, AWS provides an exceptional capability to build systems as close to the metal as you like, while still avoiding the burden and inelasticity of owning your own hardware. It also provides some useful abstraction layers and services above the machine level.

By allowing virtual machines (Amazon EC2 instances) to be provisioned quickly and inexpensively, a small engineering team can stay more focused on the development of key product features. Since stopping and starting these instances is painless, its easy to quickly adapt to changing engineering or needs perhaps scaling up to support 10x more users or shutting down a feature after pivoting a business model.

AWS also provides many other useful services that help save engineering time. Many standard systems, such as load balancers or Hadoop clusters, that normally require significant time and specialized knowledge to deploy, can be deployed automatically on Amazon EC2 for almost no setup or maintenance cost.

Simple, but powerful services like Amazon S3 and the newly released Amazon DynamoDB make building complex features on AWS even easier. Because bandwidth is fast and free between all AWS services, plugging together several of these services is a great way to bootstrap a scalable infrastructure.

Thanks for your time, Greg & best of luck to the Pulse team! 


Related: Pulse Engineering – Scaling to 10M on AWS

IT-Lifeline – Disaster Recovery Using AWS

by Jeff Barr | on | in Amazon EC2, Amazon S3, Customer Success, Success Stories | | Comments

Matt Gerber,CEO of managed service provider IT-Lifeline spent some time talking about AWS on camera. Here’s the full video:

Matt enumerates the three main benefits that AWS provides him: cost, elasticity, and time to market (sometimes saving 6 months to a year for a new offering). He also talks about security and compliance, and about working with the AWS team. Matt notes that traditional disaster recovery (DR) is a capital-intensive business, and that they’ll be able to pass savings of 25-50% on to their customers. He says that the carrying cost for disaster recovery is effectively reduced to zero — “It doesn’t cost us anything to know that that AWS infrastructure is there for when our customers need it for a disaster.”

We’ve pulled together some additional DR and storage resources on our new Backup and Storage page.

— Jeff;

AWS Summer Startups: Discovr

by Jeff Barr | on | in Amazon EC2, Amazon Elastic Load Balancer, Amazon RDS, Amazon S3, Asia Pacific, AWS Startup Challenge, Case Studies, Customer Success, Mobile, Success Stories, Summer Startups | | Comments

Over the summer months, we’d like to share a few stories from startups around the world: what are they working on and how they are using the cloud to get things done. Today, we’re profiling Filter Squad from Perth, Australia!

Discovr team

In one of Werner Vogels‘ many travels through Australia this summer, he tweeted about a lean startup he had met, Filter Squad. Lean startups, not to be confused with bootstrapped startups, are built by adapting agile software development methodologies to business. Some of the concepts include building a minimum viable product, testing business assumptions with real market data, measuring results, and then quickly iterating or pivoting depending on what that data shows you. The concept of a “lean startup” was coined by entrepreneur & author Eric Ries

I reached out and spoke to Stuart Hall, CTO of Filter Squad, and creator of, among others, the Discovr Apps and Discovr Music for the iPhone & iPad.

Meet Filter Squad
Filter Squad is a startup focused on building apps that find what you like, according to CTO Stuart Hall. They began with a #1 selling iPad/iPhone app called Discovr Music in January 2011 and expanded the discovery product suite to include Discovr App in June 2011, which has been a #1 category application in 17 countries. As the name implies, Discovr Music makes it easy for users to find music they like based on their preferences, while Discovr App recommends apps the user might like based on the ones you’re already using. We have been extremely happy with AWS and we also plan to use it for our future products. We are big fans of products such as Amazon RDS and the Elastic Load Balancer to give a complete app scaling solution with Amazon EC2, says Stuart.  
Take a look at the Discovr Music app review from Fox News:

AWS & Lean Startups
Because we are a small, lean team, we were looking for a hosting solution that was going to be easy for us to setup, be reliable, and be easy to scale up and down throughout our product iterations. We looked at a large number of providers, but AWS stood out immediately for a number of reasons:
  • Low maintenance
  • Easy to scale
  • Simple to setup
  • Provided good redundancy
We couldn’t find anyone else who could match the AWS products and price. The number of other large, successful companies also using the service was very reassuring.
Building a Native iPhone/iPad App on AWS
Native mobile apps often need server-side components to create a rich user experience. For our Discovr Apps and Discover Music apps, we have used the following AWS products: 
  • Amazon EC2 – because we had no idea of the market reaction to the application when we launched, flexibility in adding and removing virtual servers based on demand was key. 
  • Amazon RDS – we needed a database that would also be easy to scale and be easy to maintain. Amazon RDS provided easy scaling, easy replication for slave instances, and a system where minor software updates are handled entirely by AWS.
  • Amazon S3 – S3 provides a great and cheap way to host static resources, one with which we had worked before and found ideal for our use case.
  • Amazon Elastic Load Balancer –  the load balancer is provided straight out of the box: it doesn’t require any installing and it needs very little configuration. The load balancer provides built in health checks and takes out instances that are not behaving. Elastic load balancing has been faultless since we launched.
  • Caching: the only thing missing was a caching solution, which AWS has since launched and we will be soon moving to. This was also a big consideration, the pace at which AWS are iterating and improving their service matches our philosophy to application development.
We are also big fans of New Relic for monitoring our AWS instance performance. 


Scaling up Ruby on Rails with AWS
We use Ruby on Rails server side, Objective C, and Java for client side. More details of our stack, including our architecture and test data, can be seen detailed on our blog

Words of Wisdom for Other Startups:
Understand that you can do it from anywhere, you dont have to be based in Silicon Valley, or even a big city. With the help of the internet and web services such as the AWS cloud, anyone can deliver great products from anywhere in the world.
For example were based in Perth, Australia. Its a five hour flight to Sydney and our hometown is most definitely not the tech capital of the world! To sum up:
  • Build a great product, then don’t forget to market it!
  • Treat your customers like precious gold.
  • Make it easy for your customers to talk to you and listen to what they say.
  • Cross-promote your app with other apps that youve also built.

We’ve collected some lessons learned on our blog: how we got 250k downloads in 4 days.


8 Days Left to Enter Your Startup in the AWS Start-up Challenge!
This year’s AWS Start-up Challenge is a worldwide competition with prizes at all levels, including up to $100,000 in cash, AWS credits, and more for the grand prize winner. Learn more and enter today!

You can also follow @AWSStartups on Twitter for startup-related updates.


Facebook Developer Update: Meet RootMusic, Funzio, and 50Cubes

by Jeff Barr | on | in Amazon CloudFront, Amazon EC2, Amazon Elastic Load Balancer, Amazon RDS, Amazon S3, Auto Scaling, Customer Success, Developer Tools, Games, Mobile, Music, Success Stories | | Comments

In honor of today’s Facebook Developer Conference, I’d like to recognize the success of our existing Facebook app developers and invite even more developers to kick-start their next Facebook app project with Amazon Web Services.

Quick Numbers
We crunched some numbers and found out that 70% of the 50 most popular Facebook apps leverage one or more AWS services. Many of their developers rely on AWS to provide them with compute, network, storage, database and messaging services on a pay-as-you-go basis. In addition to Zyngas popular FarmVille and CafeWorld, or games from Playfish and Wooga, many of the most exciting and popular Facebook apps are also running on AWS.

Here are a few examples:

RootMusic‘s BandPage app (currently the #1 Music App on Facebook, and #8 overall app on Facebook) helps bands and musicians build fan pages that will attract and hold the interest of an audience. RootMusic enables artists to tap into the passion their fans feel for their art and keep them engaged with an interactive experience. More than 250,000 bands of all shapes and sizes, from Rihanna and Arctic Monkeys, to bands you haven’t heard of yet but may soon discover, have already made RootMusics BandPage their central online space for connecting with their fans. Artists use it to share music, release special edition songs/albums here, share photos, and list events/shows. BandPage now supports 30 million monthly active users from all over the world. Behind all the capabilities that ignite BandPages music fan communities lies a well-thought out, highly-distributed and highly-scalable backend, powered by Amazon Web Services:

In 20 seconds, we can double our server capacity. In a high-growth environment like ours, it’s very important for us to trust that we have the best support to give to the music community around the world. Five years ago, we would have crashed and been down without knowing when we would be back. Now, because of Amazons continued innovation, we can provide the best technology and scale to serve music communities needs around the world, Christopher Tholen, RootMusic CTO.

Funzio‘s Crime City is #7 in the top 10 Facebook apps, and its the highest rated Facebook game to reach 1 million daily users with an average user rating of 4.9 out of 5. Crime City currently has 5.5 million monthly active users, with 10 million monthly active users at its peak. The iPhone version was recently listed among the top 5 games in the Apple Appstore and #1 free game in 11 countries and counting. Crime City sports modern, 3D-like graphics that look great on both Facebook and iPhone, and has a collection of hundreds of virtual items that players can collect.

Powering this incredibly rich user experience across multiple platforms is their business acumen in promoting the app, as well as a strong backend that leverages many AWS products to serve their viral and highly active user base. Funzio uses Amazon EC2 to quickly scale up and down based on demand, Amazon RDS to store game and current state information. They use Amazon CloudFront to optimize the delivery to a global, widely-distributed audience and to meet Facebook’s SSL certificate requirements.

At Funzio, we use AWS exclusively to host the infrastructure for our games. When developing social games, you need to be ready for that traffic burst for a hit game in a moment’s notice. AWS provides us with the flexibility to quickly and efficiently scale our applications at all layers, from increasing database capacity in RDS, to adding more application or caching servers within minutes in EC2. Amazon’s cloud services allow us to focus our efforts on developing quality games and not on worrying about managing our technology operations. – Ram Gudavalli, Funzio CTO.

50Cubes, the creator of Mall World, is a startup that has developed one of the most highly-regarded and longer-running successful female focused social game on Facebook. With over 5 million monthly active users, Mall World has a track record of being not only one of the first but also the top game of its kind for the past 1.5 years and continues to entice users world-wide.

50Cubes powers Mall World and other games they developed with a suite of AWS products. Out of these, they value the Amazon Auto-scaling and EBS features the most these products helps them effortlessly scale up and down their exclusive use of Amazon EC2 instances with user demand. Their database clusters are a mix of MySQL and other key value storage databases, all hosted and managed by the team on Amazon EC2 using EBS for Cloud Storage.

One thing that impresses me the most about AWS services is that they have rapidly iterated and improved their products and services over the past year and half, executing almost like a startup of our scale.” – Fred Jin, 50cubes CTO.

Get Started: Your Facebook App, Powered by AWS
Doug Purdy, Director of Developer Relations at Facebook, said:

AWS is great for Facebook developers you can start small, test and prove your ideas. As your app grows, you can easily scale up your resources to keep your users engaged and connected. AWS allows developers to build highly-available, highly-scalable, cost-efficient apps that provide the type of rich and responsive user experiences that our global audience has grown to expect.

To make it as easy as possible for you to get started, we’ve updated our Building Facebook Apps on AWS page. We have also improved and refreshed our Facebook App AMI. The new AMI uses AWS CloudFormation to install the latest versions of the Facebook PHP SDK and the AWS SDK for PHP at startup time. If you want to learn more about developing AWS applications in PHP, feel free to check out the free chapters of my AWS programming book (or buy a complete copy).

— Jeff;

AWS Summer Startups: Peritor/Scalarium

by Jeff Barr | on | in Amazon EC2, Architecture, AWS Startup Challenge, Case Studies, Customer Success, Europe, Germany, Success Stories, Summer Startups | | Comments

Over the summer months, we’d like to share a few stories from startups around the world: what are they working on and how they are using the cloud to get things done. Today Im speaking to Jonathan and Thomas, two of the creators of Scalarium, from Berlin, Germany! 

Peritor team

R: Hi guys, could you briefly describe Scalarium and the background of your team?

With Scalarium, we’ve created an easy management service for EC2 clusters. Scalarium helps our customers deploy Rails, Node.js, PHP, Java, Python or any other stack. It automates the initial setup and continuous configuration of servers. Scalarium also takes care of scaling, security, monitoring, and a lot more.
We started as an IT consultancy in 2005 and used EC2 from the early days on to help our clients scale out. Doing so, we realized that we repeated ourselves in this kind of projects. So we created Scalarium as a framework that helps customers automate EC2 deployments.


R: How have you incorporated Amazon Web Services as part of your own architecture? What services are you using and how?
We heavily use EC2, EBS and S3. And in our stack you will find Ruby, CouchDB, Redis, RabbitMQ, Chef and other nice and shiny stuff. We brought you a little illustration that shows you how we run Scalarium on EC2. But before that, you will need to understand a little more about what we do.
As said, Scalarium helps customers run apps on EC2. But instead of offering you some restrictive and expensive PaaS solution, we offer you an elegant way to automate everything on your servers. So you will still maintain root access to all servers and are able to configure each and every setting.


R: How does Scalarium help customers run apps on AWS?
In the cloud, each server goes through something that we would describe as a server life cycle. Each and every server in your cluster comes to existence at some time, it experiences some changes and it goes at some point later. Some of them have a rather short lifespan like application servers that are used to burst out, others have long lifespans like database servers. But all of them go trough this cycle.
We defined events in this life cycle which we and you can hook into to execute scripts on the servers. The life cycle events that are used in Scalarium are the following ones. 
  • Setup is used to update a base image and install everything you need on the fly as soon as the server comes into existence.
  • Configure is triggered by any change in the cluster – new servers coming or old ones going.
  • Deploy executes scripts that should run during the deployment of an application on the servers. You can hook into the deployment with before_migrate or any other hook you know from Capistrano.
  • Undeploy – this is triggered if you want to remove an application.
  • Shutdown is triggered if you gracefully stop a server. You can copy stuff around or inform other servers about the absence of the server in advance.
Now imagine a very basic setup with one load balancer, a couple of app servers and a single database.What would you need to do if you wanted to add another app server to your stack? (click image below to enlarge)
You would need to boot an AMI (Amazon Machine Image), log in to the machine, install updates and dependencies, configure all services, cron jobs and so on and last but not least deploy your application. But you are not done yet. You also need to log in to the database server and grant access to the new app server by adding the IP to your ACL. After that you have to log in to your load balancer and add the app server to the load cycle.
This procedure is rather tedious even for easy and basic setups like this, but as you can imagine, the number of dependencies and tasks grows very fast as soon as you have more tiers and servers in your cluster.
What would you do if one of your servers dies or isnt reachable due to some temporary network issues? Have a look at the Netflix Tech Blog and learn about the chaos monkey and his friends if you think your servers will be always on and flawless forever.
We created Scalarium to take care of this type of concerns automatically. You can extend the abilities of Scalarium as you like because you can react to all life cycle events and hook into them. This enables you to do just about everything. You always start with a vanilla OS and in the end you have a totally customized setup on your server and all other servers in the cluster know how to react and reconfigure themselves. We offer a broad selection of predefined stacks and examples. You can change them easily or add your own ones.


R: How does the bootstrapping of an instance work?
In this picture you see roughly what happens behind the curtains if a new server is added to a cluster (click image to enlarge):
As soon as a new server is requested, we ask Amazon for it. Once the server finished booting it downloads the Scalarium agent and a custom certificate, installs the agent and connects back to Scalarium in an encrypted and signed way. We check what kind of server you instructed it to be and execute the appropriate Chef recipes. Chef is an open-source system integration framework, similar to Puppet or CFEngine. Check out our example cookbooks on github to get a feeling about how easy it is to use Chef. You will find the main Scalarium cookbooks there too.
The server bootstraps and will be your new app server, database or whatever you wanted it to be. This process usually takes just one or two minutes depending on the stack you want to install and the size of the server.
After successful bootstrapping of a new server, all existing servers in the cluster get informed. This step is very important. Because now, recipes bound to the configure event are executed on each server in the cluster. That way, load balancers can execute recipes that ensure that they are aware of all running app servers and that they can safely remove stopped app servers from their load cycle. A database server can check if it has granted access to the available app servers. But of course you also could do advanced things like adding new database servers and re-balance your data, update your nagios alerting or your graylog2 server to catch all the logs you want.
If you are done with your basic setup you can easily add time or load based servers, add and deploy applications to your cluster or clone the complete environment to create a staging system. All that can be done via the UI or the Scalarium API.

R: How do you run on AWS yourself?

Below is a simplified visualization of our own architecture. We use two main databases for Scalarium. One is CouchDB, used to store information like the cluster configurations, server descriptions and current state, applications, deployment definitions. The other one is Redis, used for accounting, events, monitoring and metering data.

We chose CouchDB for high availability, easy replication, clustering, robustness, and a short recovery time. Redis is awesome for the very dynamic, fast growing, and non critical data we have.

Scalarium itself a Rails app, the Scalarium API is a Sinatra app. Workers are based on RabbitMQ/Nanite.

Our setup spans multiple regions and availability zones to guarantee a high uptime. CouchDBs awesome replication features are used to have a master/master replication across regions. Redis uses a master/slave setup for data replication. (click image below to enlarge)

R: Why did you decide to use AWS?

Thats simple. We use AWS because its the only big, global distributed and reliable source for IaaS out there. Amazon kicks some serious ass and develops tons of new features and services. Last but not least we eat our own dog food – Scalarium runs on Amazon and is managed with Scalarium.

By using AWS and Scalarium we can grow in no time to handle as many customers we like, spin up staging environments, deploy fast and often and do all that completely automated. All fail over, scaling, backup tasks, monitoring and so on is automated. You will love doing that. You can concentrate on developing your app without hassling with data centers and servers.

Amazon enables us to have clients ranging from start ups with one server, over SaaS offerings and agencies with a couple of servers, to the worlds biggest social game providers like wooga or Plinga with an incredible number of servers running their games all over the globe.

If you like you can see a rather old video in which Jonathan explains the complete process to create a Rails cluster, add a Rails app and deploy it. Or even better, sign up and try Scalarium for yourself.

R: Any last words?

 Yes. Take part in the Global AWS Start-Up Challenge! It is a short application form. You can win cash, AWS credits and get a lot of visibility. And if thats not enough we give every semi finalist half a year free Scalarium on top.

So apply for the Start-Up Challenge now!


AWS Summer Startups: ShowNearby

by Jeff Barr | on | in Amazon CloudFront, Amazon CloudWatch, Amazon EC2, Amazon RDS, Amazon S3, Amazon SQS, Asia Pacific, Auto Scaling, AWS Startup Challenge, Case Studies, Customer Success, Success Stories, Summer Startups | | Comments


Over the summer months, we’d like to share a few stories from startups around the world: what are they working on and how they are using the cloud to get things done. Today, we’re profiling ShowNearby, from Singapore!

ShowNearby team

About ShowNearby

ShowNearby is a leading location-based service in Singapore and an early adopter of the Android platform. Unlike many mobile apps out there, ShowNearby started with deployment on Android and then moved on to the iPhone by mid 2010 and Blackberry by fall of 2010. Today, the ShowNearby flagship app is available on Android, iPhone and Blackberry and reports approximately 100 Million mobile searches conducted across all its platforms.

I spoke to Stephen Bylo, Senior Cloud Architect at ShowNearby, who added a bit of color to the experience of running, planning, and meeting the requirements of a popular mobile app. If you’re not from Singapore and would like to see the app, here’s a quick video demo of ShowNearby.
Surviving Our Success with AWS

Due to the success of our application, we had a very big growth in a short period of time. When we launched on the popular platforms of iOS and subsequently BlackBerry, we were blown away by the huge surge of users that started using ShowNearby. In fact in December of 2010, ShowNearby became the top downloaded app in the App store, edging out thousands of other popular free apps in Singapore! It was then that we realized we needed a scalable solution to handle the increasing load and strain on our servers that our existing provider was unable to provide.

Our infrastructure at the time was hosted with a local service provider, but was unable to cope with the high traffic peaks we were facing.We analyzed a few vendors and decided to go ahead with Amazon because of it’s reliability, high availability, range of services and pricing, but mostly because of its solid customer support.

As part of our deployment, we added AWS services incrementally. Currently we use extensively Amazon EC2 instances with auto scaling, Relational Database Service (RDS), Simple Queue Service (SQS), Cloudwatch and Simple Storage Service (S3).

Next item on our list is to focus on automating the deployment of infrastructure environments with cloud formations, as well as optimizing content delivery globally with Cloudfront.

Choosing the Tech Stack That Makes Business Sense

ShowNearby currently leverages on the LAMP stack for most our web services. Delivery of accurate, always available, location based data is ShowNearbys top priority.That is why we chose AWS.

Other important things why to choose cloud/AWS: Speed and agility to create and tear down infrastructure as and when it is needed.  Good and fast network accessibility for our app.  Ability to scale up and out when needed.  Ability to duplicate infrastructure into new regions.

Reaching Automation Nirvana with AWS

We chose to use AWSs Linux based AMI and dynamically build on top of it using well defined, automatic configuration.  Now, every time an instance is started, we are sure the infrastructure is always in a known state.  Admittedly, a lot of hard work is involved to achieve Automation Nirvana, but knowing precisely what works at the end of the day helps us sleep at night.

  • We use Amazon S3 to store infrastructure configuration and user provided content/images. ShowNearbys business is currently in, and marching into new, regions, so S3 is a natural precursor to AWSs CloudFront content distribution service.
  • We use SQS to help process user behaviour and to determine usage patterns.  
  • We use this to provide our dear users with a better, and hopefully, more personalised experience.We use spot instances for early development & testing servers.
  • We use CloudWatch extensively – how could we do without it?
  • We use RDS, for our hosted mySQL databased needs, of course
  • We use the command-line and PHP AWS API tools to a large extent, which provides us increased business agility.

Words of Wisdom for Mobile Startups

We would tell them to find partners who can be good friends at the same time. The race is long and tough, so better do it enjoying every step of the way. There is a window of opportunity in Asia now open to unleash your full potential, show what you are capable of and you’ll be rewarded.

Today, if we need to refresh or update a web application, we restart new instances and flush out the old.  Moving forward, we are looking into reducing the time between releases still further and so, we are working to improve on our already solid infrastructure and configuration management.  Further automation in the form of Chef and/or Puppet or similar is being investigated.


Enter Your Startup in the AWS Start-up Challenge!
This year’s AWS Start-up Challenge is a worldwide competition with prizes at all levels, including up to $100,000 in cash, AWS credits, and more for the grand prize winner. 7 Finalists receive $10,000 in AWS credits and 5 regional semi-finalists receive $2,500 in AWS credits. All eligible entries receive $25 in AWS credits. Learn more and enter today!

You can also follow @AWSStartups on Twitter for updates.


AWS Summer Startups: Classle

by Jeff Barr | on | in Amazon CloudFront, Amazon CloudWatch, Amazon EC2, Amazon Elastic MapReduce, Amazon RDS, Amazon S3, Amazon SDB, Asia Pacific, AWS Startup Challenge, Case Studies, Customer Success, Success Stories, Summer Startups | | Comments

Over the summer months, we’d like to share a few stories from startups around the world: what are they working on and how they are using the cloud to get things done. Today, we’re profiling Classle, from Chennai, India!

Classle team

I recently read Mark Susters blog on Avoiding Monoculture – which is why Im happy to share with you what Ive learned about Classle, a  startup from India, focused on solving education problems for areas of the world that experience serious resource constraints. Classle has the big goal of changing the world around them by encouraging students and experts to share knowledge and expertise, and using the AWS cloud to facilitate this exchange. 

I reached out to Vaidya Nathan, Founder and CEO of Classle:

About Classle

Classle is a Social Learning infrastructure company with specific focus on Education, Learning and Knowledge communities. Using the main Classle product, Cloud Campus platform, Classle creates and manages private and public social learning environments and offers services based on it.

Classle helps rural students access higher education and reach opportunities unavailable before. Our company partners with a wide network of colleges throughout India, which act as internet-connected “learning nodes” that distribute educational materials to students. When the student goes home for the day with their downloaded lectures and other materials from the library, Classle makes use of mobile technology and SMS-based quizzes to keep students engaged and actively learning. The entire system was designed to work with simple, $10 phones, not smartphones, and the students are entirely addicted to these quizzes – they cant get enough of them.
All these services are provided free of charge to both students and colleges. Classle monetizes by partnering with companies who are looking to hire top talent from among the students, and by selling their cloud-based learning platform for training purposes within companies.

Starting Small and Growing with Business

We are using AWS since our inception in early 2009. Our first steps involved two small Amazon EC2 instances and Amazon EBS to store our database. Over the years, our use has expanded to match our business growth. Our selection criteria covered tactical as well as strategic points. From a tactical perspective, we wanted a quicker time for provisioning, which AWS on-demand instances enabled, and the option to secure our resource needs through Reserved Instances.

At a strategic level, we wanted to provide the best experience for our customers and it was key to build Classle on top of services, products, and infrastructure designed for growth and scale. To date, we have established relationships with over 30 educational organizations and that list is constantly growing. Thanks to AWS, we are effectively competing with some large and strong players in the e-learning space.

Sharing the AWS Lessons:

We are a small, LAMP stack team and we started using AWS in 2009. Currently, the products we are using are below. For reference, we are also happy to share our Classle architecture diagram, which is included in our case study with AWS.

  • Amazon Elastic Compute Cloud (EC2)
  • Elastic Load Balancing (ELB)
  • AutoscalingAmazon Elastic Bock Storage (EBS)
  • Amazon Simple Storage Service (S3)
  • Amazon Reduced Redundancy Storage (RRS)
  • Amazon CloudFront with both streaming and download
  • Amazon Cloud Watch
  • Amazon Relational Database Service (Amazon RDS) with Multi-AZ and Read replication.
  • Amazon SimpleDB
  • Amazon Simple Notification Service (Amazon SNS)
  • Amazon Route 53

Pretty soon, we would be using Amazon Elastic Map-Reduce clusters for our analytics requirements..

Words of Wisdom to Startups

Starting a company is always hard, whether youre from India or anywhere else. However, its worth to keep in mind that its never been easier to go out there and try things out –  with Open Source for robust software and cloud service providers like AWS for infrastructure, you can test your ideas and run a business at very low cost.

Being from in India, where we dont have a strong start-up mentality like in the U.S., certainly poses some unique challenges. There are many more problems to solve, and it is exciting to try and translate the existing limitations into innovations, solutions and hence opportunities.

If I had to boil down my advice, I would say to my fellow entrepreneurs to: venture with confidence, design for scale, start small & architect for growth.


Enter Your Startup in the AWS Start-up Challenge!
This year’s AWS Start-up Challenge is a worldwide competition with prizes at all levels, including up to $100,000 in cash, AWS credits, and more for the grand prize winner. Learn more and enter today!

You can also follow @AWSStartups on Twitter for updates.


AWS Summer Startups: TellApart

by Jeff Barr | on | in Amazon EC2, Case Studies, Customer Success, Success Stories, Summer Startups | | Comments

Over the summer months, we’d like to share a few stories from startups around the world: what are they working on and how they are using the cloud to get things done. Today, we’re profiling one of last year’s finalists from the AWS Start-up ChallengeTellApart, from Burlingame, California!

TellAparts application stood out to all AWS reviewers as a very well written entry to the AWS Start-up Challenge – pun intended, it was indeed easy from the beginning to tell it apart from the thousands that we reviewed.  As we met the team at our final event and got to know them more over the past year, we grew to be as excited as they are about the future of advertising and real-time ad bidding systems on AWS. We took some time to catch up with Josh McFarland, CEO of TellApart:

About TellApart

TellApart helps online retailers identify (or tell apart) their best customers and prospects. They do so by employing a suite of marketing tools – a customer data platform, predictive customer analytics, and a next-generation display ad retargeting application. TellApart believes it is the first company to bring all of these components together in a cloud-based platform. Theyve also introduced a fresh, innovative business model to the markettheir customers, typically online retailers, pay TellApart only when shoppers click through on TellApart-served display ads and actually make a purchase.

Real-Time Ad Bidding System, At Scale

TellAparts vision revolves around helping our e-commerce clients tell apart their highest value shoppers from the rest. We assess each of our clients customers and then use our predictions to place personalized display ads in front of the best visitors to drive new sales.

We do this in real time… and thats not hyperbole; we must respond to bid requests from the Google/DoubleClick Ad Exchange in under 120ms. And we handle more than 10,000 of these requests per second today, making TellApart one of the top five companies in this space. These capabilities require big data and serious compute power — something that, as an ex-Google founding & early engineering team, were used to having on demand. AWS makes this possible at scale, as we outlined in our recent case study.

No Technical Debt Burden

Some of you might say that data warehousing, predictive analytics, and customized marketing solutions are not new concepts at all – and you would be right. Many organizations provide these services  today – and many of them have extensive data-centers that support those computational needs. What makes TellApart interesting is that their technology choices and cloud-based implementation make their business decision process happen in a much more agile and easy way, while keeping costs low. 

When youre growing a customer base and product usage as quickly as we are, you quickly learn that scalability is the first order of business. Getting that right frees up engineering for innovation instead of maintenance. AWS has allowed TellApart to quickly build an infrastructure that is elastically scalable, redundant, fast and cost efficient – something that was not possible just a few years ago.

On Growing, AWS Monthly Bill, and Impact of AWS Start-up Challenge

Since being honored as an AWS Startup Challenge finalist in December of last year, TellApart has continued on an insane trajectory, more than doubling our client list, revenue and headcount! About the only thing that hasnt grown by at least 2x is our AWS bill. Weve made smart use of the recently announced EC2 Spot Instances for our less urgent Hadoop-based data processing jobs, and weve implemented an architecture based on EC2 Reserved Instances to reduce costs for our more predictable front-end ad serving. Innovations like these pricing tiers are awesome, and they show the AWS team is responding to our needs. PS — Were hiring! (For all positions except NOC Admin…)

Words of Wisdom to Entrepreneurs

Our advice for aspiring entrepreneurs is borrowed from the Greek goddess of victory: Just do it! Youll be surprised at how much you can accomplish quickly as you work toward finding product-market fit. We recently completed a $13M Series B funding round, and the venture capitalists we spoke with during the process were astounded at how much we had accomplished in our first two years — and how little we spent to do it! It goes without saying that AWS continues to be a pillar of our success and efficiency.


Enter Your Startup in the AWS Start-up Challenge!
This year’s AWS Start-up Challenge is a worldwide competition with prizes at all levels, including up to $100,000 in cash, AWS credits, and more for the grand prize winner. Learn more and enter today!

You can also follow @AWSStartups on Twitter for updates.


AWS Summer Startups: Appscend

by Jeff Barr | on | in Case Studies, Customer Success, Europe, Mobile, Success Stories, Summer Startups | | Comments

Over the summer months, we’d like to share a few stories from startups around the world: what are they working on and how they are using the cloud to get things done. Today, we’re profiling Appscend, from Bucharest, Romania!

Appscend is a sign that catching the “startup bug” and being able to run a high-tech company is possible waay outside Silicon Valley, in Romania, where startups don’t have the benefit of tapping into a support system and extensive mentorship. I spoke to Gabi Dobocan, Appscend Cofounder, who was quick to tell me that his small team enjoys the challenge and is eager to work at least 10 times as hard as their counterparts in other countries in order to compete. To date, they have received support from local angel investor and mentor, Peter Barta, and from an extended entrepreneur network from Western Europe. We wish Gabi and his team the best of luck :)

About Appscend

Appscend is “ WordPress for mobile” software. Since day one, the vision for Appscend was to create a robust, highly customizable product that can produce feature-rich mobile apps in record time. Today, a few months after launch, Appscend has enlisted over 400 registered users working on about 300 mobile apps. Among these customers, Appscend works with Domo, one of the largest local retailers, the Romanian National Television, and a couple of important local publishers. Appscend currently supports iOS apps and their development team is working on releasing an Android version in the next few months.
“100% AWSomeness”

Appscend offers a series of products geared at enabling our customers to build rich mobile apps apps using an intuitive, customizable interface. Our target audience is made up of digital publishers and agencies, so our products need to be easy to use and offer a wide array of functionality: we offer a mobile CMS, accessible for anyone that doesn’t have a way to host their content elsewhere and have made it easy for users to import and export content. We help customers who build their mobile apps through Appscend track social activity around their content and we also offer them the ability to implement in-app purchases. In addition, to help our customers further monetize their mobile apps, we have enabled them to display ads inside their apps through an ad management server.

To build all these products, we use most of the cloud web services provided by Amazon Web Services and are delighted to call our implementation 100% AWSomeness.

  • Amazon S3 buckets are used for storing all the multimedia content used in the apps, optimized for mobile consumption.
  • The S3 buckets are connected to a Amazon CloudFront distribution to insure minimum latency wherever our clients’ end-users are located.
  • Our customer dashboard, as well as all background processes, are hosted on Amazon EC2 instances.
  • The Amazon EC2 instances are placed into Elastic Load Balancers, so that we can enable auto-scaling through Amazon CloudWatch when needed.
  • Our ad server is based on Google Adwhirl and is hosted on an Amazon EC2 instance.
  • The software connects to Amazon SimpleDB domains that hold information regarding to the apps and statistics.
  • Everything is powered by a MySQL 5 Enterprise server hosted on an Amazon RDS instance. We find it a good solution because we can scale very easily and the performance provided is excellent.

Finally, we keep in touch with our clients by sending email messages through Amazon SES, and we’ve also configured email alerts for our operators using the same service. In the future, we will look at implementing AWS CloudFormation in order to be able to provision resources as quickly as possible.

On Building a Startup in Romania and the Mobile Market 

Right now, we’re living our dream and working on our passion. It’s really what makes us happy, what makes us grow, and at the end of the (12 hour working) day, that’s all that matters. We survive, we take nothing for granted, and we do good with limited resources.

The mobile market is really still a young, fragmented market and we work hard to always be alert, listen and learn as much as we can. We’re probably gonna change a lot of this product in the next year, and we welcome that :) Challenging your own beliefs and being flexible and open-minded is the only real way to evolve I think – as a company and as a person.


Enter the AWS Start-up Challenge!
Join Appscend and many other AWS startups who are entering this year’s worldwide AWS Start-up Challenge! You can also follow @AWSStartups on Twitter for updates.


AWS Summer Startups: Yottaa

by Jeff Barr | on | in Case Studies, Customer Success, Success Stories, Summer Startups | | Comments

Over the summer months, we’d like to share a few stories from startups around the world: what are they working on and how they are using the cloud to get things done. Today, we’re profiling Yottaa from Cambridge, MA!

I first met Coach Wei, CEO and founder of Yottaa, last year, as a finalist in the AWS Start-up Challenge.  With a name like Coach and a deep passion for speed and web performance, he sounded like he was the perfect guy to run a startup like Yottaa. 

What is Yottaa?
Yottaa prides itself in bringing fast web performance to any website. The start-up provides software-as-a-service tools for website performance monitoring (free) and automated performance optimization (paid).

Anybody can visit, enter a URL and learn about its performance and user experience over time, and across multiple geographies. The Yottaa free monitoring tool, Yottaa Insight, also enables users to link their Google Analytics account to provide deeper insight into the correlation between site performance and business objectives. Their paid product, Yottaa Optimizer, is a web performance optimization service that aims to double the speed of your website within a few clicks, without code changes, software installs or hardware purchases. 

Q&A with Coach Wei:

Creating a Free Product Offering

AWS is a big reason that we are able to provide Yottaa Insight, a premium web performance monitoring service, for free. We run hundreds of machines at any single moment, but the system dynamically scales up and down at many different geographic locations in response to the workload demands.

Growing and Solving the Data Management Problem with AWS 

The amount of performance monitoring data we collect for our users have grown over many terabytes. How to manage the large amount of data is a common challenge for startups with data-centric web applications.  Like most web startups, we started by storing all the data in our database initially. It quickly outgrew what a database can handle. Luckily we have quite a few options with AWS. One of the recent actions we took is to move most of the historical data onto Amazon S3 and only keep the recent data in our database cluster. This design essentially makes our data storage almost infinitely scalable. Another action that we took is to implement data replication to another geographic location, which gives us redundancy and disaster recovery protections. 

AWS Start-up Challenge Impact on Yottaa: Confidence & Validation

In our particular situation, the biggest difference AWS contest made for us is “confidence” or “bragging rights” : it gives us a certain level of confidence that we are onto something.  Yottaa is taking a new approach to solve some difficult problems by leveraging the AWS cloud.  
A large part of the unknowns came from whether it was feasible to do wire-speed optimization of web pages, both technically and financially. To our knowledge, no one had solved those issues before.  After the contest (and in particular, after talking to the AWS Startup Contest judges, the folks who helped created the entire cloud computing movement), we can confidently say that we were not crazy to attempt this. :-)

Sharing the Wisdom with other Asia-Pacific Start-ups:

SaaS and Cloud start-ups are blazing new trails with innovative approaches, architectures, and delivery models.  Amazon Web Services provides a fantastic foundation to build on.  Our advice for start-ups is to embrace the cloud and design your applications architecture to leverage multiple AWS services. For example, Yottaa architected the system to leverage Spot Instances and multiple AWS regions and Availability Zones.  Decisions like these create both a robust platform and the ability to scale infrastructure and costs as your startup begins to get traction. 

Another quick thought is the “audacity of dreaming”.  Dream big.  Cloud provides a cost-effective way to figure out the validity and viability of your dream..

The AWS Startup Challenge
We’re getting ready to launch this year’s edition of our own annual contest, the AWS Startup Challenge. You can sign up to get notified when we launch it, or you can follow @AWSStartups on Twitter.