AWS Case Study: Praekelt Foundation

The Foundation has recently migrated most of its core products and tools to Amazon Web Services (AWS). Young Africa Live (YAL), a portal to help people living with HIV/AIDS, is hosted on Amazon Elastic Compute Cloud (Amazon EC2). Since its launch on December 1, 2009 (World AIDS Day), the portal has grown to over 270,000 users. Simon de Haan, Chief Engineer for Praekelt Foundation, says, “We’re seeing over 70, 000 daily page views and over 2,000 comments placed each day. These are all users with very low-end mobile phones accessing the Internet via GPRS or EDGE data connections and only on the Vodacom mobile operator network.”

The architecture involves using Nginx, which proxies and load balances to a number of Django-based workers in the backend. The database is PostgreSQL, and the development team makes extensive use of Memcached for caching. De Haan adds, “We’re in the process of adding HAProxy to the mix. HAProxy allows for fairer load balancing, and we’ll keep Nginx to allow for secure socket layers (SSL). We’re investigating moving to PostgreSQL 9.x, primarily for its streaming replication capabilities.” The diagram below illustrates the architecture:

De Haan explains the decision to use AWS: “International bandwidth is very expensive in Africa, making local hosting impractical in many cases. For us, the benefit of using AWS is that we can scale up as demand requires it. ”

The target audience is in South Africa, but the Foundation does not have a hosting center there. So, network latency was a major concern at the beginning. However, after a number of tests, the team found they had the best network connectivity to Ireland, and are now hosting all of their Amazon EC2 instances there.

The Foundation is working on another project, called Vumi, a highly scalable conversation engine for the delivery of short message service (SMS), Star Menu Unstructured Supplementary Services Data (USSD), and chat messages to diverse audiences. De Haan explains: “Vumi is in use in TxtAlert, our patient appointment reminder system. With TxtAlert, we send out SMS reminders to patients to inform them of appointments. By simply sending these reminders, we’ve seen an appointment attendance increase of 27%.”

De Haan explains that Vumi is designed for elastic environments such as Amazon EC2. Vumi supports multiple conversation transports, SMS and USSD being used most frequently in Africa. The transports funnel all incoming messages to RabbitMQ and transmit all outgoing messages to the various gateways.

RabbitMQ routes the incoming messages to the appropriate workers using various metrics such the original transport, the sender, and the recipient. Each of the workers has a “conversation engine,” a python-based, domain-specific language that allows the team to script text-based conversations. The DSL is transport agnostic, so a single DSL can be used to manage conversations with a large and diverse audience across multiple transports. The diagram below illustrates the architecture:

“As a database, we’re currently using PostgreSQL, primarily because of our familiarity with it,” says de Haan. The team is investigating other options such as HBase and Cassandra.

De Haan notes, “Vumi is currently being used in the development of a Bill and Melinda Gates Foundation grant SANGONet project of which we’re a technical implementation partner.”

To learn more, visit http://www.praekeltfoundation.org/ .

Added April 21, 2011