Introduction to Software Load Balancing with Amazon EC2

By Sean-Philip Oriyano, computer security consultant and PJ Cabrera, freelance software developer and writer

The purpose of this article is to discuss load balancing and how to implement and configure the technology in the Amazon Web Services™ (AWS) Amazon Elastic Compute Cloud (EC2) Beta environment. We will examine the purpose of load balancing and some of the features that it provides, and we will take a closer look at two practical examples of load balancing: DNS round robin and HA Proxy.

The purpose of this article is to discuss how load balancing can work in the context of Amazon EC2. Although many applications and technologies can be implemented in the Amazon EC2 environment, we will take a closer look at two of the most requested implementions: round robin DNS and HA Proxy. We will also discuss some of the other options that can be used in the proposed environment.

Before we delve further into how to implement and configure load balancing, let us take a quick look at what exactly load balancing is and what it accomplishes.

In the purest sense, load balancing is any technique that distributes a load among multiple entities such as CPUs, disk drives, servers, or any type of device. The goal of load balancing is primarily to obtain much greater utilization of available resources than can be obtained from a group of standalone objects (such as servers). In practice, load balancing can be provided through the use of hardware or software.

Load balancing can be provided through specialized devices, such as a multilayer switch that can route packets within a cluster. Typically, implementing, configuring, and maintaining hardware-based solutions require costly investments in time and capital. Hardware load balancing is generally considered impractical for hosted environments and is mentioned here only in the interest of completeness.

Software load balancing is provided in one of two ways: as a part of an operating system or as an add-on application. Load balancing that is implemented as a software-based solution can often offer ease of deployment and maintenance as well as performance similar to that of hardware-based solutions. Some common implementations of software-based load balancing include those bundled as part of Microsoft Windows or Linux, and add-ons such as HA Proxy, which is discussed later in this article.

In this article, we will discuss only the software-based aspect of load balancing.

Load balancing offers many benefits to the hosted application. When considering a load-balancing solution and its suitability for your situation, consider the following benefits:

• Asymmetric load balancing. You can use this technique to disproportionately distribute loads across servers, generally to allocate different loads to servers with differing amounts of resources (e.g., CPU, memory).
• Priority activation. Priority activation occurs when a predefined number of available servers drops below a given threshold. Typically, this threshold is determined by load or the availability of servers to pick up increased load. In this scenario, you can allocate additional standby servers to handle a given need.
• SSL acceleration and distribution. Applications that make heavy use of Secure Socket Layer (SSL) can offload SSL-associated loads to servers that are better suited to handling an increased load. The offloading of SSL to different servers can increase overall performance of the group.
• TCP buffering. The load balancer can buffer responses from the server and spoon-feed the data to slow clients, allowing the server to move on to other tasks.
• Content filtering. Some load-balancing packages can modify traffic as it passes through the package.
• HTTP security. Some load-balancing packages can sanitize traffic and can employ other measures to obscure the clients behind them.
• Priority queuing. This feature provides the ability to prioritize traffic, based on need.
• Content-aware switching. You can use this advanced feature to direct traffic to a specific server, based on the URL being requested.
• Firewall. For security reasons, direct connections to backend servers are prevented.
  

Note: Some of these features and benefits are specific to certain technologies or applications.

Although several services can be hosted in a load-balanced environment, not all may benefit from the configuration. The following points should be considered when assessing services to be hosted with load balancing:

• Load balancing should not be viewed as a replacement for fault-tolerant architectures. When load balancing is implemented, a server that fails will remain offline and, in the event of round robin DNS, will still be sent requests.
• Applications that require a persistent connection to a specific database or backend server are generally incompatible by default. In certain situations, you might be able to put persistent connections in place outside of load balancing, but this option should be considered prior to implementing load balancing.
• Additional connections to a server cannot be guaranteed without additional work outside of load balancing.

Using a technique known as round robin, you can perform a basic form of load balancing. Let's take a closer look at this technique and how to use it.

Round robin functions simply by providing a list of IP addresses instead of providing the traditional single IP used in standard DNS implementations. The server responds to requests by providing the first IP address on the list to the first requestor, the second IP address to the next requestor, and so on until the last IP is handed out. The server then repeats the process from the beginning. In this setup, each server to which traffic is redirected presumably has the same content (i.e., hosts the same website).

Round robin is ideally suited for setups in which servers are located in geographically diverse areas or in which content is located in multiple data centers, servers, or availability zones.

DNS deployed in this configuration offers several advantages and disadvantages:

• Pros
• Relatively simple configuration
• Effective distribution of traffic among multiple servers
• Cons
• No mechanism to deal with unavailable servers
• No means of ensuring that subsequent requests will be fulfilled by the same server
  
• Each server will require public IP addresses

The process for configuring round robin is fairly simple, and the steps are generally the same for each DNS server.

In the following example, we assume that several web servers host an organization's website. For this scenario, we don't care where the servers are located; we just want to ensure that each can respond to requests as necessary.

To configure the setup, we will add multiple A records that collectively have the same name but that each have a different IP address. The following is a snippet of the zone file:

www IN A 192.168.0.1 
www IN A 192.168.0.2 
www IN A 192.168.0.3 
www IN A 192.168.0.7 
www IN A 192.168.0.8 

Alternatively, you could use the following format, which will yield the same result:

www 	IN A 192.168.0.1 
	IN A 192.168.0.2 
	IN A 192.168.0.3 
	IN A 192.168.0.4 
	IN A 192.168.0.5 
	IN A 192.168.0.6 

In both scenarios, the web server will respond to requests by handing out the next IP address in the list to the requesting client.

By default, DNS does not offer any control over how responses are serviced. Specifically, DNS does not control the order in which it returns IP addresses. However in some implementations of BIND DNS, you can set the order of the responses to be returned, based on settings that you can define through the rrset-order option.

In the Amazon EC2 environment, you can more effectively manage the order or round robin requests through the use of Elastic IPs. By using Elastic IPs, you can remap the IP address from a failed instance to another working instance. You have two avenues to complete this task: programmatically or by directly altering the DNS entries on the server. Both methods are presented here.

Note:A full discussion of Elastic IPs is outside the scope of this document. For more informatin about the Elastic IP feature, please refer to this guide.

HA Proxy is another technology that you can use with load balancing. HA Proxy is a software solution designed to provide proxying for both TCP- and HTTP-based applications. In the following example, you will install HA Proxy into the Amazon EC2 environment and perform some initial configuration to get the solution running.

The first step to implementing HA Proxy on Amazon EC2 is to customize an Amazon Machine Image (AMI). In this article, we will use ami-6a57b203, the most recent version (as of this writing) of Eric Hammond's Ubuntu 8.04 Hardy Heron images, listed at alestic.com. To launch this AMI, open a command-line window. Set up a command-line environment by using the EC2_PRIVATE_KEY and EC2_CERT environment variables, then issue the command ec2-run-instances ami-6a57b203 -k my-keypair, where my-keypair is the name of the keypair you were issued to log in remotely to public Amazon EC2 AMIs.

pjcabrera@dopey:~ $ export EC2_PRIVATE_KEY=~/.ec2/pk-my_aws_credentials.pem 
pjcabrera@dopey:~ $ export EC2_CERT=~/.ec2/cert-my_aws_credentials.pem 
pjcabrera@dopey:~ $ ec2-run-instances ami-6a57b203 -k my-keypair 

RESERVATION r-8913dee0 my_aws_userid default
INSTANCE i-57b8693e ami-6a57b203 ec2-72-44-48-213.compute-1.amazonaws.com 
domU-12-31-39-00-A5-E8.compute-1.internal pending my-keypair 0 m1.small 2008-06-28T19:17:30+0000

A couple of minutes after the ec2-run-instances command finishes, issue the ec2-describe-instances command to find out whether the instance has finished booting.

pjcabrera@dopey:~ $ ec2-describe-instances 

RESERVATION r-8913dee0 my_aws_userid default
INSTANCE i-57b8693e ami-6a57b203 ec2-72-44-48-213.compute-1.amazonaws.com 
domU-12-31-39-00-A5-E8.compute-1.internal running my-keypair 0 m1.small 2008-06-28T19:17:30+0000

Note: If you prefer GUIs over command-line tools, you can use the fantastic Firefox extension Elasticfox to launch and manage your instances.

Once the AMI instance is running, log in and start installing HA Proxy. Modify the following command with the path to your Amazon EC2 keypair and the domain name of your running instance:

pjcabrera@dopey:~ $ ssh -i ~/.ec2/id_rsa-my-keypair root@ec2-72-44-48-213.compute-1.amazonaws.com

Install the Screen package first because it lets you reconnect to your command-line session on the instance, even if the connection is interrupted. This package isn't installed in Ubuntu by default.

root@domU-12-31-39-00-A5-E8:~# apt-get install screen 
root@domU-12-31-39-00-A5-E8:~# screen

Next, upgrade Ubuntu to the most recent packages.

root@domU-12-31-39-00-A5-E8:~# apt-get update && apt-get upgrade

Now you are ready to install HA Proxy. The most recent version (as of this writing) for Ubuntu Hardy is 1.3.12.

root@domU-12-31-39-00-A5-E8:~# apt-get -y install haproxy

We like to set up HA Proxy to use syslog to log to /var/log/haproxy.log.

root@domU-12-31-39-00-A5-E8:~# sed -i 's/SYSLOGD=\"\"/SYSLOGD=\"-r\"/g' /etc/default/syslogd
root@domU-12-31-39-00-A5-E8:~# echo 'local0.* /var/log/haproxy.log' >> /etc/syslog.conf && /etc/init.d/sysklogd restart

Note: If you choose not to use this command, HA Proxy will send its messages to /var/log/messages.

In Ubuntu, HA Proxy does not start up automatically by default, so you need to change that rule.

root@domU-12-31-39-00-A5-E8:~# sed -i 's/ENABLED=0/ENABLED=1/g' /etc/default/haproxy 

And now, you can start HA Proxy for the first time.

root@domU-12-31-39-00-A5-E8:~# /etc/init.d/haproxy start 

Finally, you install the Heartbeat package, which lets you monitor how the system is running. This package lets you set up rules to restart Apache services if they stop responding to monitoring or if resource use is too high (to manage memory or file handle leaks, for example.)

root@domU-12-31-39-00-A5-E8:~# apt-get -y install heartbeat-2 

Now that you have installed HA Proxy and Heartbeat, you can create your own AMI. Doing so will let you launch instances with HA Proxy already installed and waiting for the specific configuration for your application or solution.

First, you need to upload your Amazon EC2 private key and certificate to your current running instance so that you can sign your AMI bundle, or Amazon EC2 won't be able to decrypt your AMI. Run the following command from your workstation, changing the filenames as appropriate for your Amazon EC2 private key and certificate and the domain name of your running instance:

pjcabrera@dopey:~ $ scp -i ~/.ec2/id_rsa-my-keypair ~/.ec2/pk-my_aws_credentials.pem \
~/.ec2/cert-my_aws_credentials.pem root@ec2-72-44-48-213.compute-1.amazonaws.com:/mnt/ 

Make sure that you upload these files to the /mnt directory. Later, when you create your AMI bundle, you will tell the AMI tools to ignore this folder, thus keeping your private key and certificate private.

Returning to the running Amazon EC2 instance, issue the following command to begin creating your tailored AMI. Replace the -k, -c, and -u parameters with your Amazon EC2 private key file, certificate file, and AWS user ID, respectively.

root@domU-12-31-39-00-A5-E8:~# ec2-bundle-vol -d /mnt -k /mnt/pk-my_aws_credentials.pem \
-c /mnt/cert-my_aws_credentials.pem -u my_aws_userid -r i386 -p haproxy

Once the AMI bundle gets created, we can upload it to Amazon S3 for storage. Replace the -b, -a, and -s parameters with your S3 bucket name, your AWS access key and AWS secret key, respectively.

root@domU-12-31-39-00-A5-E8:~# ec2-upload-bundle -b my_s3_bucket -m /mnt/haproxy.manifest.xml \
-a my_aws_access_key_id -s my_aws_secret_access_key

When you upload your bundle, you can disconnect from your running instance and terminate it.

root@domU-12-31-39-00-A5-E8:~# logout 
pjcabrera@dopey:~ $ ec2-terminate-instances i-57b8693e
pjcabrera@dopey:~ $ ec2-register my_s3_bucket/haproxy.manifest.xml
IMAGE ami-02bf5b6b 

This AMI is the identifier of the image that you created with HA Proxy preinstalled. Feel free to use it in your HA Proxy projects.

To continue setup, launch two instances of your AMI, logging in and configuring HA Proxy.

pjcabrera@dopey:~ $ ec2-run-instances ami-02bf5b6b -k my-keypair -n 2
RESERVATION r-8913dee1 my_aws_userid default
INSTANCE i-57b8693f ami-02bf5b6b ec2-72-44-48-215.compute-1.amazonaws.com
domU-12-31-39-00-A5-E9.compute-1.internal running my-keypair 0 m1.small 2008-06-29T21:47:40+0000 
RESERVATION r-8913dee2 my_aws_userid default
INSTANCE i-57b86940 ami-02bf5b6b ec2-72-44-48-216.compute-1.amazonaws.com
domU-12-31-39-00-A5-EA.compute-1.internal running my-keypair 0 m1.small 2008-06-29T21:47:51+0000 

Now you can log in. Open a different command-line window for each of the following ssh logins:

pjcabrera@dopey:~ $ ssh -i ~/.ec2/id_rsa-my-keypair root@ec2-72-44-48-215.compute-1.amazonaws.com
pjcabrera@dopey:~ $ ssh -i ~/.ec2/id_rsa-my-keypair root@ec2-72-44-48-216.compute-1.amazonaws.com 

In Ubuntu, HA Proxy comes with sample configuration files. These files will need to be configured to conform to the particular goals of your specific project or plan. The configuration files are located at /etc/haproxy.cfg and in folder /etc/ha.d/. To get a full listing of the options that are available in each of these files, consult the HA Proxy documentation at the main HA Proxy site.

In this article, we discussed the concept of load balancing as well as some services that are well suited to being implemented in this environment. Additionally, we explored the pros and cons of some of the more commonly requested applications that can be implemented with load balancing. Finally, we explored how to implement round robin DNS and HA Proxy in Amazon EC2, using load balancing techniques.

This article highlights a few aspects of working with AWS. Here are a few more resources available to help you learn more.

• Amazon Web Services™
• Amazon Elastic Compute Cloud (Amazon EC2) Beta
• Elastic IP Addressing Feature Guide
• alestic.com, a web site listing Debian and Ubuntu AMIs for Amazon EC2
• HA Proxy

PJ Cabrera is a freelance software developer specializing in Ruby on Rails e-commerce and content management systems development. PJ's interests include Ruby on Rails and open-source scripting languages and frameworks, agile development practices, mesh networks, compute clouds, XML parsing and processing technologies, microformats for more semantic web content, and research into innovative uses of Bayesian filtering and symbolic processing for improved information retrieval, question answering, text categorization, and extraction. You can read his weblog at pjtrix.com/blawg/.

Sean-Philip Oriyano has been actively working in the IT field since 1990. Throughout his career, he has held positions such as support specialist to consultants and senior instructor. Currently, he is an IT instructor who specializes in infrastructure and security topics for various public and private entities. Sean has instructed for the U.S. Air Force, U.S. Navy, and U.S. Army at locations in North America and internationally. Sean is certified as a CISSP, CHFI, CEH, CEI, CNDA, SCNP, SCPI, MCT, and MCITP. He is also a member of EC-Council, ISSA, Elearning Guild, and FBI Infragard.