Eclipse Support for AWS Elastic Beanstalk Worker Environment Tiers

A web application is typically concerned with quickly sending responses to incoming queries from its users. This model works really well for things like rendering a web page based on a couple of database queries or validating some user input and storing it to a database. The user makes a request to the application to perform some work, the user’s browser waits while the application does the work, and then the application returns a response, which the user’s browser renders to inform him or her of the result of the action.

In some cases, however, servicing a user request requires some serious computation, such as compiling a month-end report based on multiple large scans over the database. The typical web application model gives a poor user experience in this case—the user’s browser simply “spins” with no feedback until the response is completely ready. In the meantime, the job is taking up resources on your front-end web servers, potentially leading to degraded experience for other users if their requests happen to land on the host running the complicated request.

The Solution: Worker Environment Tiers for Elastic Beanstalk

The solution to these problems is to offload the expensive computations to a back-end application tier asynchronously. The user receives a response from the front-end tier as soon as the work item is queued, indicating that the work is in progress. He or she can check back at his or her leisure to see the progress of the request and view the final result once it becomes available. And since the work is being done by a separate set of back-end hosts which are not serving normal user requests, there’s no chance that running this complicated job will negatively impact other customers.

Recently, the fine folks who make AWS Elastic Beanstalk introduced an exciting new feature called worker environment tiers that make it super easy to implement and deploy these kinds of asynchronous background-processing workers within your application. A worker environment tier accepts work requests from your front-end web environment tier via an Amazon SQS queue. All you have to do is write the business logic to be run when a work request is received, deploy it to an Elastic Beanstalk worker environment tier, then start writing asynchronous work requests to the environment’s queue. You can read some more in-depth information about worker environment tiers here.

The AWS Toolkit for Eclipse now includes support for Elastic Beanstalk worker environment tiers, so you can stand up and start deploying code to a worker environment tier with just a few quick clicks. This post will walk you through the process of setting up a new worker environment tier and deploying code to it, all without leaving the comfort of your favorite IDE!

Create a new worker tier project

The easiest way to get started with worker tiers is to create a new AWS Java Web Project. You can find this new project type under the AWS folder in the New Project dialog; it’s the same application type you may have used in the past to create a web application for Elastic Beanstalk. On the New AWS Java Web Project dialog, select the option to start from a Basic Amazon Elastic Beanstalk Worker Tier Application. This will create a basic Java EE Web Application that accepts work requests via an HTTP POST request to a servlet.

• From the main menu bar, select File -> New -> Project …. Choose the AWS Java Web Project wizard.

• Give your project a name and select the Basic Amazon Elastic Beanstalk Worker Tier Application option.

As with the Basic Java Web Application template, the build path for the newly-created project is preconfigured with the latest version of the AWS SDK for Java, so you’re already set up to interact with AWS services like Amazon S3 and Amazon DynamoDB as part of handling a work request. For example, you may want to have your worker write the result of its processing to Amazon S3 for later retrieval by the front-end tier, or use Amazon DynamoDB to track the progress of any work items that are currently being processed.

The example application demonstrates parsing a simple work request from the POST body, simulates doing some complicated work by sleeping for 10 seconds, then writes the “result” of its work to Amazon S3. Take a quick glance at the code to see how it works.

Create a new Elastic Beanstalk worker tier server

Next, we’ll create a new environment through the Eclipse Web Tools Platform’s Servers view. On the first page of the New Server wizard, select the AWS Elastic Beanstalk for Tomcat 7 server type from within the Amazon Web Services folder. After clicking Next, select the AWS region where your application will be hosted, choose Worker Environment from the Environment Type drop-down, and give your new application and environment names.

• Right-click on the Servers view and select New -> Server.

• Choose the AWS Elastic Beanstalk for Tomcat 7 option.

• Choose a region, application name, environment name, and environment type.

On the next page of the wizard, you can modify other optional settings for the environment. If you have an existing SQS queue you would like your worker environment to read from, you can configure that here; otherwise, a new SQS queue will be automatically created for you.

You can also choose to associate an IAM role with your environment on this page, which is an easy way to give your application permission to access AWS resources such as S3. By default, a new role will be created for you that grants your environment permission to access your SQS queues and publish metrics into Amazon CloudWatch. Your environment won’t be able to function correctly if it doesn’t have these permissions, so if you pick a custom role here, make sure it includes those permissions as well.

• Configure advanced settings for the environment.

On the final page, select your worker project to be deployed to the environment.

Now that everything is configured, right-click your new environment in the Servers view and start it. This process may take several minutes as Elastic Beanstalk provisions EC2 instances and deploys your application to them. Once the process has completed and your application is displayed as “Started”, your workers are ready to go!

• Start your environment.

By default, your environment starts out with a single worker process running on a t1.micro EC2 instance, and will auto-scale up to as many as four instances if CPU usage on the workers is high. Double-clicking the environment in the Servers view will open up a configuration page where you can tweak these settings, along with many more.

Testing out your new worker

Unlike a traditional web application, your worker environment cannot be invoked directly from within your web browser to debug it. Instead, you need to send work requests to your environment via the SQS queue it is subscribed to. You can do this programmatically (as you will ultimately do from your front-end application):

AmazonSQS sqs = ...;

Sring workRequest =
    "{" +
    "  "bucket": "my-results-bucket"," +
    "  "key": "my-work-item-key"," +
    "  "message": "Hello, World"" +
    "}";

sqs.sendMessage(new SendMessageRequest()
    .withQueueUrl(MY_QUEUE_URL)
    .withMessageBody(workRequest));

For testing things out, you can also easily send messages to your application by clicking on the Queue URL in the Environment Resources tab of the server editor (available by double-clicking on the newly-created server in the Servers view). This will bring up a dialog allowing you to quickly send work requests to your environment via the SQS queue in order to test out how it handles them.

Conclusion

Now you’re all set up to use an Elastic Beanstalk worker tier in your application. Just fill in the appropriate code to handle the different kinds of asynchronous work requests your application requires, and with a couple mouse-clicks your updated code can be deployed out to a fleet of back-end workers running in the cloud. Nifty! Are you deploying code to Elastic Beanstalk (either worker tiers or traditional web server tiers) from within Eclipse? Let us know how it’s working in the comments below!