AWS Developer Blog

Ruby at re:Invent 2012

by Trevor Rowe | on | in Ruby | Permalink | Comments |  Share

Loren and I will be attending AWS re:Invent in Las Vegas next week. I will be presenting a session, and we will both be answering questions in the Developer Lounge, and assisting with the AWS re:Invent Code Challenges.

Stop by to say "Hi", get your questions answered, or participate in a Code Challenge. We love to meet developers, get feedback, and to talk Ruby.

Getting Productive with the AWS SDK for Ruby, Wednesday, November 28 from 4:30 p.m. – 5:20 p.m.

Join us in the Venetian, Room 3001A. You will learn about how to get the most of the aws-sdk gem, and we will be building a sample application live that uses three different AWS services.

See you in Las Vegas!

Deploying Ruby Applications to AWS Elastic Beanstalk with Git

by Loren Segal | on | in Ruby | Permalink | Comments |  Share

With the release of Ruby container support on AWS Elastic Beanstalk, it is now possible to deploy Rails (or any Rack-based) applications onto the AWS infrastructure in just a few easy steps. The container is backed by Amazon EC2 and automatically provisions the machines to use Phusion Passenger (with nginx under the hood). You can run your own local database with your application, or better yet, you can have Elastic Beanstalk automatically set you up with a MySQL database backed by Amazon Relational Database Service (RDS). All of the machines sit behind a load balancer with auto-scaling support automatically configured so you don’t even have to think about ops. Scaling is built-in.

But enough with the awesome features. Let’s deploy an app!

Setting up the application

Before we can deploy an application to Elastic Beanstalk, we first have to create one. Let’s generate an empty Rails skeleton and set it up against a Git repository. We will be using the repository later to deploy our app.

$ rails new fooapp
$ cd fooapp
$ git init && git add -A && git commit -m "Initial commit"

Initializing our Elastic Beanstalk environment

At this point we have an application, but we have not yet configured it to deploy to Elastic Beanstalk. In order to do that, we need to install the AWS Elastic Beanstalk command line tool. This package provides us a binary, eb, which we will use to manage and deploy our applications.

Note that you do need to have Python (2.7 or 3.0) installed in order to use the CLI tools. If you are on OS X or a modern Linux OS, you should already have this installed. If not, visit Python’s website for download information. In order to verify if you have Python on your machine, you can type python --version

We can now initialize our application with the CLI tool by typing eb init. The tool will ask us for some information about our deployment such as our AWS credentials, the deployment region, and the solution stack we want to deploy. For our case, we want to use the "64bit Amazon Linux running Ruby 1.9.3" stack. The tool also asks if we want to setup an RDS database instance for our environment. Let’s answer "no" on that one for now.

$ eb init
To get your AWS Access Key ID and Secret Access Key, 
  visit "https://aws-portal.amazon.com/gp/aws/securityCredentials".
Enter your AWS Access Key ID: ****
Enter your AWS Secret Access Key: ****
Select an AWS Elastic Beanstalk service region.
Available service regions are:
2) US West (Oregon)
...
Select:  (1 to 6): 2
Enter an AWS Elastic Beanstalk application name (auto-generated value is "fooapp"): 
Enter an AWS Elastic Beanstalk environment name (auto-generated value is "fooapp-env"): 
Select a solution stack.
Available solution stacks are:
...
10) 64bit Amazon Linux running Ruby 1.9.3
Select:  (1 to 10): 10
Create an RDS DB Instance? [y/n]: n         

Note that this command will create a .elasticbeanstalk directory in our project and add it to our .gitignore. This directory contains the private environment configuration data provided above, as well as other customized settings. We then commit this change to our .gitignore with the following command:

$ git add .gitignore && git commit -m "Ignore .elasticbeanstalk from Git"

You can see more examples of setting up a project with the eb tool on the Amazon Web Services Blog.

Starting the environment

Now that our environment is configured, we simply need to start it. Starting the environment can take a couple of minutes. During this time, Elastic Beanstalk is spawning a set of EC2 instances, setting up a load balancer, and establishing health monitoring. Fortunately we only have to do this once per environment. Once Elastic Beanstalk is done, we can start the environment with the following command:

$ eb start
Starting application "fooapp".
Waiting for environment "fooapp-env" to launch.
... 
Application is available at "fooapp-env-djr9ywimiu.elasticbeanstalk.com".

After a little bit of waiting, our environment should be live. We can use the environment in a web browser with the URL provided above:

AWS Elastic Beanstalk Ruby Container Welcome Page

You can now also type eb status to see the status of your environment:

$ eb status
URL : fooapp-env-djr9ywimiu.elasticbeanstalk.com
Status  : Ready
Health  : Green

Deploying with Git

The eb init command we ran above actually did some nifty things behind the scenes. The most notable thing is that it setup a Git subcommand named aws.push that can push our code to the environment. This means that whenever we need to deploy our app, we just have to type git aws.push. Since our environment is running, and our app is ready, let’s do that right now!

$ git aws.push

If everything succeeds, you should have initiated a deploy to your environment. Typing eb status should now tell us that a deploy is in progress:

$ eb status
URL : fooapp-env-djr9ywimiu.elasticbeanstalk.com
Status  : Updating
Health  : Grey

Eventually, the status will be green, and we will be able to access our new Rails application:

Rails Welcome Page

Notice that gems listed in our Gemfile were automatically installed, assets were precompiled, and, if we had any, database migrations would have been run too. Migrations and asset compilation happen by default for any Rails 3.x application.

All we have to do now is update our application, commit to our Git repository, and execute the command git aws.push to push our new changes to deploy new versions of the app. Migrations will be run, and our assets will get updated.

Customizing your environment with .ebextensions

To run commands and drop environment variables onto your system via your app during a deployment, you can add a special .config file to the .ebextensions directory in your project root. The config file is formatted as YAML, and is documented on the Customizing and Configuring AWS Elastic Beanstalk Environments documentation page.

For reference, here is a sample .ebextensions/redmine.config that runs some extra hooks for installing Redmine on Elastic Beanstalk. Note that the environment variables are not necessary; BUNDLE_WITHOUT and RACK_ENV already default to the values below. We’re just setting them as an example of some config we might want to perform.

# Sample configuration for Redmine Rails installation

option_settings:
  - option_name: BUNDLE_WITHOUT
    value: "test:development"
  - option_name: RACK_ENV
    value: production

packages:
  yum:
    git: []

container_commands:
  01_secret:
    command: rake generate_secret_token

Skipping migrations and asset compilation

I mentioned you can set environment variables using the option_settings key in the .ebextensions/*.config files. Two specific variables you might want to set are:

  • RAILS_SKIP_MIGRATIONS — set this to true if you don’t want Elastic Beanstalk to run migrations on your behalf. You may want to manage migrations yourself.
  • RAILS_SKIP_ASSET_COMPILATION — set this to true if you don’t use asset compilation in your Rails app. Note, however, that your Elastic Beanstalk container is already configured with a JS runtime to properly handle asset compilation, you do not have to do anything fancy like install the therubyracer gem. It should just work.

Setting up Amazon RDS

The final-but-very-optional step for using the Ruby solution stack in Elastic Beanstalk is to hook up an RDS Instance to your application. If you are running any kind of real world application, you most likely have some database. If your application uses a relational database, you can take advantage of Amazon RDS for access to a MySQL (or Oracle) database.

Fortunately, the setup for this is extremely simple. Remember when we said "no" to creating an RDS DB Instance when we ran eb init? Well, we can initialize another application in the exact same way, but instead of no, we can say yes. We will be prompted to provide a password for this database, but everything else will be directly configured by Elastic Beanstalk on our behalf.

How do we configure it with our Rails app, you ask? Easy. All of the DB information will be passed to our application as environment variables. In Rails, this means we can setup our application to use RDS with a simple database.yml snippet:

production:
  adapter: mysql2
  encoding: utf8
  database: <%= ENV['RDS_DB_NAME'] %>
  username: <%= ENV['RDS_USERNAME'] %>
  password: <%= ENV['RDS_PASSWORD'] %>
  host: <%= ENV['RDS_HOSTNAME'] %>
  port: <%= ENV['RDS_PORT'] %>

The only other thing we have to do is add mysql2 to our Gemfile. It would look something like:

gem 'mysql2'

And RDS should now be fully configured for our Rails application.

Testing locally with Passenger

Because we use Passenger in your container, it’s really easy to test your deployment locally in a very similar webserver environment. If you don’t already have passenger installed on your local machine, you can set it up and initialize your server with:

$ gem install passenger
$ passenger start

A few quick notes if you’ve never deployed to a Passenger server before:

  • Passenger relies on a config.ru Rack file in your root application directory by convention.
  • Passenger also relies on a public/ directory for your static assets located in your application root.

Passenger needs both of these paths to exist in order to detect your Rack/Rails application. In a Rails 3.x app, these paths already exist, but if you are rolling your own low-level Rack application, you need to make sure that they are created. Once they have been setup, your application is ready to be tested as if it were deployed to Elastic Beanstalk. You can consult the Phusion Passenger documentation for more information about how to run your application on this snappy webserver.

Conclusion

So that’s it. We just went through the nuts and bolts of deploying a Ruby application to AWS Elastic Beanstalk. Although there is a lot here to read through, the steps themselves are pretty simple. In short, you can deploy an application with a few commands:

  1. Run eb init on your Git-backed application
  2. Run eb start to load up your environment
  3. Run git aws.push to deploy your code to Elastic Beanstalk
  4. Profit!

Setting up RDS is just a matter of using the configuration values provided to your application through standard environment variables. And of course if you need to dig deeper and customize settings, you can do so by adding a .ebextensions/config.yml file.

There is of course plenty you can do with an Elastic Beanstalk application, and there are plenty more things to talk about with the Ruby solution stack, but hopefully this helps you get a Rack-based application up and running in just a few minutes!

Storing Java objects in Amazon DynamoDB tables

by zachmu | on | in Java | Permalink | Comments |  Share

The AWS SDK for Java makes it easy to store objects in Amazon DynamoDB and get them back out again, all without having to write the code to transform your objects into table items and vice versa. All you need to do is annotate your domain classes in a few places and the SDK will handle the work of getting objects into and out of the database. For example, here’s a minimal User class we want to store in DynamoDB:

@DynamoDBTable(tableName = "users")
public class User {
 
    private Integer id;
    private Set<String> friends;
    private String status;
 
    @DynamoDBHashKey
    public Integer getId() { return id; }
    public void setId(Integer id) { this.id = id; }
 
    @DynamoDBAttribute
    public Set<String> getFriends() { return friends; }
    public void setFriends(Set<String> friends) { this.friends = friends; }
 
    @DynamoDBAttribute
    public String getStatus() { return status; }
    public void setStatus(String status) { this.status = status; }
}

The DynamoDBMapper utility class makes it simple to store instances of this class in DynamoDB.

AmazonDynamoDB dynamo = new AmazonDynamoDBClient(awsCredentials);
DynamoDBMapper mapper = new DynamoDBMapper(dynamo);
 
// save a new item
mapper.save(newUser);
 
// update the item
newUser.setStatus("active");
newUser.getFriends().remove("Jeremy");
mapper.save(newUser);
 
// delete the item
mapper.delete(newUser);

DynamoDBMapper can also handle query and scan operations with automatic result pagination, optimistic locking, automatic hash key generation, custom marshaling logic, and much more. For more detail, see this article.

New Blog, New Org, and New Issue Reporting!

by Loren Segal | on | in Ruby | Permalink | Comments |  Share

Hey everyone!

My name is Loren Segal, and I will be working with Trevor Rowe on our shiny new blog here at Amazon Web Services. We both work on the AWS SDK for Ruby, and over the coming weeks, months, and years, we will be using this spot to post new updates, tips, and tricks about the Ruby SDK. We hope these posts will be informative and shed some light on what’s coming down the pipe, but we also want to get the word out about all the cool things we already support. We’re really excited about sharing all this new information with you.

And with that, we already have new announcements to make: we are changing our GitHub organization name and enabling GitHub Issues in our new organization for easier issue and feature tracking in the Ruby SDK!

A better GitHub organization name

We decided to move our GitHub amazonwebservices organization to the much shorter and easier-to-remember aws organization. With that change, we’ve also renamed the project from "aws-sdk-for-ruby" to a simpler "aws-sdk-ruby". We will be following this naming convention across our other projects to make things easier for our collaborators.

As of this blog post, we will no longer be developing against our old repository, but we will leave this old repository around so that users can still access the code. Note, however, that in about a week, we replace the contents of the repository with a README guiding our new users to the proper repository location.

Remember to update your Git remotes and Gemfiles!

If you’re currently pointing your remotes to our old organization, now is the time to update the location to the new repository. You can do so by typing the following command in your local working copy (assuming the remote is origin):

$ git remote set-url origin git://github.com/aws/aws-sdk-ruby

If you are pointing your Gemfiles to our Git repository, you will also want to update those by pointing to the new repo:

gem 'aws-sdk', :git => "git://github.com/aws/aws-sdk-ruby.git"

Note that for Gemfiles, we would much rather you point to the gem released on RubyGems.org, though!

GitHub Issues are enabled!

Finally, with our newly renamed organization, we’ve decided to enable GitHub Issues on the new aws-sdk-ruby project. You can now post bug reports, submit pull requests, and track development progress right on our issues page.

Since we’re on the topic, here are a couple of notes about submitting new issues:

Submitting pull requests: include tests

We love when users contribute code to the SDK! Really! If you’ve found a bug in the SDK and you’ve tracked down the issue and have a local working patch, submitting a pull request will help us get the fix out to our users much faster than just submitting a bug report and greedily keeping that awesome bugfix to yourself!

However, if you do plan on submitting pull requests, please try to include tests for your new code. The better the tests, the more easily we can guarantee that your patch will continue to work, and the quicker we can pull it in. Now that we are using Travis-CI for continuous integration, GitHub will happily tell us that your changes will not break the build.

Submitting bug reports: include reproduction steps

If you are having issues with the SDK not working properly, we want to hear about it and fix it. GitHub Issues now allows you to post your problems directly on our GitHub repository. If you do report an issue, please make sure to include reliable reproduction steps, as well as much extra relevant information as possible. Please make sure you do not post your AWS credentials in bug reports, however! We will never ask you for this information— that is your data.

Shiny new things!

We’re really happy about these new changes. The new blog will be here to provide new insight on what we’re up to on the Ruby team, and hopefully the new GitHub re-organization will make it much easier to contribute code and submit bugs on the AWS SDK for Ruby! Go check out our new org right now:

http://github.com/aws/aws-sdk-ruby

Welcome to the AWS SDKs and Tools Java blog

Welcome to the AWS SDKs and Tools Java blog. We’re glad to see you here!

This blog features information for AWS Java developers, including:

  • Demonstrations of new features in the AWS SDK for Java
  • Announcements of new features in the AWS Toolkit for Eclipse
  • Best practices for using the SDK
  • Important AWS product announcements for Java developers
  • Lots of code samples

You’ll see content from many of our team members, including developers from the SDKs and Tools team, as well as Java developers from service teams.

We hope you come back often to visit or subscribe to our blog using the RSS feed button at the top of the page. If you’d like us to cover any specific topics, please let us know and we’ll do our best.