Faster AWS Tests with VCR

Performing integration testing against a live AWS service can often be inefficient since you are forced to make requests across the network for every test. Fortunately, there are tools that you can use to help reduce the amount of network traffic your tests require and make them run faster in the process. We will look at how to do this with the VCR gem available for Ruby.

What is VCR?

VCR is a library that is able to record HTTP network traffic over the wire and then replay those requests locally. Since recording occurs only once, all subsequent executions of your tests do not need to hit the network, and will likely be much faster.

Installing and Using VCR

To install VCR, you need to install two gems: the VCR gem, and the FakeWeb HTTP mocking library:

gem install vcr fakeweb

To use VCR, you simply configure the library with the path on disk to store fixture data (the recorded network traffic), as well as the HTTP mocking library to use (we use fakeweb as the HTTP mocking library):

require 'vcr'

VCR.configure do |c|
  c.cassette_library_dir = 'fixtures/vcr_cassettes'
  c.hook_into :fakeweb
end

After you’ve configured VCR, you can use it with a simple VCR.use_cassette method that wraps and records a block of arbitrary network code:

VCR.use_cassette('some_cassette') do
  # Perform some network transfer
  Net::HTTP.get_response(URI('http://example.com/'))
end

On subsequent executions of this code, VCR uses the data loaded from the fixtures on disk located in fixtures/vcr_cassettes/some_cassette.yaml.

Testing AWS with VCR

Now that you know how to configure and use VCR, it’s easy to plug this library into your existing tests that use the AWS SDK for Ruby to record and replay requests over the network. In our case, we will use the RSpec testing framework, but you can use any testing framework you like.

Test Setup

The top of your test file or helper file requires a bit of configuration. For one, you need to configure VCR (as shown previously), but you also need to configure RSpec to wrap all of the test blocks inside of the VCR.use_cassette method. If you are using RSpec for your testing, you can add the following helper/configuration code to any test suite to get VCR to automatically record and replay your tests for improved speed:

require 'rspec'
require 'aws-sdk'
require 'vcr'

VCR.configure do |c|
  c.cassette_library_dir = 'fixtures/vcr_cassettes'
  c.hook_into :fakeweb
end

# Fixes a missing attribute in Fakeweb's stubbed HTTP client
class FakeWeb::StubSocket; attr_accessor :read_timeout end

RSpec.configure do |c|
  c.around(:each) do |example|
    VCR.use_cassette(example.metadata[:full_description]) do
      example.run
    end
  end
end

Test Example

Now that you’ve set up your test environment, let’s see how it affects your tests. Say, for example, you had a test that uploaded a bunch of files to Amazon S3 and tried to read them back out. The test might look something like this:

describe 'Uploading files to S3' do
  let(:s3) { AWS::S3.new }
  let(:bucket) { s3.buckets['test_s3_upload'] }
  before { s3.buckets.create(bucket.name) }

  it 'uploads multiple files to S3 and reads them out' do
    # Uploads items
    25.times do |i|
      bucket.objects["file#{i}.txt"].write("DATA")
    end

    # Reads items back
    25.times do |i|
      bucket.objects["file#{i}.txt"].read.should == "DATA"
    end
  end
end

Depending on latency, running this test the first time (when VCR is recording) could take anywhere from 5 to 15 seconds. For instance:

$ rspec test_s3.rb 
.

Finished in 14.18 seconds
1 example, 0 failures

But VCR has now recorded our data inside fixtures/vcr_cassettes, and your subsequent execution will be much faster:

$ rspec test_s3.rb
.

Finished in 0.48604 seconds
1 example, 0 failures

Using VCR, this test just went from taking 14 seconds down to 0.5 seconds— MUCH faster. This is because VCR loaded the server-side response from a local file on disk instead of sending the request over the wire.

Clear Your Fixture Cache Often

Although VCR can make your tests much faster, you do lose out on some of the accuracy provided by testing directly against live AWS services. VCR is useful to temporarily cache test results in fast development cycles, but it should not be a complete replacement to full-on integration testing.

One easy way to use the power of a library like VCR and still have the accuracy of full integration testing is to delete your fixtures/vcr_cassettes cache often. You may even want to ignore it from your repository in order to emphasize that the data is, in fact, temporary. That way you will still have fast tests most of the time but still get up-to-date responses from the service at regular intervals during development.

Conclusion

Testing with VCR can make your tests much faster and reduce your costs when testing against AWS services. Best of all, adding this performance optimization is almost fully transparent to your tests, uses actual data from the AWS services, and is simpler than manually creating fixtures that mock the server responses yourself. Try it out and see how much time you can shave off of your tests!

AWS Developer Tools Blog