AWS Developer Tools Blog

Using Amazon SQS with Spring Boot and Spring JMS

By favoring convention over configuration, Spring Boot reduces complexity and helps you start writing applications faster. Spring Boot allows you to bootstrap a framework that abstracts away many of the recurring patterns used in application development. You can leverage the simplicity that comes with this approach when you use Spring Boot and Spring JMS with Amazon SQS. Spring can be used to manage things like polling, acknowledgement, failure recovery, and so on, so you can focus on implementing application functionality.

In this post, we will show you how to implement the messaging of an application that creates thumbnails. In this use case, a client system will send a request message through Amazon SQS that includes an Amazon S3 location for the image. The application will create a thumbnail from the image and then notify downstream systems about the new thumbnail that the application has uploaded to S3.

First we define the queues that we will use for incoming requests and sending out results:

Screenshots of Amazon SQS queue configurations

Here are a few things to note:

  • For thumbnail_requests, we need to make sure the Default Visibility Timeout matches the upper limit of the estimated processing time for the requests. Otherwise, the message might become visible again and retried before we have completed the image processing and acknowledged (that is, deleted) the message.
  • The Amazon SQS Client Libraries for JMS explicitly set the wait time when it polls to 20 seconds, so the Receive Message Wait Time setting for any of the queues will not change polling behavior.
  • Because failures can happen as part of processing the image, we define a Dead Letter Queue (DLQ ) for thumbnail_requests: thumbnail_requests_dlq. Messages will be moved to the DLQ after three failed attempts. Although you can implement a separate process that handles the messages that are put on that DLQ, that is beyond the scope of this post.

Now that the queues are created, we can build the Spring Boot application. We start by creating a Spring Boot application, either command-line or web application. Then we need to add some dependencies to make this work with Amazon SQS.

The following shows the additional dependencies in Maven:

    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-jms</artifactId>
    </dependency>

     <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk</artifactId>
        <version>1.9.6</version>
    </dependency>

    <dependency>
      <groupId>com.amazonaws</groupId>
      <artifactId>amazon-sqs-java-messaging-lib</artifactId>
      <version>1.0.0</version>
      <type>jar</type>
    </dependency>

This will add the Spring JMS implementation, the AWS SDK for Java and the Amazon SQS Client Libraries for JMS. Next, we will configure Spring Boot with Spring JMS and Amazon SQS by defining a Spring Configuration class using the @Configuration annotation:

@Configuration
@EnableJms
public class JmsConfig {

    SQSConnectionFactory connectionFactory =
            SQSConnectionFactory.builder()
                    .withRegion(Region.getRegion(Regions.US_EAST_1))
                    .withAWSCredentialsProvider(new DefaultAWSCredentialsProviderChain())
                    .build();


    @Bean
    public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
        DefaultJmsListenerContainerFactory factory =
                new DefaultJmsListenerContainerFactory();
        factory.setConnectionFactory(this.connectionFactory);
        factory.setDestinationResolver(new DynamicDestinationResolver());
        factory.setConcurrency("3-10");
        factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
        return factory;
    }

    @Bean
    public JmsTemplate defaultJmsTemplate() {
        return new JmsTemplate(this.connectionFactory);
    }

}

Spring Boot will find the configuration class and instantiate the beans defined in the configuration. The @EnableJms annotation will make Spring JMS scan for JMS listeners defined in the source code and use them with the beans in the configuration:

  • The Amazon SQS Client Libraries for JMS provides the SQSConnectionFactory class, which implements the ConnectionFactory interface as defined by the JMS standard, allowing it to be used with standard JMS interfaces and classes to connect to SQS.

    • Using DefaultAWSCredentialsProviderChain will give us multiple options for providing credentials, including using the IAM Role of the EC2 instance.
  • The JMS listener factory will be used when we define the thumbnail service to listen to messages.

    • Using the DynamicDestinationResolver will allow us to refer to Amazon SQS queues by their names in later classes.
    • The values we provided to Concurrency show that we will create a minimum of 3 listeners that will scale up to 10 listeners.
    • Session.CLIENT_ACKNOWLEDGE will make Spring acknowledge (delete) the message after our service method is complete. If the method throws an exception, Spring will recover the message (that is, make it visible).
  • The JMS template will be used for sending messages.

Next we will define the service class that will listen to messages from our request queue.

@Service
public class ThumbnailerService {

    private Logger log = Logger.getLogger(ThumbnailerService.class);

    @Autowired
    private ThumbnailCreatorComponent thumbnailCreator;

    @Autowired
    private NotificationComponent notification;

    @JmsListener(destination = "thumbnail_requests")
    public void createThumbnail(String requestJSON) throws JMSException {
        log.info("Received ");
        try {
            ThumbnailRequest request=ThumbnailRequest.fromJSON(requestJSON);
            String thumbnailUrl=
                     thumbnailCreator.createThumbnail(request.getImageUrl());
            notification.thumbnailComplete(new ThumbnailResult(request,thumbnailUrl));
        } catch (IOException ex) {
            log.error("Encountered error while parsing message.",ex);
            throw new JMSException("Encountered error while parsing message.");
        }
    }

}

The @JmsListener annotation marks the createThumbnail method as the target of a JMS message listener. The method definition will be consumed by the processor for the @EnableJms annotation mentioned earlier. The processor will create a JMS message listener container using the container factory bean we defined earlier and the container will start to poll messages from Amazon SQS. As soon as it has received a message, it will invoke the createThumbnail method with the message content.

Here are some things to note:

  • We define the SQS queue to listen to (in this case thumbnail_requests) in the destination parameter.
  • If we throw an exception, the container will put the message back into the queue. In this example, the message will be eventually put onto the DLQ queue after three failed attempts.
  • We have autowired two components to process the image (ThumbnailCreatorComponent) and to send notifications (NotificationComponent) when the image has been processed.

The following shows the implementation of the NotificationComponent that is used to send a SQS message:

@Component
public class NotificationComponent {

    @Autowired
    protected JmsTemplate defaultJmsTemplate;

    public void thumbnailComplete(ThumbnailResult result) throws IOException {
        defaultJmsTemplate.convertAndSend("thumbnail_results", 
                                          result.toJSON());
    }

}

The component uses the JMS template defined in the configuration to send SQS messages. The convertAndSend method takes the name of the SQS queue that we want to send the message to.

JSON is the format in the body of our messages. We can easily convert that in our value objects. Here is an example with the class for the thumbnail request:

public class ThumbnailRequest {

    String objectId;
    String imageUrl;

    public String getObjectId() {
        return objectId;
    }

    public void setObjectId(String objectId) {
        this.objectId = objectId;
    }

    public String getImageUrl() {
        return imageUrl;
    }

    public void setImageUrl(String imageUrl) {
        this.imageUrl = imageUrl;
    }

    public static ThumbnailRequest fromJSON(String json) 
                               throws JsonProcessingException, IOException {
        ObjectMapper objectMapper=new ObjectMapper();
        return objectMapper.readValue(json, ThumbnailRequest.class);
    }
}

We will skip implementing the component for scaling the image. It’s not important for this demonstration.

Now we need to start the application and start sending and receiving messages. When everything is up and running, we can test the application by submitting a message like this on the thumbnail_requests:

{
    "objectId":"12345678abcdefg",
    "imageUrl":"s3://mybucket/images/image1.jpg"
}

To send a message, go to the SQS console and select the thumbnail_requests queue. Choose Queue Actions and then choose Send a Message. You should see something like this in thumbnail_results:

{
    "objectId":"12345678abcdefg",
    "imageUrl":"s3://mybucket/images/image1.jpg",
    "thumbnailUrl":"s3://mybucket/thumbnails/image1_thumbnail.jpg"
}

To see the message, go to the SQS console and select the thumbnail_results queue. Choose Queue Actions and then choose View/Delete Messages.