Q: What is Amazon SQS?
Amazon Simple Queue Service (Amazon SQS) is a web service that gives you access to message queues that store messages waiting to be processed. With Amazon SQS, you can quickly build message queuing applications that can run on any computer.
Amazon SQS offers a reliable, highly-scalable, hosted queue for storing messages in transit between computers. With Amazon SQS, you can move data between diverse, distributed application components without losing messages and without requiring each component to be always available.
Amazon SQS can help you build a distributed application with decoupled components, working closely with the Amazon Elastic Compute Cloud (Amazon EC2) and the other AWS infrastructure web services.
Q: What can I do with Amazon SQS?
Because Amazon SQS is highly-scalable and you pay only for what you use, you can start small and grow your application alongside your business needs, with no performance or reliability compromises. Amazon SQS lets you stop worrying about how your messages are stored and managed and helps you focus on building robust, sophisticated message-based applications.
Here are just a few ideas:
- Integrate Amazon SQS with other AWS services to make applications more flexible and reliable.
- Use Amazon SQS to create work queues with each message as a task to be completed by a process. Let one (or many) computers read tasks from the message queue and process them.
- Build a microservice architecture and use message queues to connect your microservices.
- Keep notifications of significant business events in an Amazon SQS message queue. Each event can have a corresponding message in a message queue, and applications that need to be aware of the event can read and process the messages.
Q: How can I get started using Amazon SQS?
You can get started with Amazon SQS in a few steps:
- Register for an AWS account.
- After signing up, visit the AWS Management Console, select SQS, and on the next page select Create New Queue.
- In the Create New Queue dialog box, enter a Queue Name (for example, MyQueue), and click Create Queue.
- Select a queue, and from the Queue Actions drop-down list select Send a Message.
- In the Send a Message to MyQueue dialog box, on the Message Body tab, enter the text of your message and click Send Message.
The message is sent and a confirmation with the sent message attributes is displayed. Click Close to finish.
Q: What are the benefits of Amazon SQS over homegrown or packaged message queuing systems?
Using Amazon SQS provides several advantages over building your own software for managing message queues or using commercial or open-source message queuing systems that require significant up-front time for development and configuration.
These alternatives require ongoing hardware maintenance and system administration resources. The complexity of configuring and managing these systems is compounded by the need for redundant storage of messages that ensures messages are not lost if hardware fails.
In contrast, Amazon SQS requires no administrative overhead and little configuration. Moreover, Amazon SQS works on a massive scale, processing billions of messages per day. You can scale the amount of traffic you send to Amazon SQS up or down without any configuration. Amazon SQS also provides extremely high message durability, giving you and your stakeholders added confidence.
Q: When should I use Amazon Simple Workflow Service (Amazon SWF) instead of Amazon SQS?
- Amazon SWF is a service designed for orchestrating highly-scalable applications; it also provides auditability.
- Amazon SQS provides a reliable, highly-scalable, hosted queue for sending and receiving messages.
While you can use Amazon SQS to build basic workflows to coordinate your distributed application, you can get this facility out-of-the-box with Amazon SWF, alongside other application-level capabilities.
We recommend trying both Amazon SQS and Amazon SWF to determine which solution best fits your needs.
Q: Does Amazon use Amazon SQS for its own applications?
Yes. Developers at Amazon use Amazon SQS for a variety of applications that process large numbers of messages every day. Key business processes in both Amazon.com and Amazon Web Services use Amazon SQS.
Q: What can I do with the Amazon SQS Free Tier?
The Amazon SQS Free Tier provides you with 1 million requests per month at no charge.
Many small-scale applications are able to operate entirely within the limits of the Free Tier. However, data transfer charges might still apply. For more information, see Amazon SQS Pricing.
The Free Tier is a monthly offer. Free usage does not accumulate across months.
Q: How much does Amazon SQS cost?
You pay only for what you use, and there is no minimum fee.
In most regions, the cost of Amazon SQS is $0.50 for every 1 million requests, plus data transfer charges for data transferred out of Amazon SQS (unless data is transferred to Amazon EC2 instances or to AWS Lambda functions within the same region). For more information, see Amazon SQS Pricing.
Q: Will I be charged for all Amazon SQS requests?
Yes. All Amazon SQS requests are chargeable, and they are billed at the same rate.
Q: Do Amazon SQS batch operations cost more than other requests?
No. Batch operations (that include SendMessageBatch, DeleteMessageBatch, and ChangeMessageVisibilityBatch) all cost the same as other Amazon SQS requests. By grouping messages into batches, you can reduce your Amazon SQS costs.
Q: How will I be charged and billed for my use of Amazon SQS?
There are no initial fees to begin using Amazon SQS. At the end of the month, your credit card will be automatically charged for the month’s usage.
You can view your charges for the current billing period at any time on the AWS website:
- Log into your AWS account.
- Under Your Web Services Account, select Account Activity.
Q: Do your prices include taxes?
Except as noted otherwise, our prices do not include any applicable taxes and duties such as VAT or applicable sales tax.
For customers with a Japanese billing address, the use of AWS in any region is subject to Japanese Consumption Tax. For more information, see the Amazon Web Services Consumption Tax FAQ.
Q: Can I use Amazon SQS with other AWS services?
Yes. You can make your applications more flexible and scalable by using Amazon SQS with compute services such as Amazon EC2, Amazon EC2 Container Service (Amazon ECS), and AWS Lambda, as well as with storage and database services such as Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB.
One common use case is a distributed, decoupled application whose multiple components and modules need to communicate with each other, but can’t do the same amount of work simultaneously. In this case, Amazon SQS message queues carry messages to be processed by the application running on Amazon EC2 instances.
The Amazon EC2 instances can read the message queue, process the job, and then post the results as messages to another Amazon SQS message queue (for example, for further processing by another application). Because Amazon EC2 allows applications to scale up and down dynamically, application developers can vary the number of compute instances based on the amount of messages in the Amazon SQS queues using Auto Scaling, to ensure that jobs are executed in a timely manner.
Q: Can you give me an example use case for Amazon SQS?
Here is how a video transcoding website uses Amazon EC2, Amazon SQS, Amazon S3, and Amazon DynamoDB together:
- End users submit videos to be transcoded to the website.
- The videos are stored in Amazon S3, and a request message is placed in an incoming Amazon SQS queue with a pointer to the video and to the target video format within the message.
- The transcoding engine that runs on a set of Amazon EC2 instances reads the request message from the incoming queue, retrieves the video from Amazon S3 using the pointer, and transcodes the video into the target format.
- The converted video is put back into Amazon S3 and another response message is placed in another outgoing Amazon SQS queue with a pointer to the converted video.
- At the same time, metadata about the video (format, date created, length, and so on) is indexed into Amazon DynamoDB for querying.
During this workflow, a dedicated Auto Scaling instance can constantly monitor the incoming queue. Based on the number of messages in the incoming queue, the Auto Scaling instance dynamically adjusts the number of transcoding Amazon EC2 instances to meet the response time requirements of the website's customers.
Q: How do I interact with Amazon SQS?
You can access Amazon SQS using the AWS Management Console, which has a visual, web-based interface for setting up and managing Amazon SQS.
Amazon SQS also provides a web services API and it is integrated with the AWS SDKs, allowing you to work in the programming language of your choice.
Q: What are the available operations for message queues?
For information on message queue operations, see Amazon SQS Product Details.
Q: Who can perform operations on a message queue?
Only an AWS account owner (or an AWS account that the account owner has delegated rights to) can perform operations on an Amazon SQS message queue.
Q: Can I use Java Message Service (JMS) with Amazon SQS?
Yes. You can take advantage of the scale, low cost, and high availability of Amazon SQS without the worry and high overhead of running your own JMS cluster.
Q: How are messages identified in the system?
All messages have a global unique ID that Amazon SQS returns when the message is delivered to the message queue. The ID isn’t required to perform any further actions on the message, but it is useful for tracking the receipt of a particular message in the message queue.
When you receive a message from the message queue, the response includes a receipt handle that you must provide when deleting the message.
Q: How are unsuccessfully-processed messages handled?
In Amazon SQS, you can use the API or the console to configure dead letter queues, which are queues that you configure to receive messages from other source queues.
Any queue can become a dead letter queue that will receive messages after a maximum number of processing attempts cannot be completed. You can use dead letter queues to isolate messages that can't be processed for later analysis.
For more information, see Using Amazon SQS Dead Letter Queues.
Q: Does Amazon SQS provide first-in-first-out (FIFO) access to messages?
Amazon SQS provides a loose-FIFO capability that attempts to preserve the order of messages. However, we have designed Amazon SQS to be massively scalable using a distributed architecture. Thus, we can't guarantee that you will always receive messages in the exact order you sent them (FIFO).
If your system requires the order of messages to be preserved, place sequencing information in each message so that messages can be ordered when they are received.
Q: Does Amazon SQS provide at-least-once delivery of messages?
Yes. Amazon SQS guarantees that each message is delivered at least once. Amazon SQS stores copies of your messages on multiple servers for redundancy and high availability. On rare occasions, one of the servers that stores a copy of a message might be unavailable when you receive or delete the message.
If this occurs, the copy of the message will not be deleted on that unavailable server, and you might get a copy of that message again when you receive messages (at-least-once delivery).
You must design your applications to be idempotent (that is, they must not be affected adversely when processing the same message more than once).
Q: What is a visibility timeout?
The visibility timeout is a period of time during which Amazon SQS prevents other consuming components from receiving and processing a message. For more information, see Visibility Timeout.
Q: How does Amazon SQS allow multiple readers to access the same message queue without losing messages or processing them multiple times?
Every Amazon SQS queue has a configurable visibility timeout. A message is not visible to any other reader for a designated amount of time when it is read from a message queue. As long as the amount of time it takes to process the message is less than the visibility timeout, every message is processed and deleted.
If the component processing of the message fails or becomes unavailable, the message again becomes visible to any component reading the message queue once the visibility timeout ends. This allows multiple components to read messages from the same message queue, each one working to process different messages.
Q: What is the maximum limit for message visibility?
The maximum visibility timeout for an Amazon SQS message is 12 hours.
Q: Does Amazon SQS support message metadata?
Yes. An Amazon SQS message can contain up to 10 metadata attributes. You can use message attributes to separate the body of a message from the metadata that describes it. This helps process and store information with greater speed and intelligence because your applications do not have to inspect an entire message before understanding how to process it.
Amazon SQS message attributes take the form of name-type-value triples. The supported types include string, binary, and number (including integer, floating-point, and double). For more information, see Using Amazon SQS Message Attributes.
Q: How can I determine the time-in-queue value?
To determine the time-in-queue value, you can request the SentTimestamp attribute when receiving a message. Subtracting that value from the current time results in the time-in-queue value.
Q: What is the typical latency for Amazon SQS?
Typical latencies for SendMessage, ReceiveMessage, and DeleteMessage API requests are in the tens or low hundreds of milliseconds.
Q: For anonymous access, what is the value of the SenderId attribute for a message?
When the AWS account ID is not available (for example, when an anonymous user sends a message), Amazon SQS provides the IP address.
Q: What is Amazon SQS long polling?
Amazon SQS long polling is a way to retrieve messages from your Amazon SQS queues. While the regular short polling returns immediately, even if the message queue being polled is empty, long polling doesn’t return a response until a message arrives in the message queue, or the long poll times out.
Long polling makes it inexpensive to retrieve messages from your Amazon SQS queue as soon as the messages are available. Using long polling might reduce the cost of using SQS, because you can reduce the number of empty receives. For more information, see Amazon SQS Long Polling.
Q: Is there an additional charge for using Amazon SQS long polling?
No. Long-polling ReceiveMessage calls are billed exactly the same as short-polling ReceiveMessage calls.
Q: When should I use Amazon SQS long polling, and when should I use Amazon SQS short polling?
In almost all cases, Amazon SQS long polling is preferable to short polling. Long-polling requests let your queue consumers receive messages as soon as they arrive in your queue while reducing the number of empty ReceiveMessageResponse instances returned.
Amazon SQS long polling results in higher performance at reduced cost in the majority of use cases. However, if your application expects an immediate response from a ReceiveMessage call, you might not be able to take advantage of long polling without some application modifications.
For example, if your application uses a single thread to poll multiple queues, switching from short polling to long polling will probably not work, because the single thread will wait for the long-poll timeout on any empty queues, delaying the processing of any queues that might contain messages.
In such an application, it is a good practice to use a single thread to process only one queue, allowing the application to take advantage of the benefits that Amazon SQS long polling provides.
Q: What value should I use for my long-poll timeout?
In general, you should use maximum 20 seconds for a long-poll timeout. Because higher long-poll timeout values reduce the number of empty ReceiveMessageResponse instances returned, try to set your long-poll timeout as high as possible.
If the 20-second maximum doesn't work for your application (see the example in the previous question), set a shorter long-poll timeout, as low as 1 second.
All AWS SDKs work with 20-second long polls by default. If you don't use an AWS SDK to access Amazon SQS, or if you configured your AWS SDK to specifically have a shorter timeout, you might need to modify your Amazon SQS client to allow longer requests or to use a shorter long-poll timeout.
Q: What is the AmazonSQSBufferedAsyncClient for Java?
The AmazonSQSBufferedAsyncClient for Java provides an implementation of the AmazonSQSAsyncClient interface and adds several important features:
- Automatic batching of multiple SendMessage, DeleteMessage, or ChangeMessageVisibility requests without any required changes to the application
- Prefetching of messages into a local buffer that allows your application to immediately process messages from Amazon SQS without waiting for the messages to be retrieved
Working together, automatic batching and prefetching increase the throughput and reduce the latency of your application while reducing your costs by making fewer Amazon SQS requests. For more information, see Client-Side Buffering and Request Batching.
Q: Where can I download the AmazonSQSBufferedAsyncClient for Java?
You can download the AmazonSQSBufferedAsyncClient as part of the AWS SDK for Java.
Q: Do I have to rewrite my application to use the AmazonSQSBufferedAsyncClient for Java?
No. The AmazonSQSBufferedAsyncClient for Java is implemented as a drop-in replacement for the existing AmazonSQSAsyncClient.
If you update your application to use the latest AWS SDK and change your client to use the AmazonSQSBufferedAsyncClient for Java instead of the AmazonSQSAsyncClient, your application will receive the added benefits of automatic batching and prefetching.
Q: How can I subscribe Amazon SQS message queues to receive notifications from Amazon Simple Notification Service (Amazon SNS) topics?
- In the Amazon SQS console, select an Amazon SQS message queue.
- Under Queue Actions, select Subscribe Queue to SNS Topic from the drop-down list.
- In the dialog box, select the topic from the Choose a Topic drop-down list, and click Subscribe.
For more information, see Subscribing a Queue to an Amazon SNS Topic.
Q: How can I fan-out identical messages to multiple Amazon SQS queues?
- Use Amazon SNS to create a topic.
- Create and subscribe multiple Amazon SQS message queues to the Amazon SNS topic.
- Whenever a message is sent to the Amazon SNS topic, it is fanned out to the Amazon SQS message queues.
Amazon SNS delivers the message to all Amazon SQS message queues subscribed to the topic.
Q: Can I delete all messages in a message queue without deleting the message queue itself?
Yes. You can delete all messages in an Amazon SQS message queue using the PurgeQueue action.
When you purge a message queue, all the messages previously sent to the message queue are deleted. Because your message queue and its attributes remain, there is no need to reconfigure the message queue; you can continue using it.
To delete only specific messages, use the DeleteMessage or DeleteMessageBatch actions.
Q: How reliable is the storage of my data in Amazon SQS?
Amazon SQS stores all message queues and messages within a single, highly-available AWS region with multiple redundant Availability Zones (AZs), so that no single computer, network, or AZ failure can make messages inaccessible. For more information, see Regions and Availability Zones.
Q: How can I secure the messages in my message queues?
Authentication mechanisms ensure that messages stored in Amazon SQS message queues are secured against unauthorized access. You can control who can send messages to a message queue and who can receive messages from a message queue. For additional security, you can build your application to encrypt messages before they are placed in a message queue.
Amazon SQS has its own resource-based permissions system that uses policies written in the same language as AWS Identity and Access Management (IAM) policies: for example, you can use variables, just like in IAM policies. For more information, see Amazon SQS Policy Examples.
Amazon SQS supports the HTTP over SSL (HTTPS) and Transport Layer Security (TLS) protocols. Most clients can automatically negotiate to use newer versions of TLS without any code or configuration change. Amazon SQS supports versions 1.0, 1.1, and 1.2 of the Transport Layer Security (TLS) protocol in all regions.
Q: Why are there separate ReceiveMessage and DeleteMessage operations?
When Amazon SQS returns a message to you, the message stays in the message queue whether or not you actually receive the message. You are responsible for deleting the message and the deletion request acknowledges that you’re done processing the message.
If you don’t delete the message, Amazon SQS will deliver it again on when it receives another receive request. For more information, see Visibility Timeout.
Q: Can a deleted message be received again?
Yes. Under rare circumstances, you might receive a previously-deleted message a second time. This can happen in the rare situation when a DeleteMessage operation doesn’t delete all copies of a message because one of the servers in the distributed Amazon SQS system isn’t available at the time of deletion. This message copy can be delivered again.
To avoid this behavior, design your application to be idempotent (that is, no errors or inconsistencies occur if you receive a deleted message a second time).
Q: What happens if I issue a DeleteMessage request on a previously-deleted message?
When you issue a DeleteMessage request on a previously-deleted message, Amazon SQS returns a success response.
Q: Is Amazon SQS PCI DSS certified?
Yes. Amazon SQS is PCI DSS Level 1 certified. For more information, see PCI Compliance.
Q: Is Amazon SQS HIPAA compliant?
No. Amazon SQS is not yet eligible for HIPAA compliance.
However, you can use the Extended Client Library to send Amazon SQS message payloads through Amazon S3 (Amazon S3 is an HIPAA-eligible service).You can achieve HIPAA compliance in this manner, because no personally identifiable information (PII) is transferred via Amazon SQS.
For more information, see Using the Amazon SQS Extended Client Library for Java.
Q: How long can I keep my messages in Amazon SQS message queues?
Longer message retention provides greater flexibility to allow for longer intervals between message production and consumption.
You can configure the Amazon SQS message retention period to a value from 1 minute to 14 days. The default is 4 days. Once the message retention limit is reached, your messages are automatically deleted.
Q: How do I configure Amazon SQS to support longer message retention?
To configure the message retention period, set the MessageRetentionPeriod attribute using the console or using the Distributiveness method. Use this attribute to specify the number of seconds a message will be retained in Amazon SQS.
You can use the MessageRetentionPeriod attribute to set the message retention period from 60 seconds (1 minute) to 1,209,600 seconds (14 days). For more information on working with this message attribute, see the Amazon SQS API Reference.
Q: How do I configure the maximum message size for Amazon SQS?
To configure the maximum message size, use the console or the SetQueueAttributes method to set the MaximumMessageSize attribute. This attribute specifies the limit on bytes that an Amazon SQS message can contain. Set this limit to a value between 1,024 bytes (1 KB), and 262,144 bytes (256 KB).
For more information, see Using Amazon SQS Message Attributes.
To send messages larger than 256 KB, use the Amazon SQS Extended Client Library for Java. This library lets you send an Amazon SQS message that contains a reference to a message payload in Amazon S3 that can be as large as 2 GB.
Q: What kind of data can I include in a message?
Amazon SQS messages can contain up to 256 KB of text data, including XML, JSON and unformatted text. The following Unicode characters are accepted:
#x9 | #xA | #xD | [#x20 to #xD7FF] | [#xE000 to #xFFFD] | [#x10000 to #x10FFFF]
For more information, see the XML 1.0 Specification.
Q: How large can Amazon SQS message queues be?
A single Amazon SQS message queue can contain an unlimited number of messages. However, there is a 120,000 limit for the number of inflight messages per queue. Messages are inflight after they have been received from the queue by a consuming component, but have not yet been deleted from the queue.
Q: How many message queues can I create?
You can create any number of message queues.
Q: Is there a size limit on the name of Amazon SQS message queues?
Queue names are limited to 80 characters.
Q: Are there restrictions on the names of Amazon SQS message queues?
You can use alphanumeric characters, hyphens (-), and underscores (_).
Q: Can I reuse a message queue name?
A message queue's name must be unique within an AWS account and region. You can reuse a message queue's name after you delete the message queue.
Q: What happens if there is no activity against a message queue for an extended period of time?
We reserve the right to delete a message queue if none of the following requests are issued against the message queue for more than 30 consecutive days:
Queues that act as dead letter queues are not deleted as long as any of their source queues still exist.
Q: How do I share a message queue?
You can associate an access policy statement (and specify the permissions granted) with the message queue to be shared. Amazon SQS provides APIs for creating and managing access policy statements:
For more information, see the Amazon SQS API Reference.
Q: Who pays for shared queue access?
The message queue owner pays for shared message queue access.
Q: How do I identify another AWS user I want to share a message queue with?
The Amazon SQS API uses the AWS account number to identify AWS users.
Q: What do I need to provide to an AWS user I want to share a message queue with?
To share a message queue with an AWS user, provide the full URL from the message queue you want to share. The CreateQueue and ListQueues operations return this URL in their responses.
Q: Does Amazon SQS support anonymous access?
Yes. You can configure an access policy that allows anonymous users to access a message queue.
Q: When should I use the permissions API?
The permissions API provides an interface for sharing access to a message queue to developers. However, this API cannot allow conditional access or more advanced use cases.
Q: When should I use the SetQueueAttributes operation with JSON objects?
The SetQueueAttributes operation supports the full access policy language. For example, you can use the policy language to restrict access to a message queue by IP address and time of day. For more information, see Amazon SQS Policy Examples.
Q: What regions is Amazon SQS available in?
For region availability, see the AWS Global Infrastructure Region Table.
Q: Can I share messages between queues in different regions?
No. Each Amazon SQS message queue is independent within each region.
Q: Is there a pricing difference between regions?
Amazon SQS pricing is the same for all regions, except Asia Pacific (Tokyo) and AWS GovCloud (US). For more information, see Amazon SQS Pricing.
Q: What is the pricing structure between various regions?
You can transfer data between Amazon SQS and Amazon EC2 or AWS Lambda free of charge within a single region.
When you transfer data between Amazon SQS and Amazon EC2 or AWS Lambda in different regions, you will be charged the normal data transfer rate. For more information, see Amazon SQS Pricing.