Amazon Simple Queue Service (Amazon SQS) offers a reliable, highly scalable, hosted queue for storing messages as they travel between computers. By using Amazon SQS, developers can simply move data between distributed components of their applications that perform different tasks, without losing messages or requiring each component to be always available. Amazon SQS makes it easy to build an automated workflow, working in close conjunction with the Amazon Elastic Compute Cloud (Amazon EC2) and the other AWS infrastructure web services.
Amazon SQS works by exposing Amazon’s web-scale messaging infrastructure as a web service. Any computer on the Internet can add or read messages without any installed software or special firewall configurations. Components of applications using Amazon SQS can run independently, and do not need to be on the same network, developed with the same technologies, or running at the same time.
Developers can create an unlimited number of Amazon SQS queues with an unlimited number of messages.
A queue can be created in any region.
The message body can contain up to 64 KB of text in any format.
Messages can be sent, received or deleted in batches of up to 10 messages or 64kb. Batches cost the same amount as single messages, meaning it can be even more cost effective for customers that use it.
Messages can be retained in queues for up to 14 days.
Messages can be sent and read simultaneously.
When a message is received, it becomes “locked” while being processed. This keeps other computers from processing the message simultaneously. If the message processing fails, the lock will expire and the message will be available again. In the case where the application needs more time for processing, the “lock” timeout can be changed dynamically via the ChangeMessageVisibility operation.
Developers can securely share Amazon SQS queues with others. Queues can be shared with other AWS accounts and Anonymously. Queue sharing can also be restricted by IP address and time-of-day.
When combined with Amazon Simple Notification Service (SNS), developers can 'fanout' identical messages to multiple SQS queues in parallel. When developers want to process the messages in multiple passes, fanout helps complete this more quickly, and with fewer delays due to bottlenecks at any one stage. Fanout also makes it easier to record duplicate copies of your messages, for example in different databases.
Amazon SQS provides significant advantages over the complexity of home-grown messaging systems or the ongoing expense of licensed or hosted systems.
Reliable Amazon SQS runs within Amazon’s high-availability data centers, so queues will be available whenever applications need them. To prevent messages from being lost or becoming unavailable, all messages are stored redundantly across multiple servers and data centers.
Simple Developers can get started with Amazon SQS by using only five APIs: CreateQueue, SendMessage, ReceiveMessage, ChangeMessageVisibility, and DeleteMessage. Additional APIs are available to provide advanced functionality.
Scalable Amazon SQS was designed to enable an unlimited number of services to read and write an unlimited number of messages at any time.
Secure Authentication mechanisms are provided to ensure that messages stored in Amazon SQS queues are secured against unauthorized access.
Inexpensive No up-front or fixed expenses. The only costs of sending messages through Amazon SQS are small per-request handling fees and data transfer fees.
Using Amazon SQS with Other AWS Infrastructure Web Services
Amazon SQS can be used with Amazon EC2, as well as Amazon S3 and Amazon SimpleDB, to make applications more flexible and scalable. A common use case is to create an integrated and automated workflow, where multiple components or modules need to communicate with each other, but can’t all process the same amount of work simultaneously. In this case, SQS queues carry messages to be processed in an orderly fashion by the user’s application running on Amazon EC2 instances. The Amazon EC2 instances can read the queue, process the job, and then post the results as messages to another SQS queue (possibly for further processing by another application). Because Amazon EC2 allows applications to scale up and down dynamically, application developers can easily vary the number of compute instances based on the amount of work in the SQS queues, to ensure that jobs are executed in a timely manner.
For example, here is how a video transcoding website uses Amazon EC2, Amazon SQS, Amazon S3, and Amazon SimpleDB together. End users submit videos to be transcoded to the website. The videos are stored in Amazon S3, and a message (“the request message”) is placed in an Amazon SQS queue (“the incoming queue”) with a pointer to the video and to the target video format in the message. The transcoding engine, running on a set of Amazon EC2 instances, reads the request message from the incoming queue, retrieves the video from Amazon S3 using the pointer, and transcodes the video into the target format. The converted video is put back into Amazon S3 and another message (“the response message”) is placed in another Amazon SQS queue (“the outgoing queue”) with a pointer to the converted video. At the same time, metadata about the video (e.g., format, date created and length) can be indexed into Amazon SimpleDB for easy querying. During this whole workflow, a dedicated Amazon EC2 instance can constantly monitor the incoming queue and, based on the number of messages in the incoming queue, is able to dynamically adjust the number of transcoding Amazon EC2 instances to meet customers’ response time requirements.
You can get started with Amazon SQS for free. New and existing
customers receive 1 million Amazon SQS queuing requests for free each
month. Some applications may be able to operate within this free
$0.50 per 1 million Amazon SQS Requests ($0.0000005 per SQS Request)
Batches of up to 10 messages or 64KB per request cost the same amount as single messages.
The pricing below is based on data transferred “in” and “out” of Amazon SQS.
Data transfer “in” and “out” refers to
transfer into and out of Amazon SQS. Data transferred between Amazon
SQS and Amazon EC2 within a single region is free of charge (i.e.,
$0.00 per GB). Data transferred between Amazon SQS and Amazon EC2 in
different regions will be charged at Internet Data Transfer rates on
both sides of the transfer.
* Your free usage is calculated each month across all regions and automatically applied to your bill – free usage does not accumulate. ** As part of AWS’s Free Usage Tier, new AWS customers will receive free 15 GB of data transfer out each month aggregated across all AWS services for one year. *** Data transfer out rate tiers aggregate outbound data transfer across Amazon EC2, Amazon EBS, Amazon S3, Amazon Glacier, Amazon RDS, Amazon SimpleDB, Amazon SQS, Amazon SNS, Amazon DynamoDB, AWS Storage Gateway, and Amazon VPC.
Amazon SQS can be easily accessed through the AWS Management Console, which gives users a point-and-click, web-based interface to setup and manage Amazon SQS.
Basic Queue Requests
Amazon SQS employs a simple interface that is easy to use and highly flexible. The following requests are provided:
CreateQueue: Create queues for use with your AWS account.
ListQueues: List your existing queues.
DeleteQueue: Delete one of your queues.
SendMessage: Add messages to a specified queue.
SendMessageBatch: Add multiple messages to a specified queue.
ReceiveMessage: Return one or more messages from a specified queue.
ChangeMessageVisibility: Change the visibility timeout of previously received message.
ChangeMessageVisibilityBatch: Change the visibility timeout of multiple previously received messages.
DeleteMessage: Remove a previously received message from a specified queue.
DeleteMessageBatch: Remove multiple previously received messages from a specified queue.
SetQueueAttributes: Control queue settings like the amount of time that messages are locked after being read so they cannot be read again.
GetQueueAttributes: Get information about a queue like the number of messages in it.
GetQueueUrl: Get the queue URL.
AddPermission: Add queue sharing for another AWS account for a specified queue.
RemovePermission: Remove an AWS account from queue sharing for a specified queue.
Amazon SQS Message Lifecycle
Messages that are stored in Amazon SQS have a lifecycle that is easy to manage but ensures that all messages are processed.
A system that needs to send a message will find an Amazon SQS queue, and use SendMessage to add a new message to it.
A different system that processes messages needs more messages to process, so it calls ReceiveMessage, and this message is returned.
Once a message has been returned by ReceiveMessage, it will not be returned by any other ReceiveMessage until the visibility timeout has passed. This keeps multiple computers from processing the same message at once.
If the system that processes messages successfully finishes working with this message, it calls DeleteMessage, which removes the message from the queue so no one else will ever process it. If this system fails to process the message, then it will be read by another ReceiveMessage call as soon as the visibility timeout passes.