Amazon Simple Queue Service

Amazon Simple Queue Service (SQS) is a fully managed message queuing service offered by AWS. It allows us to send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.

Key Features

• Decoupling: It helps to decouple and connect microservices, distributed systems, and serverless applications.
• Scalability: SQS can handle large volumes of data and scale elastically based on usage.
• Reliability: It ensures that messages are reliably delivered and stored across multiple servers.
• Security: SQS provides secure message transmission and storage using AWS Key Management Service (KMS).
• Cost-effective: We only pay for what we use, making it a cost-effective solution for message queuing.

How It Works

1. Creating a Queue: You start by creating a queue in SQS. This queue acts as a temporary repository for messages that are awaiting processing.

2. Sending Messages: Applications or services send messages to the queue. These messages can contain any type of information that needs to be processed by another component.

3. Storing Messages: SQS stores these messages redundantly across multiple servers to ensure high availability and durability.

4. Receiving Messages: The receiving component retrieves messages from the queue. This can be done on-demand or by polling the queue at regular intervals.

5. Processing Messages: Once a message is retrieved, it is processed by the receiving component. After processing, the message is deleted from the queue to prevent it from being processed again.

6. Visibility Timeout: SQS provides a visibility timeout feature, which ensures that once a message is retrieved, it is not visible to other components for a specified period. This prevents multiple components from processing the same message simultaneously.

7. Dead-Letter Queues: SQS supports dead-letter queues, which are used to store messages that cannot be processed successfully after a specified number of attempts. This helps in debugging and handling errors gracefully.

Types of queues

Standard Queues

• High Throughput: Standard queues support a very high, nearly unlimited number of API calls per second, making them ideal for processing large volumes of messages quickly.
• At-Least-Once Delivery: Messages are delivered at least once, but there may be occasional duplicates.
• Best-Effort Ordering: While SQS attempts to deliver messages in the order they were sent, it does not guarantee this. Messages may arrive out of order, especially under high throughput or failure recovery conditions.
• Durability and Redundancy: Messages are stored redundantly across multiple AWS Availability Zones to ensure high durability.
• Visibility Timeout: We can configure a visibility timeout to control how long a message stays hidden after being received, ensuring that other consumers do not process the message until it has been fully handled or the timeout expires.

FIFO Queues

• Exactly-Once Processing: FIFO queues ensure that each message is processed exactly once and remains available until it is successfully processed and deleted.
• Preserved Order: Messages are processed in the exact order they are sent, making FIFO queues ideal for applications where the order of operations is crucial.
• High Throughput Mode: FIFO queues can process up to 3,000 messages per second per API method when using batching. Without batching, they support up to 300 API calls per second per API method.
• Automatic Deduplication: FIFO queues handle automatic deduplication of messages, ensuring that duplicate messages are not processed.

Reference:

• Message Queuing Service - Amazon Simple Queue Service - AWS
• What is Amazon Simple Queue Service
• What is AWS SQS (Simple Queue Service)?: 5 Comprehensive Aspects
• AWS Simple Queue Service(SQS) - GeeksforGeeks
• AWS SNS vs. SQS - What Are the Main Differences? - AWS Fundamentals
• en.wikipedia.org

Related Posts: Google Cloud Pub/Sub