Amazon SQS




Below sourceQueue is the queue message are published to by the publisher and deadQueue is the dead letter queue (DLQ) use by the sourceQueue.

  • Types are Standard and FIFO, sourceQueue and deadQueue need to have matching types.
  • sourceQueue and deadQueue must be on the same region of the AWS Account
  • Message Retention, the max value is 14 days for any queue and the default is 4 days.

Some testing code to understand the magic.

Source Queue

  • Visibility timeout, once a message is received by a consumer it is not visible to other consumers for the period of this timeout. Useful when your consumer application scales in K8s
  • RedrivePolicy, specifies the source queue, the dead-letter queue and the conditions under which Amazon SQS moves messages from sourceQeueu to deadQueue if the consumer of the sourceQueue fails to process a message. This will contain the ARN of the deadQueue and the value of the maxReceiveCount
  • maxReceiveCount, used by the redrive policy to know how many times the message can be received before being auto-magically pushed to the deadQueue
  • Heartbeat, used when you dont know how long it will take the consumer to process the message once receieved. Specify the initial visibility timeout (for example, 2 minutes) and then—as long as your consumer still works on the message—keep extending the visibility timeout by 2 minutes every minute.
  • Delete, once processed the consumer must manually delete the message, else it will increment maxReceiveCount

Dead Letter Queue

  • Retention Period, the expiration of a message is always based on its orginal enqueue timestamp. So it starts counting from the sourceQueue. The period in the deadQueue must always be larger than the sourceQueue.
  • Helps with message failures to isolate the failed message
  • Replay Policy, this is manual, AWS suggest using a Lambda. A message could fail for a number of reasons including downstream dependancy borked or serialization failure.