Goals

Get to know software patterns using queues to perform distributed processing of workloads.

Introduction

Context

Competing Consumers Pattern

In a queue-centric workflow pattern, work is placed in a queue and processed by multiple workers. Workers can retrieve work from the queue whenever they are ready to process it. Scaling is often based on the length of the queue or the average waiting time of a job in the queue (J). To scale, workers can be added or removed on demand.

Competing Consumers - Enterprise Integration Patterns

Example

Queues are scalable cloud services like the AWS Simple Queue Service (SQS):

From https://aws.amazon.com/sqs/

Fully Managed Message Queuing – Amazon Simple Queue Service – Amazon Web Services

System Stability

Let’s denote:

λ (lambda) as the average arrival rate of items per unit of time (e.g., customers per minute)
μ (mu) as the average service rate of items per unit of time (e.g., customers per minute)

Now we can say that a system is:

stable if λ < μ: the system is able to process all arriving items
unstable if λ ≥ μ: the system is receiving more items than it can service

In a stable system, the length of the queue is very short and not growing. All enqueued items are processed in a timely fashion. In an unstable system, the queue is getting longer and the processing time increases to infinity.

Example: Fast Service

Arrival rate λ = ****100 req/s
Service rate λ = ****200 req/s