Competing Consumers Pattern in RabbitMQ

The competing consumers pattern enables horizontal scaling of RabbitMQ processing. Multiple consumers subscribe to the same queue, and the broker delivers each message to exactly one consumer — they compete for work. This is fundamentally different from pub/sub, where each subscriber receives a copy.

The round-robin problem and the fix:

• By default, the broker distributes messages in round-robin regardless of consumer speed; a fast consumer and a slow one get alternating messages, leaving the fast one idle while the slow one builds a backlog
• The fix is prefetch tuning: setting basicQos limits how many unacknowledged messages each consumer holds; prefetch=1 gives the fairest dispatch — each consumer gets one message, processes it, acks, then gets the next, so work naturally flows to the fastest consumer

Scaling in Spring AMQP:

• Within a single instance, set concurrency and max-concurrency on the listener container to control thread count; the container creates multiple consumers on the same queue within the JVM
• Across instances, deploy multiple replicas in Kubernetes — each subscribes to the same queue and RabbitMQ treats them as additional competing consumers; KEDA can watch queue depth and adjust replicas automatically

The ordering caveat:

• Competing consumers break message ordering — Consumer A and Consumer B can process messages out of arrival sequence
• If ordering matters, use a single consumer per queue or partition work by key into separate queues, each with its own consumer

I recommend competing consumers as the default horizontal scaling strategy for task queues, with prefetch tuned to match your handler's processing capacity rather than relying on the default of 250.