RabbitMQ Publisher Confirms

Publisher confirms (also called publisher acknowledgements) let a producer know whether RabbitMQ has safely received and stored a message. By default, RabbitMQ does not confirm receipt to producers — a published message might silently fail due to a network blip or broker issue. Publisher confirms close this reliability gap on the producer side.

The Certified Mail Analogy

When you send a regular letter, you drop it in the mailbox and hope it arrives. When you send certified mail, the post office gives you a signed receipt confirming they have taken responsibility for delivery. Publisher confirms are RabbitMQ's version of certified mail — the broker signs off that it has received your message.

The Reliability Gap Without Confirms

Without publisher confirms:
  Producer ----publish----> RabbitMQ
                              |
                       network drops here?
                       broker disk full?
                       broker crashes?
                              |
                       Producer has no idea.
                       Message may be lost.

With publisher confirms:
  Producer ----publish----> RabbitMQ
                              |
                      RabbitMQ writes to disk
                              |
  Producer <-----ack--------- RabbitMQ
  "Message safely received."

Enabling Publisher Confirms

Publisher confirms are enabled per channel. Call confirm_select() on the channel to switch it into confirm mode. You cannot use a channel for both transactions and confirms — choose one.

channel.confirm_delivery()  # in pika, this enables publisher confirms

In Java (official RabbitMQ client):

channel.confirmSelect();

Three Implementation Strategies

Strategy 1: Publish One, Wait for Ack (Simple but Slow)

The producer publishes one message and blocks until RabbitMQ confirms it before publishing the next.

channel.confirm_delivery()
for message in messages:
    channel.basic_publish(exchange='', routing_key='my-queue', body=message)
    channel.wait_for_confirms_or_die()  # blocks until ack or raises on nack
    print("Message confirmed")

This guarantees every message is confirmed before sending the next. The downside is performance — the producer sends only one message at a time, waiting for a network round-trip per message.

Strategy 2: Publish in Batches

The producer publishes a batch of messages (say, 100 at once) and then waits for all of them to be confirmed together.

BATCH_SIZE = 100
channel.confirm_delivery()

for i, message in enumerate(messages):
    channel.basic_publish(exchange='', routing_key='my-queue', body=message)
    if (i + 1) % BATCH_SIZE == 0:
        channel.wait_for_confirms_or_die()
        print(f"Batch of {BATCH_SIZE} confirmed")

Batching is faster than one-at-a-time. The downside is that if a batch fails, you do not know which specific message in the batch was the problem — you must retry the entire batch.

Strategy 3: Asynchronous Confirms (Fastest)

The producer publishes messages continuously without blocking. Confirms arrive asynchronously via callback functions. This is the highest-throughput approach and is how production systems should work.

Track outstanding (unconfirmed) messages in a dictionary keyed by sequence number. The confirm callback removes confirmed messages from the outstanding set.

import threading
from collections import deque

outstanding = {}
lock = threading.Lock()

def ack_handler(delivery_tag, multiple):
    with lock:
        if multiple:
            keys_to_remove = [k for k in outstanding if k <= delivery_tag]
            for k in keys_to_remove:
                del outstanding[k]
        elif delivery_tag in outstanding:
            del outstanding[delivery_tag]

def nack_handler(delivery_tag, multiple):
    print(f"Message {delivery_tag} was nacked! Retry needed.")

channel.confirm_delivery()
channel.callbacks.add(channel.channel_number, '_on_delivery_confirmation',
                      lambda *args: ...)  # simplified; use official client

Most RabbitMQ client libraries provide first-class async confirm listeners. Check the documentation for your specific language client.

Sequence Numbers

In confirm mode, RabbitMQ assigns a monotonically increasing sequence number to each published message on a channel. Confirms include this sequence number so you know exactly which message was confirmed. Track these numbers to match confirms back to their original messages.

next_publish_seq = channel.get_next_publish_seq_no()  # Java client
// Send message with this seq number, store in map
// Confirm arrives with seq number → remove from map

Nack from the Broker

A nack from the broker (not to be confused with a consumer nack) means RabbitMQ received the message but could not enqueue it — for example because the queue is full or RabbitMQ ran out of memory. The producer must handle a broker nack by retrying the message or logging the failure.

Publisher Confirms vs Transactions

RabbitMQ also supports AMQP transactions (tx.select, tx.commit). Transactions guarantee that a batch of publishes is atomic. Publisher confirms are faster because they are asynchronous and do not lock broker resources. Transactions should be avoided in high-throughput systems. Use publisher confirms instead for all reliability requirements.

When to Use Publisher Confirms

  • Any time you cannot afford to lose a published message
  • Financial transactions, order events, audit records
  • Any system where a "fire and forget" approach would cause data loss

Summary

Publisher confirms give producers a reliable signal that RabbitMQ has successfully received and stored each message. Enable them with confirm_delivery() on the channel. Three strategies exist: one-at-a-time (simple but slow), batched (medium speed), and asynchronous callbacks (fastest). Handle broker nacks by retrying messages. Prefer publisher confirms over AMQP transactions for high-throughput, reliable publishing. Without confirms, message loss on the producer side is invisible and unavoidable.

Leave a Comment

Your email address will not be published. Required fields are marked *