RabbitMQ Request Reply Pattern

The request-reply pattern (also called RPC over RabbitMQ) allows a client to send a request message and wait for a corresponding reply from a server. Unlike fire-and-forget messaging where the producer moves on immediately, request-reply means the client expects a response before continuing. RabbitMQ provides the plumbing to do this asynchronously and safely.

The Customer Support Analogy

You call a customer support line (send a request). The agent puts you on hold (async waiting), looks up your account, and comes back with an answer (sends the reply). You stay on the line (block or listen) until you get your answer. Request-reply in RabbitMQ follows the same flow, but messages travel through queues instead of a phone system.

How Request-Reply Works

[Client (Requester)]
    |
    | 1. Publishes request message to "rpc-queue"
    |    with reply_to="amq.gen-ABC" (client's temp queue)
    |    and correlation_id="req-001"
    v
[rpc-queue]
    |
    | 2. Server picks up request
    v
[Server (Responder)]
    |
    | 3. Processes request
    | 4. Publishes reply to "amq.gen-ABC"
    |    with correlation_id="req-001"
    v
[Client's reply queue: amq.gen-ABC]
    |
    | 5. Client reads reply, matches correlation_id
    v
[Client continues processing]

Two Essential Message Properties

reply_to

The client creates a temporary exclusive queue just for receiving the reply. It sets the reply_to property on the request message to tell the server where to send the response.

correlation_id

The client sets a unique identifier (like a UUID) on the request. The server copies this value onto the reply. The client uses the correlation_id to match the incoming reply to the original request. This is essential when the client sends many requests — it must know which reply belongs to which request.

Why a Temporary Queue for Replies

The client declares an exclusive, auto-delete queue with a server-generated name. This queue exists only for the lifetime of the client's connection. Each client gets its own reply queue, so replies from different clients never mix together. When the client disconnects, the reply queue disappears automatically.

result = channel.queue_declare(queue='', exclusive=True)
callback_queue = result.method.queue

Correlation ID Matching

Without correlation IDs, the client cannot distinguish between replies if multiple requests are in flight simultaneously. Every reply carries the same correlation_id as its request. The client ignores replies whose IDs do not match any pending request.

Client sends:
  Request 1 -- correlation_id = "req-001"
  Request 2 -- correlation_id = "req-002"

Server replies:
  Reply for req-002 arrives first
    --> client sees "req-002", matches request 2
  Reply for req-001 arrives second
    --> client sees "req-001", matches request 1

Blocking vs Non-Blocking Client

Blocking (Synchronous Waiting)

The simplest implementation: the client publishes a request and then polls its reply queue in a loop until a matching reply arrives. Simple but inefficient — the client cannot do other work while waiting.

Non-Blocking (Callback-Based)

The client publishes the request and sets a callback on the reply queue. When a reply arrives, the callback fires. The client continues doing other work in the meantime. This approach is more complex to code but far more efficient.

Complete Python Example (Blocking Client)

Server (processes requests and replies):

import pika

def on_request(ch, method, props, body):
    n = int(body)
    result = n * n  # example: returns square of the number
    ch.basic_publish(
        exchange='',
        routing_key=props.reply_to,
        properties=pika.BasicProperties(
            correlation_id=props.correlation_id
        ),
        body=str(result)
    )
    ch.basic_ack(delivery_tag=method.delivery_tag)

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.queue_declare(queue='rpc-queue')
channel.basic_qos(prefetch_count=1)
channel.basic_consume(queue='rpc-queue', on_message_callback=on_request)
channel.start_consuming()

Client (sends request and waits for reply):

import pika, uuid

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()

result = channel.queue_declare(queue='', exclusive=True)
callback_queue = result.method.queue

response = None
corr_id = str(uuid.uuid4())

def on_response(ch, method, props, body):
    global response
    if corr_id == props.correlation_id:
        response = int(body)

channel.basic_consume(
    queue=callback_queue,
    on_message_callback=on_response,
    auto_ack=True
)

channel.basic_publish(
    exchange='',
    routing_key='rpc-queue',
    properties=pika.BasicProperties(
        reply_to=callback_queue,
        correlation_id=corr_id
    ),
    body=str(9)
)

while response is None:
    connection.process_data_events()

print(f"Square of 9 is {response}")
connection.close()

Timeout Handling

A well-designed client does not wait forever. If the server crashes or takes too long, the client should time out and either retry or return an error. Add a timeout counter to the waiting loop:

import time
timeout = 10  # seconds
start = time.time()
while response is None:
    if time.time() - start > timeout:
        raise TimeoutError("No reply received within 10 seconds")
    connection.process_data_events(time_limit=1)

When to Use Request-Reply

When a service needs a result from another service before it can continue
When building internal APIs between microservices without a synchronous HTTP call
When you want the reliability of a message broker (retries, persistence) but still need a response

Trade-offs

Request-reply adds latency compared to fire-and-forget. The client waits for a round trip through two queues. If the server is slow, the client blocks (or times out). Use this pattern only when the client genuinely needs the result before proceeding. For one-way notifications or background tasks, the work queue or pub/sub pattern is a better fit.

Summary

The request-reply pattern enables synchronous-style communication over RabbitMQ's asynchronous infrastructure. The client creates a temporary reply queue, attaches its name via reply_to, and sets a unique correlation_id. The server processes the request and sends the reply to the specified queue with the same correlation_id. The client matches the reply using the ID. Add timeouts to avoid hanging forever if the server fails. This pattern is ideal for internal microservice calls that need a result before they can proceed.

Previous lesson

Back to course

Next lesson