RabbitMQ Clustering Basics
A RabbitMQ cluster is a group of two or more RabbitMQ nodes that work together as a single logical broker. Clustering increases capacity, enables failover, and eliminates single points of failure. When one node in a cluster goes down, clients can reconnect to another node and continue working.
The Office Building Analogy
A single RabbitMQ node is like a one-person office. If that person gets sick, the whole office shuts down. A cluster is like an office with many employees. Each employee can handle client calls. If one is out, the others keep working. The office never closes because no single employee is irreplaceable.
What Is Shared in a Cluster
All nodes in a cluster share the same metadata:
- Virtual hosts
- Exchange definitions
- Queue definitions (names, properties)
- Bindings
- Users and permissions
- Policies
What is NOT automatically shared:
- Queue contents (messages inside a queue live on one node by default)
Cluster with 3 nodes: Node1: knows about all exchanges, all queues, all users Node2: knows about all exchanges, all queues, all users Node3: knows about all exchanges, all queues, all users But queue "orders" has its messages stored ONLY on Node1. If Node1 goes down, the orders queue is unavailable (without Quorum Queues).
Forming a Cluster
To join two nodes into a cluster, run on Node2:
# Step 1: Stop the RabbitMQ app on Node2 rabbitmqctl stop_app # Step 2: Reset Node2 (clears its data, prepares it to join) rabbitmqctl reset # Step 3: Join Node2 to the cluster at Node1 rabbitmqctl join_cluster rabbit@node1 # Step 4: Start the RabbitMQ app on Node2 rabbitmqctl start_app
Repeat steps 1–4 on Node3, joining it to rabbit@node1 as well. All three nodes now form one cluster.
Prerequisites for Clustering
- Same Erlang cookie: All nodes must share the same Erlang cookie (
~/.erlang.cookie). Copy the cookie from Node1 to Node2 and Node3 before forming the cluster. - Network connectivity: All nodes must be able to reach each other on ports 4369 (Erlang port mapper) and 25672 (inter-node communication).
- Resolvable hostnames: Nodes use hostnames to communicate. Add entries to
/etc/hostsor use DNS to ensure every node can resolve every other node's hostname. - Same RabbitMQ and Erlang versions: Mixed versions in a cluster cause errors. Always run the same version across all nodes.
Cluster Node Types
Disc Node
A disc node saves cluster metadata to disk. At least one disc node must be in the cluster at all times. If all disc nodes are down simultaneously, the cluster cannot start. Most clusters use all disc nodes.
RAM Node
A RAM node stores metadata in memory only. RAM nodes have slightly lower latency for metadata operations but they are not recommended for most setups because they do not help with durability. They are occasionally used in very large clusters where metadata replication overhead is significant.
# Join as a RAM node (rarely needed) rabbitmqctl join_cluster --ram rabbit@node1 # Join as a disc node (default, recommended) rabbitmqctl join_cluster rabbit@node1
Viewing Cluster Status
rabbitmqctl cluster_status
Output:
Cluster status of node rabbit@node2
[{nodes,[{disc,[rabbit@node1,rabbit@node2,rabbit@node3]}]},
{running_nodes,[rabbit@node1,rabbit@node2,rabbit@node3]},
{cluster_name,<<"rabbit@node1">>},
{partitions,[]}]
All three nodes are listed under disc and all are in running_nodes. The partitions list should always be empty — a non-empty list indicates a network partition (split-brain scenario).
Removing a Node from the Cluster
# On the node you want to remove: rabbitmqctl stop_app rabbitmqctl reset rabbitmqctl start_app # Or remove a stopped node from another cluster member: rabbitmqctl forget_cluster_node rabbit@node3
Client Connections in a Cluster
Clients can connect to any node in the cluster. If a client connects to Node2 but wants to consume from a queue that lives on Node1, Node2 transparently proxies the connection to Node1. Clients do not need to know which node holds which queue.
Client A connects to Node2 --> Node2 proxies to Node1 (where orders queue lives) Client B connects to Node3 --> Node3 proxies to Node1 (same queue)
Load Balancing Across Cluster Nodes
Use a load balancer (HAProxy, AWS ELB, Nginx) in front of the cluster so clients connect to a virtual IP. The load balancer distributes connections across available nodes and removes a failed node from rotation automatically. Clients always connect to the load balancer address rather than individual node addresses.
Queue Location in a Cluster
By default, a queue's messages live on the node where it was first declared. All other nodes know the queue exists (metadata is shared) but do not hold its messages. This means node failure makes that queue's messages temporarily unavailable. Use Quorum Queues (covered in the next topics) to replicate queue contents across nodes for true high availability.
Summary
A RabbitMQ cluster connects multiple nodes into one logical broker. All nodes share metadata (users, exchanges, queue definitions) but queue messages are stored on one node by default. Form a cluster by stopping a node, resetting it, joining it to an existing node, and starting it again. All nodes must share the same Erlang cookie and must be able to reach each other on the network. Use a load balancer in front of the cluster for transparent failover. Use Quorum Queues for high-availability message storage across nodes.