RabbitMQ Shovel Plugin
The Shovel plugin moves messages from a source queue on one RabbitMQ broker to a destination exchange on the same or a different broker. It works like a conveyor belt — continuously picking up messages from one end and placing them at another. Shovel is the simplest tool for bridging two separate RabbitMQ installations or moving messages between environments.
The Conveyor Belt Analogy
A factory has two production lines separated by a wall. A conveyor belt (Shovel) passes through a hole in the wall, picking up finished parts from Line A and delivering them to Line B. The conveyor runs automatically, round the clock, without a worker manually carrying each part. If the conveyor stops, no parts cross. When it restarts, it picks up where it left off.
What Shovel Does
Broker A (source) Broker B (destination)
----------------------------- ---------------------------
[Queue: orders-source] ----Shovel----> [Exchange: orders-dest]
|
[Queue: orders-dest-q]
The Shovel reads messages from the source queue, publishes them to the destination exchange, and waits for acknowledgement from both sides before removing the message from the source. This ensures no message is lost during transit.
Enabling the Shovel Plugin
rabbitmq-plugins enable rabbitmq_shovel rabbitmq-plugins enable rabbitmq_shovel_management # adds Shovel UI in management
Configuring a Shovel via the Management UI
- Go to the Admin tab in the Management UI.
- Click "Shovel Management" in the left sidebar.
- Click "Add a new shovel."
- Enter the shovel name, source URI, source queue, destination URI, and destination exchange.
- Click "Add shovel."
Configuring a Shovel via rabbitmq.conf (Static Shovel)
A static shovel is defined in the configuration file and starts automatically when RabbitMQ starts:
# advanced.config (Erlang format)
[
{rabbitmq_shovel, [
{shovels, [
{my_shovel, [
{source, [
{uris, ["amqp://user:pass@broker-a.example.com"]},
{queue, <<"orders-source">>}
]},
{destination, [
{uris, ["amqp://user:pass@broker-b.example.com"]},
{exchange, <<"orders-dest">>},
{routing_key,<<"order.new">>}
]},
{ack_mode, on_confirm},
{reconnect_delay, 5}
]}
]}
]}
].
Configuring a Dynamic Shovel via HTTP API
Dynamic shovels are created at runtime via the HTTP API or Management UI without restarting the broker:
curl -u guest:guest -X PUT \
http://localhost:15672/api/parameters/shovel/%2F/my-dynamic-shovel \
-H "Content-Type: application/json" \
-d '{
"value": {
"src-protocol": "amqp091",
"src-uri": "amqp://localhost",
"src-queue": "orders-source",
"dest-protocol": "amqp091",
"dest-uri": "amqp://broker-b.example.com",
"dest-exchange": "orders-dest",
"ack-mode": "on-confirm"
}
}'
Acknowledgement Modes
The ack-mode setting controls when the Shovel acknowledges messages at the source:
on-confirm (safest): Wait for the destination broker to confirm receipt before acking at the source. No message loss possible. Lowest throughput. on-publish (faster): Ack at source as soon as the message is published to destination. Small window of message loss if destination crashes right after publish. no-ack (fastest): Ack at source immediately upon reading. Message loss possible if Shovel crashes before delivery. Only use for truly non-critical data.
Reconnection Behaviour
If either the source or destination broker is unavailable, the Shovel pauses and retries the connection. Set reconnect_delay (in seconds) to control how long it waits between retry attempts. The Shovel resumes automatically when both connections are available.
reconnect_delay = 10 # retry every 10 seconds
Shovel Status
Check the status of all running shovels:
rabbitmqctl shovel_status
Name Vhost Type State my-dynamic-shovel / dynamic running
State can be running, starting, stopped, or terminated. A terminated shovel has given up after exhausting reconnection attempts.
Use Cases for Shovel
- Cross-datacenter replication: Move messages from a broker in Region A to a broker in Region B for disaster recovery.
- Environment promotion: Move messages from a staging broker to production after testing.
- Broker migration: Move all messages from an old broker to a new one during an infrastructure upgrade without downtime.
- Queue draining: Empty a queue on one broker and republish to a different queue or exchange without losing messages.
Shovel vs Federation
Shovel Federation ----------------------------------------- Moves messages (consumes) Subscribes (messages can stay) Source queue drains Source queue unchanged Point-to-point Multi-hop possible Simpler configuration More flexible topology Best for one-way bulk transfer Best for distributed pub/sub
Summary
The Shovel plugin continuously moves messages from a source queue to a destination exchange, even across different RabbitMQ brokers. Enable it with rabbitmq-plugins enable rabbitmq_shovel. Use on-confirm ack mode for zero message loss. Static shovels are defined in advanced.config and start with the broker. Dynamic shovels are created at runtime via the HTTP API or Management UI. Shovel is the simplest tool for cross-broker message transfer, queue draining, and broker migration.