Cassandra Multi-Datacenter Setup
A multi-datacenter (multi-DC) Cassandra cluster stores copies of data in more than one physical location. This design enables geographic redundancy, disaster recovery, regional read performance, and workload isolation — all without any downtime and with zero application changes to switch between regions.
Why Run Multiple Data Centers
Goal Multi-DC Benefit ────────────────────────────────────────────────────────────── Disaster recovery Data survives full DC outage Low-latency regional reads Applications read from nearby DC Workload isolation Separate OLTP and analytics DCs Regulatory compliance Keep data within geographic borders Zero-downtime maintenance Take one DC offline; serve from others
Architecture Overview
US-East DC (RF=3) EU-West DC (RF=3)
┌──────────────────┐ ┌──────────────────┐
│ Node A Node B │ │ Node E Node F │
│ Node C Node D │ ◀────────▶ │ Node G Node H │
└──────────────────┘ Gossip └──────────────────┘
▲ ▲
│ LOCAL_QUORUM reads │ LOCAL_QUORUM reads
│ │
US App Servers EU App Servers
Step 1: Cassandra Node Configuration
Every node must know its own data center and rack. Configure this in cassandra-rackdc.properties on each node before it joins the cluster.
US-East Nodes (cassandra-rackdc.properties)
dc=us-east rack=rack1
EU-West Nodes (cassandra-rackdc.properties)
dc=eu-west rack=rack1
Step 2: Snitch Configuration
# cassandra.yaml on EVERY node: endpoint_snitch: GossipingPropertyFileSnitch
Step 3: Seed Node Configuration
Include at least one seed node from each data center in the seeds list on every node. This ensures new nodes can find the cluster regardless of which DC they start in.
# cassandra.yaml on EVERY node:
seed_provider:
- class_name: org.apache.cassandra.locator.SimpleSeedProvider
parameters:
- seeds: "10.0.0.1,10.1.0.1"
# ┬─────── US-East seed
# ┬────── EU-West seed
Step 4: Keyspace Replication with NetworkTopologyStrategy
Create or alter keyspaces to replicate across both data centers. The replication factor per DC determines how many replicas live in that region.
-- New keyspace for both DCs:
CREATE KEYSPACE ecommerce
WITH replication = {
'class': 'NetworkTopologyStrategy',
'us-east': 3,
'eu-west': 3
};
-- Alter existing keyspace to add a new DC:
ALTER KEYSPACE ecommerce
WITH replication = {
'class': 'NetworkTopologyStrategy',
'us-east': 3,
'eu-west': 3
};
-- After altering, rebuild the new DC nodes to stream data in:
nodetool rebuild -- us-east
Step 5: Application Consistency Level
Use LOCAL_QUORUM for all reads and writes. This satisfies a quorum within the local data center, avoiding cross-DC network round trips for every operation.
LOCAL_QUORUM behavior: Application in US-East: Write LOCAL_QUORUM → needs 2/3 US-East replicas to ACK → US-East replicas ACK → success in ~1 ms → EU-West replicas receive write asynchronously Application in EU-West: Write LOCAL_QUORUM → needs 2/3 EU-West replicas to ACK → EU-West replicas ACK → success in ~1 ms → US-East replicas receive write asynchronously
Cross-DC Write Propagation
After a LOCAL_QUORUM write succeeds in one DC, Cassandra replicates the mutation to the other DC asynchronously. The remote DC receives the data within milliseconds under normal network conditions. If the remote DC is unreachable, the coordinator stores a hint and delivers it when connectivity is restored.
US-East write (LOCAL_QUORUM):
Step 1: Coordinator (Node A) writes to Nodes B and C in US-East
→ Quorum satisfied → client notified
Step 2: Node A also sends write to EU-West asynchronously
→ EU-West Nodes E and F receive it
→ EU-West is now up to date (within ~50 ms)
Active-Active vs Active-Passive
Active-Active
Both data centers accept reads and writes from applications. Clients connect to their nearest DC. Both DCs replicate to each other asynchronously. This provides the lowest latency and highest availability.
Active-Passive
One DC accepts all writes (primary). The secondary DC receives replicated data and serves reads only. The secondary can promote to primary if the primary fails. Use this pattern when strict write ordering matters or to comply with data residency requirements.
Active-Passive setup: Primary DC (US-East): handles all reads + writes → replicates to → Secondary DC (EU-West): handles reads only (writes rejected locally; routed to US-East) On US-East failure: EU-West promotes → becomes primary DNS/load balancer updated → traffic rerouted
Rack Awareness in Multi-DC
Within each data center, use multiple racks so replicas spread across physical failure zones. Cassandra places replicas in different racks when using NetworkTopologyStrategy.
US-East DC with 2 racks and RF=3: Rack 1: Node A (replica 1) Rack 2: Node C (replica 2) Rack 1: Node E (replica 3 — back to rack 1 when no more racks) All 3 replicas survive a rack-level failure: Rack 2 goes down → replica 2 lost → 2 copies still available ✓
Dedicated Analytics Data Center
Add a third data center dedicated to Spark jobs, Kafka consumers, and batch analytics. Set its replication factor to 1 (no redundancy needed for analytics). Analytics traffic never contends with production OLTP traffic.
CREATE KEYSPACE ecommerce
WITH replication = {
'class': 'NetworkTopologyStrategy',
'us-east': 3, -- production
'eu-west': 3, -- production
'analytics': 1 -- Spark / batch jobs
};
-- Analytics DC nodes rebuild data from production:
nodetool rebuild -- us-east
Multi-DC Checklist
✓ GossipingPropertyFileSnitch configured on every node ✓ cassandra-rackdc.properties has correct dc and rack per node ✓ Seeds list includes nodes from every data center ✓ NetworkTopologyStrategy used for all production keyspaces ✓ system_auth replicated to all DCs (RF ≥ 3 per DC) ✓ Applications use LOCAL_QUORUM consistency ✓ nodetool repair runs regularly in every DC ✓ Monitor cross-DC replication lag and hinted handoff queues
Summary
A multi-datacenter Cassandra cluster spreads replicas across geographic regions for disaster recovery, low-latency regional access, and workload isolation. Configure GossipingPropertyFileSnitch with per-node DC and rack settings. Use NetworkTopologyStrategy with a replication factor per DC. Include seed nodes from every DC in the seeds list. Set application consistency to LOCAL_QUORUM to avoid cross-DC latency on every operation. Add a dedicated analytics DC at RF=1 to isolate Spark and batch workloads from production traffic.