Cassandra Write Path
The write path describes every step Cassandra takes when your application sends an INSERT or UPDATE. Cassandra is engineered for extremely fast writes — it achieves this by writing to memory first and to disk sequentially, avoiding the random I/O that makes writes slow in most databases.
Overview of the Write Path
Client
│
▼
Coordinator Node
│ (hashes partition key → routes to replica nodes)
│
├──▶ Replica Node A ──▶ Commit Log ──▶ MemTable
├──▶ Replica Node B ──▶ Commit Log ──▶ MemTable
└──▶ Replica Node C ──▶ Commit Log ──▶ MemTable
│
(When MemTable fills)
│
▼
SSTable (flushed to disk)
│
▼
Client ◀── write acknowledged
Step 1: Coordinator Routes the Write
Any node can receive a write request and act as coordinator. The coordinator hashes the partition key using Murmur3 to find the token, then identifies which replica nodes own that token range. It sends the write to all required replicas in parallel based on the consistency level.
Write: INSERT INTO orders VALUES (order-uuid, customer-uuid, 99.99) Partition key hash: order-uuid → Token 3,456,789 Token owner: Node B (primary), Node C (replica 1), Node D (replica 2) Coordinator sends write to B, C, D simultaneously. Consistency QUORUM (RF=3): waits for 2 acknowledgements.
Step 2: Commit Log Write (Durability)
On each replica node, the very first action is writing the mutation to the commit log — an append-only file on disk. This sequential write is extremely fast. The commit log is a safety net: if the node crashes before data flushes from memory to disk, Cassandra replays the commit log on restart and recovers all in-flight writes.
Commit Log: [2024-06-15 10:00:01] INSERT orders: order-uuid1 → 99.99 [2024-06-15 10:00:01] INSERT orders: order-uuid2 → 49.50 [2024-06-15 10:00:02] UPDATE customers: customer-uuid → email='new@mail.com' ... Append-only → sequential disk writes → very fast
Step 3: MemTable Write (Speed)
After the commit log, the write goes into the MemTable — an in-memory sorted data structure. The MemTable serves two purposes: it absorbs writes at RAM speed, and it serves as the source for recent reads. Each keyspace and table has its own MemTable.
MemTable for orders (sorted by partition key + clustering columns):
{order-uuid1 → {total: 99.99, status: 'pending'}}
{order-uuid2 → {total: 49.50, status: 'pending'}}
{order-uuid3 → {total: 15.00, status: 'shipped'}}
Step 4: Acknowledgement to Coordinator
Once the commit log write and MemTable write both succeed, the replica node sends an acknowledgement to the coordinator. The coordinator waits for enough acknowledgements to satisfy the consistency level, then notifies the client that the write succeeded.
Consistency QUORUM (RF=3, need 2 ACKs): Node B → ACK ✓ (1st ACK received) Node C → ACK ✓ (2nd ACK received — QUORUM satisfied) Node D → ACK ✓ (arrives later; noted but not needed) Coordinator → Client: "Write successful"
Step 5: MemTable Flush to SSTable
MemTables do not hold data forever. When a MemTable reaches a configured size threshold (or a time limit), Cassandra flushes it to disk as an SSTable. This flush is a sequential disk write — very fast compared to random I/O. Once flushed, the corresponding commit log segment is discarded because the data is safely on disk.
MemTable flush triggers: - MemTable size exceeds memtable_heap_space_in_mb (default: 1/4 heap) - nodetool flush is called manually - Cassandra is shutting down cleanly On flush: MemTable contents → written as new SSTable on disk Commit log segment → marked as discardable
SSTable Structure After Flush
An SSTable is immutable — once written, it never changes. Every flush creates a new SSTable. Over time, many SSTables accumulate and compaction merges them.
SSTable files created per flush: mb-1-big-Data.db (actual row data) mb-1-big-Index.db (partition index) mb-1-big-Filter.db (Bloom filter) mb-1-big-Summary.db (partition summary) mb-1-big-Statistics.db (metadata and histograms) mb-1-big-CompressionInfo.db mb-1-big-TOC.txt (table of contents)
Hinted Handoff for Offline Replicas
If a replica node is down when a write arrives, the coordinator stores the write as a "hint" and later delivers it when the replica comes back online. Hints are stored for a limited period (default: 3 hours).
Replica Node C is down during a write:
Coordinator stores hint:
{target: Node C, partition: order-uuid1, data: {...}}
Node C comes back online:
Coordinator delivers hint → Node C applies the write → in sync ✓
Write Performance Factors
Factor Effect
──────────────────────────────────────────────────────────────
Commit log on separate disk Eliminates I/O contention
between commit log and SSTables
Compression on SSTables Reduces disk I/O and storage cost
Large MemTable size Fewer flushes → fewer SSTables
→ fewer compactions
Consistency ONE Only 1 replica must ACK → lowest
latency writes
Avoid lightweight transactions Each LWT costs ~4× a normal write
due to Paxos coordination
Write Path vs Read Path Key Differences
Write Path Read Path ────────────────────────────────────────────────────────────── Sequential (fast) Random disk seeks (slower) Commit log + MemTable first SSTable + MemTable merge Immediate acknowledgement Must gather from replicas No index lookup needed Bloom filter + index needed Immutable SSTable created Multiple SSTables merged
Configuring Commit Log Location
For best write performance, place the commit log on a separate physical disk from the data directory. This prevents the commit log's sequential writes from competing with compaction's read/write activity on the data disk.
# cassandra.yaml commitlog_directory: /mnt/commitlog data_file_directories: - /mnt/data
Summary
Cassandra achieves fast writes by writing sequentially to the commit log for durability and to the MemTable for speed, then acknowledging the write to the client. MemTables flush to immutable SSTables on disk when they fill. Hinted handoff preserves writes destined for offline replicas. The entire write path involves no random disk seeks — only sequential appends — which is why Cassandra's write throughput is so high even on spinning disks.