Cassandra MemTable and SSTable
MemTables and SSTables are the two storage layers at the heart of Cassandra's write and read paths. MemTables hold recent writes in memory for fast access. SSTables hold persistent data on disk. Together they form a layered storage system called a Log-Structured Merge Tree (LSM tree) that makes Cassandra exceptionally fast for writes.
The Sticky Note and Filing Cabinet Analogy
When you receive new information during a meeting, you first jot it on a sticky note (MemTable — fast, in memory, temporary). When the meeting ends, you file those notes into the proper folders in your cabinet (SSTable — permanent, organized, on disk). You never erase old cabinet entries — you add new ones on top, and periodically clean up the cabinet by merging related folders (compaction).
MemTable
A MemTable is an in-memory, sorted data structure that holds the most recently written rows for a table. Every write goes to the MemTable immediately after the commit log, making writes extremely fast — RAM access is orders of magnitude faster than disk access.
MemTable properties: - Lives in JVM heap memory - One MemTable per table (active) + optional off-heap MemTables - Sorted by partition key, then clustering columns - Serves read queries for recent writes before they flush to disk - Flushed to SSTable when it reaches the size threshold
MemTable Flush Triggers
Trigger Configuration ────────────────────────────────────────────────────────────── Heap threshold exceeded memtable_heap_space_in_mb (default: 1/4 heap) Off-heap threshold exceeded memtable_offheap_space_in_mb Commit log size limit commitlog_total_space_in_mb Manual flush nodetool flush Clean shutdown automatic before exit
Configuring MemTable Allocation Type
# cassandra.yaml — where MemTable memory comes from: memtable_allocation_type: heap_buffers # default: JVM heap # OR memtable_allocation_type: offheap_buffers # off-heap (reduces GC pressure) # OR memtable_allocation_type: offheap_objects # fully off-heap rows
Reading from MemTable
When a read query arrives, Cassandra checks the MemTable first alongside SSTables. If a partition was recently written but not yet flushed, the MemTable provides the freshest version. The read path merges MemTable data with SSTable data using timestamps to determine the final result.
SSTable
An SSTable (Sorted String Table) is an immutable file written to disk when a MemTable flushes. Once written, an SSTable never changes. Updates and deletes create new SSTables, not modifications to existing ones. Compaction periodically merges multiple SSTables into one.
SSTable properties: - Immutable after creation - Sorted by partition key, then clustering columns - Compressed by default (LZ4 compression) - Contains Bloom filter, partition index, and data - Multiple SSTables can exist per table simultaneously
SSTable File Components
File Contents ────────────────────────────────────────────────────────────── Data.db Actual row data (columns and values) Index.db Partition key → byte offset mapping Summary.db Coarse index of the Index.db (in memory) Filter.db Bloom filter (probabilistic key lookup) Statistics.db Table metadata, histograms, min/max tokens CompressionInfo.db Compression chunk offsets TOC.txt List of all SSTable component files Digest.crc32 Checksum for data integrity verification
Data.db Layout
Data.db structure (simplified): [Partition Header: key=customer-uuid-A, token=-4.6Q] [Row: clustering=2024-01-05, col: total=99.99, status='pending'] [Row: clustering=2024-02-12, col: total=45.00, status='shipped'] [Partition Header: key=customer-uuid-B, token=-1.2Q] [Row: clustering=2024-01-20, col: total=80.00, status='pending'] ...
SSTable Compression
Cassandra compresses SSTables in chunks (default 64 KB per chunk). The CompressionInfo.db file maps each compressed chunk's offset so Cassandra can seek directly to any chunk without decompressing the entire file.
ALTER TABLE orders_by_customer
WITH compression = {
'class': 'LZ4Compressor', -- fastest; default since Cassandra 3.0
'chunk_length_in_kb': 64
};
-- Other compressor options:
-- 'SnappyCompressor' -- good balance of speed and ratio
-- 'DeflateCompressor' -- highest compression ratio, slower
-- 'ZstdCompressor' -- best ratio with tunable speed (Cassandra 4.0+)
-- 'NoopCompressor' -- disables compression
SSTable Accumulation Over Time
Each flush creates a new SSTable. Without compaction, the SSTable count grows indefinitely. A read query must check every SSTable for the requested partition — more SSTables means slower reads.
Day 1: [SST-1] Day 2: [SST-1] [SST-2] Day 3: [SST-1] [SST-2] [SST-3] ... Day 30: [SST-1] ... [SST-30] ← 30 files to check per read After compaction: Day 30: [SST-merged] ← 1 file to check per read
Viewing SSTable Information
# Count SSTables per table: nodetool tablestats ecommerce | grep "SSTable count" SSTable count: 8 # List SSTable files on disk: ls /var/lib/cassandra/data/ecommerce/orders_by_customer-*/ # Inspect SSTable metadata: sstablemetadata /var/lib/cassandra/data/ecommerce/ \ orders_by_customer-*/mb-1-big-Data.db
MemTable vs SSTable Quick Reference
Property MemTable SSTable ────────────────────────────────────────────────────────────── Location RAM (JVM heap or off-heap) Disk Mutability Mutable Immutable Lifetime Until flush Until compaction Write speed RAM speed (~ns) Disk speed (~ms) Read role Most recent writes Historical data Created by Write operations MemTable flush Removed by Flushing to SSTable Compaction
Off-Heap MemTables (Cassandra 2.1+)
By default, MemTable data lives in the JVM heap. Large MemTables increase garbage collection pressure, causing GC pauses that affect latency. Off-heap MemTables store row data outside the JVM heap, reducing GC pressure significantly for write-heavy workloads.
memtable_allocation_type: offheap_buffers
Summary
MemTables receive all writes in memory for speed, backed by the commit log for durability. When a MemTable reaches its size limit, it flushes to an immutable SSTable on disk. SSTables accumulate over time and compaction periodically merges them to maintain read performance. Compression reduces SSTable size and I/O cost with minimal CPU overhead. Understanding MemTable and SSTable behavior is essential for tuning Cassandra's memory settings, compression choices, and compaction strategy.