Cassandra Tombstones

A tombstone is a deletion marker that Cassandra writes to disk when you delete data. Instead of erasing a value immediately, Cassandra records a tombstone that says "this data was deleted at this timestamp." The actual data removal happens later during compaction. Understanding tombstones prevents common performance problems in production clusters.

Why Tombstones Exist

Cassandra is a distributed system. When you delete a row, that deletion must reach every replica — including replicas on nodes that are temporarily offline. If Cassandra deleted data immediately and a replica node was offline at the time, that node would still hold the old data and serve it to clients after coming back online, making deleted data reappear.

Tombstones solve this problem. They are durable deletion markers that travel to offline replicas during repair and prevent deleted data from resurging.

Scenario: delete a row with Node B offline

Node A: write tombstone ✓
Node B: OFFLINE (will receive tombstone during repair)
Node C: write tombstone ✓

Client reads from Node A or C → row appears deleted ✓
Node B comes back → repair delivers tombstone → row stays deleted ✓

Types of Tombstones

Tombstone Type       Created By
──────────────────────────────────────────────────────────────
Cell tombstone       DELETE on a single column value
Row tombstone        DELETE on a full row
Range tombstone      DELETE with clustering column range
Partition tombstone  DELETE with only partition key specified
TTL tombstone        Automatic expiration when TTL expires
Collection tombstone DELETE on a collection column

Cell Tombstone

DELETE phone FROM customers WHERE customer_id = [uuid];

Row Tombstone

DELETE FROM customers WHERE customer_id = [uuid];

Range Tombstone

DELETE FROM orders_by_customer
WHERE customer_id = [uuid]
  AND order_date >= '2024-01-01'
  AND order_date <  '2024-04-01';

TTL Tombstone

When a row or column expires because its TTL runs out, Cassandra generates a tombstone automatically. This tombstone behaves identically to a manually written tombstone.

gc_grace_seconds — The Tombstone Lifetime

The gc_grace_seconds table property controls how long Cassandra keeps tombstones before compaction can remove them. The default is 864,000 seconds — ten days. This window gives offline replica nodes enough time to come back online and receive the deletion via repair.

Tombstone lifecycle:

t=0      DELETE written → tombstone created
t=5 days Node B comes back online → repair delivers tombstone ✓
t=10d    gc_grace_seconds expires → tombstone eligible for removal
t=11d    Compaction runs → tombstone purged from disk ✓

ALTER TABLE events WITH gc_grace_seconds = 86400;  -- 1 day

Only shorten gc_grace_seconds if you run frequent repairs and your nodes rarely stay offline for more than a day. Setting it too low risks deleted data reappearing from a replica that missed the deletion window.

Tombstone Accumulation Problems

Tombstones become a performance problem when too many accumulate in a partition. Every read query must scan through tombstones to determine which data is still live. A partition with millions of tombstones causes slow reads and high memory usage.

Partition with tombstone accumulation:

Read query scans:
  Cell 1: tombstone (skip)
  Cell 2: tombstone (skip)
  Cell 3: tombstone (skip)
  ...
  Cell 999: tombstone (skip)
  Cell 1000: live data ← finally found

Performance cost: 999 unnecessary reads before reaching live data

Tombstone Warning in Logs

WARN  [ReadStage] Read 50 live rows and 3287 tombstone cells for
query SELECT * FROM events WHERE user_id = ...

Cassandra logs this warning when a read scans more than tombstone_warn_threshold tombstones (default: 1,000). When it scans more than tombstone_failure_threshold (default: 100,000), the query fails.

Avoiding Tombstone Accumulation

Anti-Pattern                        Better Approach
──────────────────────────────────────────────────────────────────
Inserting NULL values               Omit the column entirely
Repeatedly deleting and rewriting   Redesign to avoid churn
  collection items                  or use a separate table
Deleting individual cells in a      Delete the entire row or
  high-write table                  use TTL instead
Using a queue pattern (delete       Use a dedicated queue system;
  after read) in Cassandra          Cassandra is not ideal for
                                    queue workloads

Dropping Tombstones Early with nodetool

You can force compaction to run immediately on a table to trigger tombstone cleanup after the gc_grace_seconds window has passed:

nodetool compact mykeyspace mytable

Tombstones younger than gc_grace_seconds will not be removed even by a forced compaction — the window must have expired first.

Checking Tombstone Counts

# View SSTable statistics including tombstone counts:
nodetool tablehistograms mykeyspace mytable

# Check tombstone count in a specific SSTable:
sstablemetadata /var/lib/cassandra/data/mykeyspace/mytable-*/mb-1-big-Data.db \
  | grep -i tombstone

TTL as a Tombstone Alternative

Using TTL to expire data is cleaner than issuing DELETE statements for time-limited data. TTL tombstones are more predictable and compact more efficiently under TWCS (TimeWindowCompactionStrategy).

-- Instead of inserting and later deleting:
INSERT INTO session_tokens (token, user_id)
VALUES (uuid(), [uuid])
USING TTL 3600;

-- Row auto-expires after 1 hour; no manual DELETE needed.

Summary

Tombstones are Cassandra's mechanism for safe, distributed deletion. They ensure deleted data stays deleted even when replica nodes are temporarily offline. The gc_grace_seconds window controls how long tombstones survive before compaction removes them. Avoid patterns that generate excessive tombstones — especially nullable column writes, collection churn, and queue-style delete-after-read patterns. Use TTL as a clean alternative to manual deletion for data with a natural expiration time.