Cassandra Virtual Nodes

Virtual nodes, commonly called vnodes, allow each physical Cassandra node to own many small token ranges scattered around the ring instead of one large contiguous range. This design automates data balancing, speeds up cluster expansion, and makes node replacement far easier than the old manual token approach.

Why vnodes Were Introduced

In early Cassandra deployments, each node owned exactly one token range. Administrators had to manually calculate token values to spread data evenly. If one calculation was off, some nodes ended up with much more data than others. Adding or removing a node required manual rebalancing. Virtual nodes solved all of these problems by fragmenting ownership into many small pieces distributed randomly.

Old approach (single token per node):
  Node A owns: [Token 0       to Token 2.3Q]   — 25% of ring
  Node B owns: [Token 2.3Q    to Token 4.6Q]   — 25% of ring
  Node C owns: [Token 4.6Q    to Token 6.9Q]   — 25% of ring
  Node D owns: [Token -9.2Q   to Token 0   ]   — 25% of ring

  Adding Node E: must manually choose a token that splits one
  node's range exactly in half. Complex and error-prone.

vnode approach (256 tokens per node):
  Node A owns 256 scattered ranges covering ~25% of the ring
  Node B owns 256 scattered ranges covering ~25% of the ring
  Adding Node E: automatically receives 64 ranges from each
  existing node. Balanced by design.

How vnodes Work

With vnodes enabled, Cassandra assigns each physical node a set of tokens chosen randomly from the full token space. By default, each node gets 256 tokens. These 256 tokens divide the ring into 256 small arcs, each owned by that node. Data for any partition key hashes to one of these arcs and goes to the owning node.

Ring with 4 physical nodes × 256 vnodes = 1024 total arcs

Visual (simplified to 20 total arcs):

Position: 1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16 17 18 19 20
Owner:     A  C  B  D  A  B  C  D  B  A  D  C  A  B  D  C  B  A  D  C

Each letter appears roughly 5 times → ~25% ownership each

Configuring vnode Count

# cassandra.yaml
num_tokens: 256    # default; recommended value

Higher values give more even distribution but consume more memory for the token metadata. Values of 16 and 256 are the most common choices.

num_tokens effect on distribution (simulated, 4 nodes):

num_tokens=1 :  Node A=28%, B=22%, C=26%, D=24%  (uneven without planning)
num_tokens=16:  Node A=25.8%, B=24.3%, C=25.1%, D=24.8%
num_tokens=256: Node A=25.02%, B=24.98%, C=25.01%, D=24.99%  (near-perfect)

Benefits of vnodes

1. Automatic Data Balancing

When you add a new node, it automatically takes a proportional share of token ranges from each existing node. No manual token calculation is needed. Each existing node transfers only a fraction of its data to the new node, spreading the transfer load across the whole cluster.

4-node cluster, adding Node E (with 256 vnodes):

Node A transfers 64 of its 256 arcs → Node E
Node B transfers 64 of its 256 arcs → Node E
Node C transfers 64 of its 256 arcs → Node E
Node D transfers 64 of its 256 arcs → Node E

Result: 5 nodes each own ~20% of the ring.
Transfer load is spread across all 4 existing nodes simultaneously.

2. Faster Node Recovery

When a node fails and is replaced, the new node streams data from many nodes in parallel (one stream per vnode owner). With single-token nodes, the replacement streams from just one or two neighbors, making recovery slower. With 256 vnodes, 256 neighbors each contribute a small piece in parallel — much faster.

Node failure recovery time:

Single token: 1 neighbor streams ALL the missing data
  → Bottleneck on one node, slow recovery

256 vnodes: up to 256 nodes each stream a small portion
  → Parallel recovery, 10–50× faster

3. Heterogeneous Clusters

If some nodes have more powerful hardware, you can assign them more vnodes so they hold a proportionally larger share of data.

# Powerful node (more RAM, faster disk):
num_tokens: 512   # holds ~2× the data of standard nodes

# Standard node:
num_tokens: 256

# Less powerful node (legacy hardware):
num_tokens: 128

Viewing vnode Token Assignments

nodetool ring

Address    Rack    Status  State   Token
────────────────────────────────────────────────────────────────
10.0.0.1   rack1   Up      Normal  -9213093298666736028
10.0.0.2   rack2   Up      Normal  -9208148236388885498
10.0.0.1   rack1   Up      Normal  -9200152453065596720
10.0.0.3   rack1   Up      Normal  -9194752434820099017
...

(256 lines per node in a 256-vnode cluster)

Disabling vnodes (Single Token Mode)

Set num_tokens: 1 and specify initial_token to revert to the old single-token behavior. This requires manual token planning and is not recommended for new deployments.

# cassandra.yaml (legacy single-token mode)
num_tokens: 1
initial_token: -4611686018427387904

vnode Limitations

Limitation                           Notes
──────────────────────────────────────────────────────────────────
Higher memory for token metadata     256 tokens per node × cluster
                                     size = larger system.peers table
Repairs touch more nodes             Each vnode range must be repaired
                                     independently; incremental repair
                                     (with repair_by_keyspace) helps
Not easy to change after bootstrap   Changing num_tokens requires
                                     wiping and re-adding the node

Summary

Virtual nodes assign each physical node 256 scattered token ranges instead of one large contiguous range. This automatic fragmentation distributes data evenly, speeds up node addition and recovery, and supports heterogeneous clusters with different hardware capacities. Leave num_tokens at 256 for most production clusters. The token ring topology powered by vnodes is a core reason Cassandra scales so smoothly from a few nodes to hundreds.