Significance of Vnodes in Cassandra

Question

From the url: http://www.datastax.com/dev/blog/virtual-nodes-in-cassandra-1-2, they say:

"If instead we have randomized vnodes spread throughout the entire cluster, we still need to transfer the same amount of data, but now it’s in a greater number of much smaller ranges distributed on all machines in the cluster. This allows us to rebuild the node faster than our single token per node scheme."

Itseems that the above sentence convey, when we replace a dead node with a new node with same num_tokens (say num_tokens:4), then replaced node contain the same token value as the dead node had before releasing those token values.

But Vnodes generates random token values for every node, then how is it possible to replace a node with same Vnodes token value?

The URL seems confusing in explaining the concept of replacing a dead node with new node using the concept of VNODES. It would be nice if someone can clarify how a Vnode is used for replacing the dead node with exact token value ranges.

Thanks in advance.

Answer 1

First, the vnode parameter num_tokens should be set to a small number, the current recommendation from DataStax is eight (8). The original default was 256, which experience found to be too high.

With traditional token ranges, you only have as many ranges as nodes. But, using vnodes, the number of token ranges is virtualized and much larger. You can not mix vnodes and token ranges in the same data center (ring).

Node Failure With Token Ranges:

In this DataStax example above with token ranges, data for ranges C, D and E reside on just three nodes:

• Range C is owned by node 3 and replicated on nodes 4 and 5
• Range D is owned by node 4 and replicated on nodes 5 and 6
• Range E is owned by node 5 and replicated on nodes 6 and 1

In this example, when node 5 fails, Ranges C, D, and E are reloaded and streamed from only three of the remaining five nodes: 1, 3 and 4. Nodes 2 doesn't have any of the node 5 data and node 6 has the same data being streamed by node 1. Thus nodes 2 and 5 are idle during the rebuilding.

Node Failure With Vnodes:

However, when using vnodes the token ranges are split up into smaller ranges and randomized across the entire cluster of 6 nodes. With smaller ranges, a portion of node 5's data is replicated to every one of the other nodes.

When rebuilding node 5, data can now be streamed from all 5 of the available nodes in the cluster.

The primary advantages of vnodes are:

• rebalancing a cluster manualy is no longer required when adding or removing nodes
• rebuilding can stream data from all available nodes