Impact of modifying value of batch_size_fail_threshold_in_kb in cassandra.yaml

Question

I have a question regarding batch_size_fail_threshold_in_kb, a property cassandra.yaml. My current setup has batch_size_fail_threshold_in_kb: 50.

I have seen DataStax 6 documentation, it shows 640 kb as default. https://docs.datastax.com/en/dse/6.0/dse-dev/datastax_enterprise/config/configCassandra_yaml.html

So is it fine to change batch_size_fail_threshold_in_kb property? What all is can get impacted by it?

Answer 1

I would not change that setting without a very good reason. The value of batch_size_fail_threshold_in_kb exists to protect the coordinator node from crashing, in the event that an extremely large batch is sent. Batch statements utilized in a RDBMS fashion (sending thousands of batched writes to the same table) are known to be problematic, and this setting helps to protect against them.

I have had application teams approach me about increasing this value (when using batches correctly), because their payload columns exceed this threshold. I take those on a case-by-case basis.

I cannot speak to how DataStax decided on their (much higher) default. My guess, is that it might have something to do with their Solr or Graph integrations.

tl;dr;

batch_size_fail_threshold_in_kb is one setting where the default should almost never need to be adjusted.