nodetool sethintedhandoffthrottlekb
Sets the hinted handoff throttle rate in KB per second.
Synopsis
Section titled “Synopsis”nodetool [connection_options] sethintedhandoffthrottlekb <throttle_in_kb>See connection options for connection options.
Description
Section titled “Description”nodetool sethintedhandoffthrottlekb controls the rate at which hints are delivered to recovered nodes. This throttle prevents hint delivery from overwhelming target nodes or consuming excessive network bandwidth.
Per-Thread Throttle
The throttle limit is applied per hinted handoff delivery thread, not as an aggregate node-wide limit. If multiple delivery threads are active, the total throughput may be a multiple of this value.
Arguments
Section titled “Arguments”| Argument | Description |
|---|---|
throttle_in_kb | Maximum hint delivery rate in KiB/s per delivery thread. Set to 0 to disable throttling. |
cassandra.yaml Parameter
The corresponding cassandra.yaml parameter changed in 4.1:
| Cassandra Version | Parameter Name | Example |
|---|---|---|
| Pre-4.1 | hinted_handoff_throttle_in_kb | 1024 |
| 4.1+ | hinted_handoff_throttle | 1024KiB |
Examples
Section titled “Examples”Set Throttle Rate
Section titled “Set Throttle Rate”# Set to 1024 KB/s (1 MB/s)nodetool sethintedhandoffthrottlekb 1024Increase for Faster Delivery
Section titled “Increase for Faster Delivery”# Increase to 2048 KB/s for faster hint deliverynodetool sethintedhandoffthrottlekb 2048Decrease for Reduced Impact
Section titled “Decrease for Reduced Impact”# Reduce to 512 KB/s to minimize impact on target nodesnodetool sethintedhandoffthrottlekb 512When to Use
Section titled “When to Use”Node Recovery Under Load
Section titled “Node Recovery Under Load”Reduce throttle when recovering node is busy:
# Slow down hint deliverynodetool sethintedhandoffthrottlekb 256
# Monitor target nodessh <target_node> "nodetool tpstats"Speed Up Hint Delivery
Section titled “Speed Up Hint Delivery”Increase throttle when hints need to be delivered quickly:
# Check pending hintsnodetool listpendinghints
# Increase throttle for faster deliverynodetool sethintedhandoffthrottlekb 4096Network Bandwidth Management
Section titled “Network Bandwidth Management”Adjust based on available network capacity:
# High-bandwidth networknodetool sethintedhandoffthrottlekb 4096
# Limited bandwidthnodetool sethintedhandoffthrottlekb 512Configuration
Section titled “Configuration”Default Value
Section titled “Default Value”The default is 1024 KiB/s per delivery thread.
# cassandra.yaml (4.1+)hinted_handoff_throttle: 1024KiB
# cassandra.yaml (Pre-4.1)# hinted_handoff_throttle_in_kb: 1024Runtime vs Persistent
Section titled “Runtime vs Persistent”| Setting | Persistence |
|---|---|
sethintedhandoffthrottlekb | Until restart |
cassandra.yaml | Permanent |
Impact Assessment
Section titled “Impact Assessment”Higher Throttle
Section titled “Higher Throttle”| Effect | Impact |
|---|---|
| Hint delivery | Faster |
| Network usage | Higher |
| Target node load | Higher |
| Recovery time | Shorter |
Lower Throttle
Section titled “Lower Throttle”| Effect | Impact |
|---|---|
| Hint delivery | Slower |
| Network usage | Lower |
| Target node load | Lower |
| Recovery time | Longer |
Monitoring Hint Delivery
Section titled “Monitoring Hint Delivery”#!/bin/bashecho "=== Hint Delivery Monitor ==="
# Initial hint countinitial=$(nodetool listpendinghints | tail -n +2 | awk '{sum+=$2} END {print sum}')echo "Initial pending hints: $initial"
# Wait and check againsleep 60
final=$(nodetool listpendinghints | tail -n +2 | awk '{sum+=$2} END {print sum}')echo "Pending hints after 60s: $final"
delivered=$((initial - final))echo "Hints delivered: $delivered"echo "Approximate rate: $((delivered / 60)) hints/second"Cluster-Wide Setting
Section titled “Cluster-Wide Setting”#!/bin/bashTHROTTLE="$1"
if [ -z "$THROTTLE" ]; then echo "Usage: $0 <throttle_kb>" exit 1fi
# Get list of node IPs from local nodetool statusnodes=$(nodetool status | grep "^UN" | awk '{print $2}')
echo "Setting hint throttle to ${THROTTLE} KiB/s cluster-wide..."for node in $nodes; do echo -n "$node: " ssh "$node" 'nodetool sethintedhandoffthrottlekb '"$THROTTLE"' && echo "OK" || echo "FAILED"'doneTroubleshooting
Section titled “Troubleshooting”Hints Delivering Too Slowly
Section titled “Hints Delivering Too Slowly”# Check current pending hintsnodetool listpendinghints
# Increase throttlenodetool sethintedhandoffthrottlekb 4096
# Monitor delivery ratewatch -n 10 'nodetool listpendinghints'Target Node Overwhelmed
Section titled “Target Node Overwhelmed”# Check target node healthssh <target_node> "nodetool tpstats"
# Reduce throttlenodetool sethintedhandoffthrottlekb 256
# Monitor target node recoveryBest Practices
Section titled “Best Practices”Throttle Guidelines
- Start conservative - Lower throttle for busy clusters
- Monitor target nodes - Watch for overload signs
- Adjust dynamically - Increase during maintenance windows
- Balance speed vs impact - Faster isn't always better
- Cluster-wide consistency - Set same value on all nodes
Throttle Selection (per delivery thread)
- 256-512 KiB/s: Low-impact, slow recovery
- 1024 KiB/s: Balanced (default)
- 2048-4096 KiB/s: Fast recovery, higher impact
- 0: Disable throttling (maximum speed, use cautiously)
Related Commands
Section titled “Related Commands”| Command | Relationship |
|---|---|
| listpendinghints | View pending hints |
| statushandoff | Check handoff status |
| pausehandoff | Pause hint delivery |
| resumehandoff | Resume hint delivery |
| getmaxhintwindow | View hint window |