nodetool disablehintsfordc
Disables hints for a specific datacenter.
Synopsis
Section titled “Synopsis”nodetool [connection_options] disablehintsfordc <datacenter>See connection options for connection options.
Description
Section titled “Description”nodetool disablehintsfordc prevents the local node from storing hints for writes destined for nodes in a specific datacenter. This is useful when a datacenter is expected to be unavailable for an extended period or is being decommissioned.
Non-Persistent Setting
This setting is applied at runtime only and does not persist across node restarts. After a restart, hints for all datacenters are enabled unless configured otherwise in cassandra.yaml.
To permanently disable hints for a datacenter, add it to the hinted_handoff_disabled_datacenters list in cassandra.yaml:
hinted_handoff_disabled_datacenters: - dc2Consistency Impact
Disabling hints for a datacenter means writes to unavailable nodes in that DC will not be recovered automatically. Use nodetool repair to restore consistency after re-enabling.
Arguments
Section titled “Arguments”| Argument | Description |
|---|---|
datacenter | Name of the datacenter to disable hints for |
Examples
Section titled “Examples”Disable Hints for a Datacenter
Section titled “Disable Hints for a Datacenter”nodetool disablehintsfordc dc2When to Use
Section titled “When to Use”Datacenter Decommissioning
Section titled “Datacenter Decommissioning”Prevent hint accumulation for a DC being removed:
# Disable hints for DC being decommissionednodetool disablehintsfordc dc_old
# Proceed with decommission operationsExtended Datacenter Outage
Section titled “Extended Datacenter Outage”When a DC will be down longer than the hint window:
# DC2 will be down for extended maintenancenodetool disablehintsfordc dc2
# Hints would expire anyway, this prevents accumulationDisk Space Management
Section titled “Disk Space Management”Prevent hints from consuming disk when a DC is unavailable:
# Check current hint usagenodetool tablestats system.hints
# Disable hints for unavailable DCnodetool disablehintsfordc dc2Cluster-Wide Operation
Section titled “Cluster-Wide Operation”Disable on All Nodes
Section titled “Disable on All Nodes”Hints are stored on coordinator nodes, so disable on all:
#!/bin/bashDATACENTER="$1"
if [ -z "$DATACENTER" ]; then echo "Usage: $0 <datacenter>" exit 1fi
# Get list of node IPs from local nodetool statusnodes=$(nodetool status | grep "^UN" | awk '{print $2}')
echo "Disabling hints for $DATACENTER on all nodes..."for node in $nodes; do echo -n "$node: " ssh "$node" "nodetool disablehintsfordc $DATACENTER" && echo "disabled" || echo "FAILED"done
echo ""echo "Remember to re-enable with: nodetool enablehintsfordc $DATACENTER"Impact Assessment
Section titled “Impact Assessment”Immediate Effects
Section titled “Immediate Effects”| Aspect | Impact |
|---|---|
| New hints for DC | Not stored |
| Existing hints | Continue delivery attempts |
| Coordinator disk | Stops growing for this DC |
Consistency Implications
Section titled “Consistency Implications”| Scenario | With Hints | Without Hints |
|---|---|---|
| DC temporarily down | Auto-recovery | Needs repair |
| Node restart | Hints delivered | Needs repair |
| Network partition | Hints queued | Data diverges |
Workflow: Datacenter Maintenance
Section titled “Workflow: Datacenter Maintenance”#!/bin/bashDC_NAME="$1"
echo "=== Datacenter Maintenance: $DC_NAME ==="
# 1. Disable hints for the DC via SSHecho "1. Disabling hints for $DC_NAME..."for node in $(nodetool status | grep "^UN" | awk '{print $2}'); do ssh "$node" "nodetool disablehintsfordc $DC_NAME" 2>/dev/nulldone
# 2. Log the actionecho "$(date): Hints disabled for $DC_NAME" >> /var/log/cassandra/dc_maintenance.log
# 3. Remind about re-enablingecho ""echo "Hints disabled for $DC_NAME"echo "After maintenance, run:"echo " - nodetool enablehintsfordc $DC_NAME (on all nodes)"echo " - nodetool repair -dc $DC_NAME"Troubleshooting
Section titled “Troubleshooting”Hints Still Growing After Disable
Section titled “Hints Still Growing After Disable”# Verify disabled on all coordinators# Must run on every node in the cluster
# Check if hints are from before disablenodetool listpendinghintsWrong Datacenter Name
Section titled “Wrong Datacenter Name”# List available datacentersnodetool status | grep "Datacenter:" | awk '{print $2}'
# Use exact name (case-sensitive)Recovery After Extended Disable
Section titled “Recovery After Extended Disable”#!/bin/bashDC_NAME="$1"
echo "=== Recovery After DC Maintenance ==="
# 1. Re-enable hints via SSHecho "1. Re-enabling hints for $DC_NAME..."for node in $(nodetool status | grep "^UN" | awk '{print $2}'); do ssh "$node" "nodetool enablehintsfordc $DC_NAME" 2>/dev/nulldone
# 2. Run repairecho ""echo "2. Running repair for $DC_NAME..."nodetool repair -dc $DC_NAME
echo ""echo "Recovery complete."Best Practices
Section titled “Best Practices”Disable Hints Guidelines
- Disable cluster-wide - Run on all coordinator nodes
- Document the reason - Log why and when disabled
- Plan for repair - Schedule repair after re-enabling
- Set reminders - Don't forget to re-enable
Considerations
- Data consistency relies on repair after re-enabling
- Don't disable indefinitely without reason
- Affects all writes to nodes in that DC
- Must disable on all nodes to be effective
Related Commands
Section titled “Related Commands”| Command | Relationship |
|---|---|
| enablehintsfordc | Re-enable hints for a datacenter |
| disablehandoff | Disable all hinted handoff |
| listpendinghints | List pending hints |
| repair | Restore consistency |