nodetool compactionstats
Displays statistics about currently running compactions and pending compaction tasks.
Synopsis
Section titled “Synopsis”nodetool [connection_options] compactionstats [-H]See connection options for connection options.
Description
Section titled “Description”nodetool compactionstats shows:
- Currently active compaction operations
- Progress of each compaction
- Pending compaction tasks by keyspace
- Compaction throughput
This is essential for monitoring compaction health and identifying backlogs.
Options
Section titled “Options”| Option | Description |
|---|---|
-H, --human-readable | Display sizes in human-readable format |
-V, --vtable | Use virtual table format for output |
Output Example
Section titled “Output Example”pending tasks: 15- my_keyspace.users: 8- my_keyspace.orders: 7 id compaction type keyspace table completed total unit progress abc123-def4-... Compaction my_keyspace users 1234567890 2345678901 bytes 52.65% def456-abc7-... Compaction my_keyspace orders 567890123 1234567890 bytes 46.01% ghi789-jkl0-... Validation my_keyspace users 45678901 123456789 bytes 37.00%Active compaction remaining time : 0h12m34sNote: The output includes pending tasks broken down by keyspace and table, plus a per-compaction id column (unless -V/--vtable is used).
Output Fields
Section titled “Output Fields”| Field | Description |
|---|---|
| pending tasks | Total compaction tasks waiting to run (with per-keyspace.table breakdown) |
| id | Unique identifier for the active compaction operation |
| compaction type | Type of operation (Compaction, Validation, Cleanup, etc.) |
| keyspace | Keyspace being compacted |
| table | Table being compacted |
| completed | Bytes processed so far |
| total | Total bytes to process |
| unit | Unit of measurement |
| progress | Percentage complete |
Compaction Types
Section titled “Compaction Types”| Type | Description |
|---|---|
| Compaction | Regular SSTable compaction |
| Validation | Building Merkle tree for repair |
| Cleanup | Removing data not owned by node |
| Scrub | Rebuilding corrupted SSTables |
| Upgrade | Upgrading SSTable format |
| Index_build | Building secondary index |
| Anticompaction | Splitting SSTables for incremental repair |
Interpreting Results
Section titled “Interpreting Results”Healthy State
Section titled “Healthy State”pending tasks: 0Active compaction remaining time : n/aNo active compactions and no backlog.
Active Compaction
Section titled “Active Compaction”pending tasks: 3 compaction type keyspace table completed total unit progress Compaction my_keyspace users 1234567890 2345678901 bytes 52.65%Active compaction remaining time : 0h05m23sNormal operation with manageable workload.
Compaction Backlog
Section titled “Compaction Backlog”pending tasks: 150 compaction type keyspace table completed total unit progress Compaction my_keyspace users 1234567890 2345678901 bytes 52.65%Active compaction remaining time : 2h45m12sHigh Pending Tasks
Large pending task count indicates:
- Writes exceeding compaction capacity
- Insufficient compaction throughput
- Need to tune compaction settings
Actions:
- Increase compaction throughput
- Add more concurrent compactors
- Reduce write rate if possible
- Review compaction strategy
Multiple Active Compactions
Section titled “Multiple Active Compactions”pending tasks: 10 compaction type keyspace table completed total unit progress Compaction my_keyspace users 1234567890 2345678901 bytes 52.65% Compaction my_keyspace orders 567890123 1234567890 bytes 46.01% Compaction other_ks events 123456789 234567890 bytes 52.63% Compaction other_ks logs 234567890 345678901 bytes 67.95%Multiple concurrent compactors running (controlled by concurrent_compactors).
When to Use
Section titled “When to Use”Monitor Compaction Health
Section titled “Monitor Compaction Health”# Periodic checknodetool compactionstats
# Continuous monitoringwatch -n 5 'nodetool compactionstats'During High Write Load
Section titled “During High Write Load”Monitor for backlog buildup:
watch -n 2 'nodetool compactionstats | head -5'During Repair
Section titled “During Repair”Validation compactions appear during repair:
nodetool compactionstats | grep ValidationAfter Adding Data
Section titled “After Adding Data”After bulk loads or migrations:
# Watch compaction catch upwatch -n 10 'nodetool compactionstats'Examples
Section titled “Examples”Basic Usage
Section titled “Basic Usage”nodetool compactionstatsHuman-Readable Output
Section titled “Human-Readable Output”nodetool compactionstats -HOutput:
pending tasks: 15 compaction type keyspace table completed total unit progress Compaction my_keyspace users 1.2 GB 2.3 GB bytes 52.65%Check Pending Tasks Only
Section titled “Check Pending Tasks Only”nodetool compactionstats | head -1Monitor Continuously
Section titled “Monitor Continuously”watch -n 5 'nodetool compactionstats -H'Compare Across Nodes
Section titled “Compare Across Nodes”for node in node1 node2 node3; do echo "=== $node ===" ssh "$node" "nodetool compactionstats | head -3"doneManaging Compaction Backlog
Section titled “Managing Compaction Backlog”Increase Throughput
Section titled “Increase Throughput”# Check current throughputnodetool getcompactionthroughput
# Increase throughput (MB/s)nodetool setcompactionthroughput 256Add Concurrent Compactors
Section titled “Add Concurrent Compactors”# Check current settingnodetool getconcurrentcompactors
# Increase (use cautiously)nodetool setconcurrentcompactors 4Concurrent Compactors
More compactors = more parallel I/O. Only increase if disk can handle it.
Stop Non-Critical Compactions
Section titled “Stop Non-Critical Compactions”# Stop current compactions (for emergency)nodetool stop COMPACTION
# Better: Stop specific typesnodetool stop INDEX_BUILDPending Tasks Analysis
Section titled “Pending Tasks Analysis”Understanding Pending Count
Section titled “Understanding Pending Count”pending tasks: 150Pending tasks accumulate when:
| Cause | Solution |
|---|---|
| High write rate | Increase throughput/compactors |
| Large SSTables | Tune compaction strategy |
| Slow disks | Improve I/O capacity |
| Repairs running | Normal during repair |
| Insufficient threads | Increase compactors |
Checking Task Distribution
Section titled “Checking Task Distribution”# See which tables have pending compactionsnodetool compactionstats -HTables with consistently high pending may need:
- Different compaction strategy
- More frequent cleanup
- Data model review
Compaction Throughput
Section titled “Compaction Throughput”Current Setting
Section titled “Current Setting”nodetool getcompactionthroughputAdjust Throughput
Section titled “Adjust Throughput”# Increase to 256 MB/snodetool setcompactionthroughput 256
# Unlimited (not recommended)nodetool setcompactionthroughput 0Throughput Guidelines
- Default: 64 MB/s
- SSD clusters: 256-512 MB/s
- HDD clusters: 64-128 MB/s
- Monitor disk utilization when increasing
Warning Signs
Section titled “Warning Signs”Continuous Backlog Growth
Section titled “Continuous Backlog Growth”# Check every minutewhile true; do echo "$(date): $(nodetool compactionstats | head -1)" sleep 60doneIf pending continuously increases, action required.
Very Long Compactions
Section titled “Very Long Compactions”Compactions running for hours may indicate:
- Very large SSTables
- Slow disk I/O
- Memory pressure (GC during compaction)
High CPU During Compaction
Section titled “High CPU During Compaction”Check with system tools:
top -p $(pgrep -f CassandraDaemon)Compression/decompression is CPU-intensive.
Related Commands
Section titled “Related Commands”| Command | Relationship |
|---|---|
| getcompactionthroughput | Check throughput setting |
| setcompactionthroughput | Adjust throughput |
| compact | Force compaction |
| tablestats | SSTable counts per table |