Skip to content
Maintained by AxonOps — production-grade documentation from engineers who operate distributed databases at scale

Compaction Issues

Compaction merges SSTables to maintain read performance and reclaim space. When compaction falls behind or behaves unexpectedly, read performance degrades and disk usage grows.

Symptoms

  • Growing pending compaction count
  • Increasing SSTable count per table
  • Degrading read latency over time
  • Disk space not being reclaimed after deletes
  • High I/O during compaction spikes
  • "Compaction stuck" or very slow progress

Diagnosis

Step 1: Check Compaction Status

nodetool compactionstats

Problem indicators: - High pending count (> 50 per table) - Same compaction running for hours - No active compactions despite pending

Step 2: Check SSTable Counts

nodetool tablestats my_keyspace | grep -E "Table:|SSTable count"

Target: Generally < 20 SSTables per table (varies by strategy).

Step 3: Check Compaction Throughput

nodetool getcompactionthroughput

Default: 64 MB/s. May need increase for high-write workloads.

Step 4: Check Disk I/O

iostat -x 1 10

Problem indicators: - Disk utilization at 100% - High await times

Step 5: Check Compaction Strategy

cqlsh -e "SELECT table_name, compaction FROM system_schema.tables WHERE keyspace_name = 'my_keyspace';"

Resolution

Case 1: Compaction Falling Behind

Increase compaction throughput:

# Check current
nodetool getcompactionthroughput

# Increase (MB/s)
nodetool setcompactionthroughput 128

Increase concurrent compactors:

# Check current
nodetool getconcurrentcompactors

# Increase
nodetool setconcurrentcompactors 4

Case 2: Compaction Disabled

# Check if auto-compaction is enabled
nodetool statusautocompaction my_keyspace my_table

# Enable if disabled
nodetool enableautocompaction my_keyspace my_table

Case 3: Wrong Compaction Strategy

For read-heavy workloads (many updates):

ALTER TABLE my_table WITH compaction = {
    'class': 'LeveledCompactionStrategy',
    'sstable_size_in_mb': 160
};

For time-series data:

ALTER TABLE my_table WITH compaction = {
    'class': 'TimeWindowCompactionStrategy',
    'compaction_window_unit': 'DAYS',
    'compaction_window_size': 1
};

For write-heavy workloads:

ALTER TABLE my_table WITH compaction = {
    'class': 'SizeTieredCompactionStrategy',
    'min_threshold': 4,
    'max_threshold': 32
};

Case 4: Force Manual Compaction

For specific table:

nodetool compact my_keyspace my_table

For major compaction (use sparingly):

# Warning: Resource intensive
nodetool compact --split-output my_keyspace my_table

Major Compaction

Major compaction creates one large SSTable and can cause significant I/O. Use --split-output to create multiple smaller SSTables instead.

Case 5: Compaction Stuck on Large SSTable

Check what's happening:

nodetool compactionstats -H

If stuck on validation:

# May need to restart node if truly stuck
# First try waiting - large SSTables take time

# If necessary, cancel specific compaction
# (Requires identifying compaction ID from logs)

Case 6: Disk Full Preventing Compaction

See Handle Full Disk.

# Quick space recovery
nodetool clearsnapshot --all

# Check space
df -h /var/lib/cassandra

Compaction Strategy Selection

Strategy Best For SSTable Behavior
STCS Write-heavy, space-efficient Many SSTables, tiered by size
LCS Read-heavy, consistent performance Fixed-size levels
TWCS Time-series, TTL data Time-based windows
UCS Flexible, Cassandra 5.0+ Unified approach

Migration Between Strategies

-- Check current strategy
SELECT compaction FROM system_schema.tables
WHERE keyspace_name = 'my_ks' AND table_name = 'my_table';

-- Change strategy (takes effect gradually)
ALTER TABLE my_table WITH compaction = {
    'class': 'LeveledCompactionStrategy'
};

-- Force migration
nodetool compact my_keyspace my_table

Tuning Parameters

Size-Tiered (STCS)

ALTER TABLE my_table WITH compaction = {
    'class': 'SizeTieredCompactionStrategy',
    'min_threshold': 4,        -- Min SSTables to compact
    'max_threshold': 32,       -- Max SSTables to compact
    'min_sstable_size': 50     -- Min size to consider (MB)
};

Leveled (LCS)

ALTER TABLE my_table WITH compaction = {
    'class': 'LeveledCompactionStrategy',
    'sstable_size_in_mb': 160  -- Target SSTable size
};

Time-Window (TWCS)

ALTER TABLE my_table WITH compaction = {
    'class': 'TimeWindowCompactionStrategy',
    'compaction_window_unit': 'DAYS',
    'compaction_window_size': 1,
    'max_threshold': 32
};

Recovery

Verify Compaction Health

# Pending should decrease
watch -n 30 'nodetool compactionstats | head -10'

# SSTable count should stabilize
nodetool tablestats my_keyspace.my_table | grep "SSTable count"

Verify Read Performance

# Latencies should improve
nodetool tablehistograms my_keyspace my_table

Monitoring

Key Metrics

Metric Warning Critical
Pending compactions > 20 > 100
SSTable count > 20 > 50
Compaction throughput Near limit At 0
Disk usage > 70% > 85%

Monitoring Script

#!/bin/bash
# monitor_compaction.sh

while true; do
    clear
    echo "=== $(date) ==="
    echo ""
    echo "--- Compaction Stats ---"
    nodetool compactionstats | head -20
    echo ""
    echo "--- SSTable Counts ---"
    nodetool tablestats 2>/dev/null | grep -E "Table:|SSTable count" | head -20
    echo ""
    echo "--- Disk Usage ---"
    df -h /var/lib/cassandra
    sleep 60
done

Prevention

  1. Choose appropriate strategy - Match to workload pattern
  2. Monitor pending compactions - Alert early
  3. Adequate disk space - Keep < 70% utilization
  4. Sufficient I/O capacity - SSD recommended
  5. Tune throughput - Match to hardware capability
  6. Regular table maintenance - Monitor SSTable counts
Command Purpose
nodetool compactionstats Current compaction status
nodetool compact Force compaction
nodetool setcompactionthroughput Adjust throughput
nodetool getconcurrentcompactors View compactor count
nodetool enableautocompaction Enable auto-compaction
nodetool disableautocompaction Disable auto-compaction