Replica Synchronization
In distributed systems, replicas can diverge when they receive updates at different times — due to network partitions, node failures, or message ordering. Synchronization mechanisms detect these divergences and propagate missing information to restore convergence.
These mechanisms are sometimes called “anti-entropy” in distributed systems literature. The term originates from thermodynamics, where entropy measures disorder. In Cassandra’s context, these processes reduce disorder by ensuring all replicas eventually hold identical data.
Mechanisms
Section titled “Mechanisms”Cassandra employs three mechanisms to maintain replica convergence:
| Mechanism | Function | Trigger | Scope |
|---|---|---|---|
| Hinted handoff | Deferred write delivery | Write to unavailable replica | Single mutation |
| Read repair | Divergence detection during reads | Query execution | Single partition |
| Anti-entropy repair | Full dataset comparison | Scheduled maintenance | Token range |
These mechanisms operate at different timescales and granularities, collectively providing eventual consistency while preserving availability during partial failures.
Hinted handoff is the first line of defense for transient node unavailability. When a replica cannot be reached during a write, the coordinator stores the mutation and delivers it when the replica recovers. Hints expire after a configurable window; outages longer than the window require repair.
Read repair operates opportunistically during reads that contact multiple replicas. When the coordinator detects that replicas hold different versions of a partition, it returns the most recent value and asynchronously propagates it to stale replicas. Read repair only covers data that is actively queried.
Anti-entropy repair is the definitive mechanism for full convergence. It systematically compares all data within a token range using Merkle tree comparison, regardless of whether that data has been read recently. Repair must complete within gc_grace_seconds to prevent resurrection of deleted data.
Related Documentation
Section titled “Related Documentation”- Hinted Handoff - Configuration, hint windows, monitoring, and limitations
- Read Repair - Reconciliation during reads, configuration, and limitations
- Repair Architecture - Merkle trees, repair modes, gc_grace_seconds, and scheduling
- Distributed Data Overview - How synchronization fits in the distributed architecture
- Consistency - How consistency levels interact with convergence
- Repair Operations - Operational procedures for repair