Cassandra Driver Retry Policy
The retry policy determines whether to retry a failed request and on which node. This policy is critical for application reliability but must be configured carefully to avoid unintended consequences.
Error Classification
Section titled “Error Classification”Not all errors are retryable. The retry policy classifies errors and decides appropriate action:
| Error Type | Typical Cause | Retryable? |
|---|---|---|
| ReadTimeoutException | Replica(s) did not respond in time | Depends on received/required |
| WriteTimeoutException | Write coordinator timeout | Usually no (non-idempotent) |
| UnavailableException | Not enough replicas alive | Usually no (same CL will fail); consider fallback CL or circuit breaker |
| OverloadedException | Coordinator is overloaded | Retry on different node |
| ServerError | Unexpected server error | Usually no |
| QueryValidationException | Syntax or schema error | No (not transient) |
Read Timeout Details
Section titled “Read Timeout Details”A read timeout includes information about how many replicas responded:
ReadTimeoutException Analysis:
Required replicas (based on CL): 2Received responses: 1Data received: false
Interpretation: - Only 1 of 2 required replicas responded - No replica sent actual data (only digest)
Retry decision: - If 0 received: replica might be overloaded, retry elsewhere - If received >= required but no data: coordinator issue, retry - If received < required: may or may not help to retryWrite Timeout Details
Section titled “Write Timeout Details”Write timeouts are more complex because writes may have partially succeeded:
WriteTimeoutException Analysis:
Write type: SIMPLERequired acknowledgments: 2Received acknowledgments: 1
Interpretation: - Write reached coordinator - At least 1 replica acknowledged - Possibly 2+ replicas received it (timeout ≠ failure)
Danger: - Retrying may cause duplicate write - For non-idempotent operations (counters), this corrupts dataIdempotency Consideration
Section titled “Idempotency Consideration”A statement is idempotent when applying it twice has the same effect as applying it once. The driver treats the idempotence flag on a statement as a gate rather than a hint. In Java driver 4.x the driver reads the flag before it consults the retry policy, and where the statement is not marked idempotent it rethrows the error to the application without asking the policy for a verdict.
Four events are gated on the flag:
| Event | Statement marked idempotent | Statement not marked idempotent |
|---|---|---|
| Write timeout | The retry policy decides | The error is rethrown |
| Error response from the server | The retry policy decides | The error is rethrown |
| Request aborted before any response arrived | The retry policy decides | The error is rethrown |
| Slow response, with speculative execution configured | A second attempt is sent in parallel | No second attempt is sent |
Two events are not gated. A read timeout and an unavailable error reach the retry policy whichever way the flag is set:
- A read does not change data, so repeating it is safe.
- An unavailable error means the coordinator rejected the request before contacting any replica. Nothing was applied, so there is nothing to duplicate.
The flag therefore decides one question: whether the driver may retry the failures in which the write may already have been applied. It has no effect on the failures in which nothing can have been applied.
The driver does not infer idempotence from the statement text. The application sets the flag, and the driver takes it at face value. See Marking Statements Idempotent.
Which Statements Are Idempotent
Section titled “Which Statements Are Idempotent”| Statement | Safe to retry | Reason |
|---|---|---|
SELECT | yes | A read does not change data. |
INSERT or UPDATE of ordinary columns with USING TIMESTAMP | yes | Every attempt writes the same cells with the same timestamp. |
DELETE with USING TIMESTAMP | yes | Every attempt writes the same tombstone with the same timestamp. |
INSERT or UPDATE of ordinary columns without USING TIMESTAMP | no | The driver builds the request message once per attempt, and generates a timestamp then, so a retry is written at a later timestamp. That changes the outcome of last-write-wins resolution against a concurrent write. |
INSERT ... USING TTL | qualified | The expiry is computed from the time the write is applied, so a retried attempt expires later than the first. Safe unless the workload depends on the exact expiry instant. |
Counter update, SET c = c + 1 | no | Each attempt applies the increment again. See Counter Updates. |
List append or prepend, SET l = l + [...] or SET l = [...] + l | no | Each attempt appends again, so a retry leaves duplicate elements. |
Set addition, SET s = s + {...}, or map put, SET m[k] = v | yes | Writing the same element or key twice produces the same collection. |
A statement calling now(), uuid() or another non-deterministic function | no | The function is evaluated again on each attempt and writes a different value. |
Lightweight transaction, IF NOT EXISTS or IF <condition> | no | See Lightweight Transactions. |
Where a value must be both generated at write time and stable across retries, the application generates it before sending the statement and passes it as a bound value. A UUID produced in application code and bound as a parameter is written unchanged by every attempt; uuid() evaluated in the statement is not.
Counter Updates
Section titled “Counter Updates”A counter update applies its increment once per attempt. UPDATE page_stats SET views = views + 1 WHERE page_id = ? adds one to the counter each time it reaches the replicas. Where the write times out after the increment was applied and the driver sends the statement again, the counter is incremented twice.
No form of the statement avoids this. Cassandra 5.0, 4.1 and 4.0 support only relative counter updates, so a counter update cannot be expressed as an absolute value that converges on repetition.
The damage also cannot be found afterwards. Reading the counter returns a number, and one increment and two increments both produce a plausible number. A read cannot establish which occurred, and there is no repair that restores the intended value.
Do not mark a counter update idempotent
Problem: A counter update adds its increment once per attempt. Marking it idempotent allows the driver to retry a write timeout, and the increment is applied a second time when the first attempt had already succeeded.
Symptoms: Counter values drift upward, in proportion to the write timeout rate. The drift is silent. Reading the counter cannot distinguish one increment from two, so the error is neither detectable nor repairable after the fact.
Instead: Leave the flag false on counter statements, and route them to an execution profile whose retry policy never retries. See Per-Statement Policy Override.
// BAD: a timed-out write is retried and the counter is double-countedSimpleStatement.builder("UPDATE page_stats SET views = views + 1 WHERE page_id = ?") .addPositionalValue(pageId) .setIdempotent(true) .build();
// GOOD: the driver rethrows the write timeout and the application decidesSimpleStatement.builder("UPDATE page_stats SET views = views + 1 WHERE page_id = ?") .addPositionalValue(pageId) .setIdempotent(false) .build();A counter update must not be marked idempotent.
Lightweight Transactions
Section titled “Lightweight Transactions”A lightweight transaction that times out has an unknown outcome. The Paxos round may have committed or it may not have, and the error does not say which. The conditional result column, [applied], does not distinguish the two.
The coordinator reports a write type with a write timeout, saying what kind of write was attempted. For a lightweight transaction it is CAS. Neither policy shipped with Java driver 4.x retries it. DefaultRetryPolicy retries a write timeout only for write type BATCH_LOG. ConsistencyDowngradingRetryPolicy handles SIMPLE, BATCH, UNLOGGED_BATCH and BATCH_LOG, and rethrows every other write type. A timed-out lightweight transaction is therefore reported to the application whatever the idempotence flag says. ConsistencyDowngradingRetryPolicy also rethrows a read timeout at a serial level, because a read issued at SERIAL or LOCAL_SERIAL was issued at that level for a reason that downgrading would discard.
A lightweight transaction must not be marked idempotent.
Do not retry a timed-out lightweight transaction in application code
Problem: reissuing the conditional write after a timeout returns [applied] = false when the first attempt had in fact committed. That is the same response returned when another client won the race, so the application concludes it lost a race it had won, and takes the wrong branch.
Symptoms: an operation reports that the row already existed, or that a condition failed, when the application's own earlier attempt is what created the row or met the condition.
Instead: read the row at SERIAL or LOCAL_SERIAL, which returns the state the Paxos round committed, and branch on that. A second conditional write cannot supply the same answer.
// BAD: the outcome of the first attempt is unknown, and this cannot tell// "someone else won" apart from "my own first attempt succeeded"session.execute(SimpleStatement.builder( "INSERT INTO accounts (id, owner) VALUES (?, ?) IF NOT EXISTS") .addPositionalValues(id, owner).build());
// GOOD: establish what committed, then decideRow row = session.execute(SimpleStatement.builder( "SELECT owner FROM accounts WHERE id = ?") .addPositionalValues(id) .setConsistencyLevel(ConsistencyLevel.SERIAL).build()).one();Retry Policy Decisions
Section titled “Retry Policy Decisions”A retry policy is a set of callbacks, one for each class of failure. The driver classifies the failure and calls the callback for that class. It passes the callback the details of the failure: how many replicas the consistency level required, how many answered, and whether one of them returned data rather than a digest. For a write it also passes the write type (SIMPLE, BATCH, UNLOGGED_BATCH, BATCH_LOG, COUNTER and CAS for a lightweight transaction), which indicates how much of the write may already be durable. Those details are the whole basis for the decision.
Decisions
Section titled “Decisions”Each callback returns a RetryVerdict. A verdict has two parts. The first is a RetryDecision:
| Decision | Meaning |
|---|---|
| RETRY_SAME | Retry on the same node |
| RETRY_NEXT | Retry on the next node in the query plan |
| IGNORE | Return an empty result to the application, as if the request had succeeded |
| RETHROW | Propagate the error to the application |
The second is a getRetryRequest(previous) method that returns the request to send on the retry. Its default implementation returns the previous request unchanged, so a retry normally repeats the original request. Overriding it lets a policy modify the request first: its consistency level, its query timestamp, or its custom payload. That method is the mechanism by which consistency downgrading works.
Decision Flow
Section titled “Decision Flow”Common Retry Policies
Section titled “Common Retry Policies”Default Retry Policy
Section titled “Default Retry Policy”Most drivers include a default policy that retries at most once and never lowers the consistency level. The Java driver's DefaultRetryPolicy is representative:
| Failure | Decision | Condition |
|---|---|---|
| Read timeout | RETRY_SAME | Enough replicas answered but none returned the data. A digest mismatch, where data was returned, is rethrown |
| Write timeout | RETRY_SAME | Only for write type BATCH_LOG. Every other write type is rethrown |
| Unavailable | RETRY_NEXT | First attempt only |
In the Java driver, DefaultRetryPolicy.onUnavailable returns RETRY_NEXT on the first attempt and RETHROW on every attempt after that. The coordinator refused the request based on its own view of which replicas were up, and that view may be stale. Another coordinator may hold a current view and accept the request. The retry does not change the consistency level, so it succeeds only if the second coordinator finds enough replicas alive.
This is a conservative policy suitable for mixed workloads.
Fallthrough (No Retry)
Section titled “Fallthrough (No Retry)”Never retries any error; always propagates it to the application.
In Java driver 4.x this comes from a custom policy whose callbacks all return RETHROW, named under the advanced.retry-policy.class key in the driver's HOCON configuration file, typically application.conf:
datastax-java-driver.advanced.retry-policy { class = com.example.FallthroughRetryPolicy}Java driver 3.x shipped this directly as FallthroughRetryPolicy.INSTANCE, set with withRetryPolicy on the cluster builder.
Use when:
- Application handles retries itself
- All operations are non-idempotent
- Debugging (to see all errors)
Aggressive Retry
Section titled “Aggressive Retry”Retries most errors multiple times:
Aggressive retry can cause cascading failures
Aggressive retry policies can cause cascading failures by amplifying load on an already struggling cluster.
Cascading Failure Scenario:
1. Node3 becomes slow (GC, disk issue)2. Requests to Node3 timeout3. Aggressive retry policy retries each request 3×4. Node3 now receives 3× the requests5. Node3 becomes slower6. Timeouts increase, more retries triggered7. Node3 overwhelmed, marks as DOWN8. Load shifts to Node1, Node29. If they were near capacity, they may also degradeDowngrading Consistency
Section titled “Downgrading Consistency”A downgrading retry policy reissues a failed request at a lower consistency level than the one requested, instead of returning the error to the application. It does this by returning a verdict whose getRetryRequest method lowers the consistency level of the original request before the driver sends it again.
The trade-off is that the application gets a weaker guarantee than the one it asked for, and is not told. A read issued at LOCAL_QUORUM and retried at ONE may not see a write that was made at LOCAL_QUORUM before it. The same applies to a read issued at QUORUM.
The Java Policy
Section titled “The Java Policy”ConsistencyDowngradingRetryPolicy ships with Java driver 4.10 and later. It downgrades a request at most once. If the downgraded request fails as well, the error is reported to the application.
On an unavailable error at a serial level (SERIAL or LOCAL_SERIAL), the verdict is RETRY_NEXT. A serial level means the failure happened in the Paxos phase of a lightweight transaction. The coordinator may be isolated from the rest of the cluster, so another coordinator may be able to complete the transaction. At any other level, the policy downgrades using the number of replicas the coordinator reported alive.
On a read timeout, the conditions are evaluated in this order:
| Condition | Verdict | Reason |
|---|---|---|
| Consistency level is serial | RETHROW | Downgrading a CAS read is never correct |
received is less than blockFor | Downgrade, based on received | Fewer replicas answered than the level required |
Enough replicas answered but dataPresent is false | RETRY_SAME | The replica the coordinator asked for the data had not yet been marked down; the same coordinator asks a different replica for it on the retry |
| Anything else | RETHROW | This is usually a digest mismatch. The read path's own reconciliation resolves it, and a retry at the same level does not |
On a write timeout, the write type decides:
| Write type | Verdict | Reason |
|---|---|---|
SIMPLE, BATCH | IGNORE if at least one replica acknowledged, otherwise RETHROW | The write is already durable on at least one replica |
UNLOGGED_BATCH | Downgrade, based on received | An unlogged batch is not atomic, so only part of it may have been persisted |
BATCH_LOG | RETRY_SAME | The timeout occurred while writing the batch log, before the batch itself was applied |
| Any other type | RETHROW |
Choosing the Downgrade Target
Section titled “Choosing the Downgrade Target”The policy downgrades to a numeric level, THREE, TWO or ONE, chosen from the count of replicas that answered or were reported alive. It does not step down a ladder of named levels.
Downgrading discards the datacenter constraint
THREE, TWO and ONE count replicas cluster-wide. A request issued at LOCAL_QUORUM and downgraded to TWO or ONE may be satisfied by replicas in any datacenter. A write intended for the local datacenter can then be acknowledged only in a remote one, and a read can be answered by a datacenter that has not yet received the latest write. The Java driver documentation carries the same warning.
| Replicas counted | Retry level |
|---|---|
| 3 or more | THREE |
| Exactly 2 | TWO |
| Exactly 1 | ONE |
| 0 | None; the verdict is RETHROW |
A request issued at EACH_QUORUM is downgraded straight to ONE regardless of the count, because on Cassandra 5.0, 4.1 and 4.0 the error reports the blocked-for and live replica counts of the datacenter that failed, not of the cluster (JAVA-1005).
The write timeout case needs equal attention, because it does not surface as an error at all.
An ignored write timeout is reported as success
On a write timeout of type SIMPLE or BATCH where at least one replica acknowledged the write, ConsistencyDowngradingRetryPolicy returns IGNORE. The driver completes the request normally, so the application sees a successful write. The write reached at least one replica rather than the number the requested consistency level demanded. The application receives no indication that the guarantee it asked for was not met.
gocql Uses an Explicit Sequence
Section titled “gocql Uses an Explicit Sequence”gocql takes the other approach. Its DowngradingConsistencyRetryPolicy holds a list of levels, ConsistencyLevelsToTry, and works through it: the initial attempt uses the level set on the query, the first retry uses the first entry in the list, the second retry uses the second entry, and the policy stops retrying once the attempt count exceeds the length of the list.
The operator supplies the sequence, so a ladder that preserves locality can be written directly. The Java policy cannot express one:
// gocql: a query issued at EACH_QUORUM steps down to QUORUM, then LOCAL_QUORUMcluster.RetryPolicy = &gocql.DowngradingConsistencyRetryPolicy{ ConsistencyLevelsToTry: []gocql.Consistency{ gocql.Quorum, gocql.LocalQuorum, },}gocql applies the level change per attempt, independently of how it classifies the error. gocql classifies errors with four values: Retry repeats the request on the same connection, RetryNextHost sends it to another host, and Ignore and Rethrow match the Java decisions of the same name. For non-serial unavailable errors and for write timeouts, the classification otherwise reaches the same outcome as the Java policy. Three cases differ:
| Failure | gocql | Java policy |
|---|---|---|
Write timeout of type COUNTER, at least one acknowledgment | Ignore | RETHROW |
| Read timeout with enough replicas and data present, that is, a digest mismatch | Retry | RETHROW |
| Read timeout with enough replicas but no data returned | Retry, at the next level in the list | RETRY_SAME, at the original level |
The third row follows from the ladder advancing on every attempt. Java retries that case at the level originally requested, because the coordinator had simply asked a replica it had not yet marked down. gocql lowers the level as well.
The COUNTER case is the one to weigh before adopting the policy. gocql reports a timed-out counter write as successful once at least one replica has acknowledged it, even though fewer replicas acknowledged than the requested level demanded. A counter increment cannot be replayed afterwards to establish what was applied.
Driver Availability
Section titled “Driver Availability”| Driver | Policy | Status |
|---|---|---|
| Java 4.10+ | ConsistencyDowngradingRetryPolicy | Shipped, opt-in and not the default (JAVA-2900) |
| Java 4.0 to 4.9 | None | Not shipped |
| Java 3.x | DowngradingConsistencyRetryPolicy | Deprecated in 3.5.0 (JAVA-1752) |
| gocql | DowngradingConsistencyRetryPolicy | Present in the Apache gocql driver, and in the gocql/gocql releases that preceded it |
| Python 3.x | DowngradingConsistencyRetryPolicy | Documented as deprecated and slated for removal in the next major release |
An application that depends on the behaviour should therefore pin the driver version and confirm the policy is present before an upgrade, rather than assume it survives one.
A downgraded retry is still a retry, so the idempotency rules in Idempotency Consideration apply unchanged. A write timeout retried at ONE can still apply a counter increment twice.
Writing a custom retry policy is a supported alternative and is often the more precise one: a custom policy can downgrade only the statements for which a weaker guarantee is acceptable, refuse to downgrade non-idempotent writes, and record every downgraded request so that the affected rows can be re-read or repaired afterwards. See Custom Retry Policies.
Custom Retry Policies
Section titled “Custom Retry Policies”For fine-grained control, implement the five callbacks directly:
| Callback | Called when | Arguments beyond the request |
|---|---|---|
onReadTimeoutVerdict | The coordinator reported a read timeout | consistency level, blockFor, received, dataPresent, retryCount |
onWriteTimeoutVerdict | The coordinator reported a write timeout | consistency level, writeType, blockFor, received, retryCount |
onUnavailableVerdict | The coordinator raised UnavailableException | consistency level, required, alive, retryCount |
onRequestAbortedVerdict | The request was aborted before any response arrived | the error, retryCount |
onErrorResponseVerdict | The server returned an error response | the error, retryCount |
These are the callback names of Java driver 4.10 and later, and the code below compiles against that shape. Before 4.10, the equivalent callbacks returned a RetryDecision directly.
The class must expose the public constructor (DriverContext context, String profileName), because the driver instantiates the policy by reflection from the advanced.retry-policy.class configuration key.
// Java driver 4.10+public class IdempotentOnlyRetryPolicy implements RetryPolicy {
public IdempotentOnlyRetryPolicy(DriverContext context, String profileName) { // Signature required by the driver }
@Override public RetryVerdict onReadTimeoutVerdict( Request request, ConsistencyLevel cl, int blockFor, int received, boolean dataPresent, int retryCount) {
// Reads are idempotent, so try another coordinator once return retryCount == 0 ? RetryVerdict.RETRY_NEXT : RetryVerdict.RETHROW; }
@Override public RetryVerdict onWriteTimeoutVerdict( Request request, ConsistencyLevel cl, WriteType writeType, int blockFor, int received, int retryCount) {
// Retry only a statement the application marked idempotent boolean idempotent = request instanceof Statement && Boolean.TRUE.equals(((Statement<?>) request).isIdempotent()); return idempotent && retryCount == 0 ? RetryVerdict.RETRY_NEXT : RetryVerdict.RETHROW; }
@Override public RetryVerdict onUnavailableVerdict( Request request, ConsistencyLevel cl, int required, int alive, int retryCount) {
// The coordinator's view of live replicas may be stale return retryCount == 0 ? RetryVerdict.RETRY_NEXT : RetryVerdict.RETHROW; }
@Override public RetryVerdict onRequestAbortedVerdict( Request request, Throwable error, int retryCount) { return RetryVerdict.RETHROW; }
@Override public RetryVerdict onErrorResponseVerdict( Request request, CoordinatorException error, int retryCount) { return RetryVerdict.RETHROW; }
@Override public void close() {}}The policy is then named in the driver's HOCON configuration file, typically application.conf:
datastax-java-driver.advanced.retry-policy { class = com.example.IdempotentOnlyRetryPolicy}Downgrading One Level at a Time
Section titled “Downgrading One Level at a Time”Because a verdict can rewrite the request, a custom policy can perform the stepwise downgrade the shipped Java policy does not offer: EACH_QUORUM to QUORUM to LOCAL_QUORUM, which keeps the request answerable within the local datacenter instead of dropping straight to ONE.
// Java driver 4.10+: a verdict that retries the request at a lower consistency levelpublic class DowngradingVerdict implements RetryVerdict {
private final ConsistencyLevel level;
public DowngradingVerdict(ConsistencyLevel level) { this.level = level; }
@Override public RetryDecision getRetryDecision() { return RetryDecision.RETRY_SAME; }
@Override @SuppressWarnings("unchecked") public <RequestT extends Request> RequestT getRetryRequest(RequestT previous) { return previous instanceof Statement ? (RequestT) ((Statement<?>) previous).setConsistencyLevel(level) : previous; }}The two members below replace onUnavailableVerdict in the IdempotentOnlyRetryPolicy skeleton above. onReadTimeoutVerdict takes the same body, and the remaining callbacks are unchanged.
// Java driver 4.10+: one step down the ladder, or null at the bottom of itprivate static final int MAX_DOWNGRADES = 2;
private static ConsistencyLevel downgrade(ConsistencyLevel current) { if (current == DefaultConsistencyLevel.EACH_QUORUM) { return DefaultConsistencyLevel.QUORUM; } if (current == DefaultConsistencyLevel.QUORUM) { return DefaultConsistencyLevel.LOCAL_QUORUM; } return null;}
@Overridepublic RetryVerdict onUnavailableVerdict( Request request, ConsistencyLevel cl, int required, int alive, int retryCount) {
// Bound the ladder: at most MAX_DOWNGRADES retries, and no step below LOCAL_QUORUM ConsistencyLevel next = retryCount < MAX_DOWNGRADES ? downgrade(cl) : null; return next == null ? RetryVerdict.RETHROW : new DowngradingVerdict(next);}The bound is the part a custom policy must get right. retryCount is zero on the first decision for a request and rises with each retry, so comparing it against MAX_DOWNGRADES caps the attempts, and downgrade returning null stops the ladder at LOCAL_QUORUM. Without one of the two, a request that keeps failing is retried indefinitely. cl is the level the failed attempt used, so the second failure arrives with cl already set to QUORUM and steps to LOCAL_QUORUM.
The same structure allows a policy to refuse to downgrade a request that is not marked idempotent, and to record each downgraded request so that the affected rows can be re-read or repaired afterwards.
Marking Statements Idempotent
Section titled “Marking Statements Idempotent”The flag is set per statement, on the statement itself or on its builder. Per-statement marking is the approach to use, because it records a decision about one query rather than about every query in the application.
// Java driver 4.xSimpleStatement statement = SimpleStatement.builder("UPDATE users SET name = ? WHERE id = ?") .addPositionalValues("Alice", userId) .setIdempotent(true) .build();
// prepared: a PreparedStatement obtained from session.prepare(...)BoundStatement bound = prepared.bind("Alice", userId).setIdempotent(true);In Java driver 4.x the value has three states: true, false, and unset. Unset is the initial state of a statement that has never been marked, and it is not the same as false. A statement whose value is unset takes the driver-wide default at execution time; a statement marked false stays false.
The driver-wide default is the configuration option basic.request.default-idempotence, and it is false.
datastax-java-driver.basic.request { # Leave this false. It applies to every statement that is not marked individually. default-idempotence = false}Setting default-idempotence to true changes the safety of every unmarked statement
basic.request.default-idempotence = true marks every statement whose own value is unset as idempotent, including counter updates, list appends and lightweight transactions. One configuration line then allows the driver to retry write timeouts across the whole application, and there is no error or warning to indicate it. Leave the option at false unless every statement the application issues has been reviewed and marked individually.
Marking a statement false, rather than leaving it unset, keeps a counter update or a conditional write safe regardless of the driver-wide default.
Per-Statement Policy Override
Section titled “Per-Statement Policy Override”In Java driver 4.x, the retry policy belongs to an execution profile rather than to a statement. A profile declares the policy, and a statement selects the profile:
datastax-java-driver.profiles.no-retry.advanced.retry-policy { class = com.example.NeverRetryPolicy}// Java driver 4.xSimpleStatement counterUpdate = SimpleStatement.builder( "UPDATE page_stats SET views = views + 1 WHERE page_id = ?") .addPositionalValue(pageId) .setExecutionProfileName("no-retry") // Never retry a counter update .build();Java driver 3.x set the policy on the statement itself, with setRetryPolicy.
Retry Metrics
Section titled “Retry Metrics”Monitor retry behavior in production:
| Metric | Description | Warning Sign |
|---|---|---|
| Retry rate | Retries per second | Sustained high rate indicates cluster issues |
| Retry success rate | Percentage of retries that succeed | Low success rate means retries are wasteful |
| Retry exhaustion | Requests that failed after all retries | Any occurrence needs investigation |
Best Practices
Section titled “Best Practices”| Practice | Rationale |
|---|---|
| Default to conservative | Better to fail fast than corrupt data |
| Mark idempotent operations explicitly | Enables safe retry for those operations |
| Monitor retry rates | High retry rates indicate underlying issues |
| Don't rely on retries for availability | Fix the root cause instead |
| Consider circuit breakers | Prevent retry storms during outages |
Related Documentation
Section titled “Related Documentation”- Load Balancing Policy: determines which node receives the retry
- Speculative Execution: alternative to retry for latency reduction
- Consistency Levels: defines
SERIAL,LOCAL_SERIALand the quorum levels a downgrade moves between - Lightweight Transactions: the CQL syntax, the
[applied]result column and the compare-and-set pattern - Paxos: the protocol behind a lightweight transaction and what a timeout leaves undecided