G1 for Cassandra
The Garbage-First collector (G1) divides the heap into equal-sized regions and collects a subset of them in each pause, choosing the subset to meet a pause goal. It compacts as it collects, so the old generation does not fragment. Pause duration scales with the live data copied during a collection rather than with the size of the heap.
G1 is available on Cassandra 5.0, 4.1 and 4.0, and is present in every JDK build those releases support. What differs between them is whether the release starts with it:
| Release | State of the G1 block | File |
|---|---|---|
| 5.0 (all patch versions) | Active on a default install (CASSANDRA-18027) | jvm11-server.options and jvm17-server.options |
| 4.1 | Present and commented out, with the project's own tuning comments | jvm11-server.options |
| 4.0 | Present and commented out, with the project's own tuning comments | jvm11-server.options |
On 4.1 and 4.0 the alternative is CMS, which is what those releases start with. Selecting G1 there is a matter of uncommenting one block and commenting out the other; Enabling G1 on 4.1 and 4.0 gives the lines and Choosing Between CMS and G1 gives the operational case for doing so.
Settings on 5.0
Section titled “Settings on 5.0”These are the values in jvm11-server.options and jvm17-server.options. They are already active; adding them again changes nothing.
| Setting | 5.0.4 and later | 5.0.0 to 5.0.3 | Effect |
|---|---|---|---|
-XX:+UseG1GC | active | active | Selects G1. |
-XX:MaxGCPauseMillis | 300 | 300 | Pause goal G1 sizes its collections against. The JVM default is 200. A larger value raises throughput and lengthens pauses, and it must stay below the request timeouts in cassandra.yaml. |
-XX:G1NewSizePercent | 50 | not set | Floor on the young generation as a percentage of the heap. The setting is experimental in the JVM, so it ships alongside -XX:+UnlockExperimentalVMOptions. |
-XX:MaxTenuringThreshold | 2 | 1 | Survivor copies an object takes before promotion to the old generation. |
-XX:G1HeapRegionSize | 16m | 16m | Fixed region size, in place of the size G1 derives from the heap. |
-XX:InitiatingHeapOccupancyPercent | 70 | 70 | Occupancy at which G1 starts a concurrent cycle. The value is a starting point only, since adaptive sizing replaces it. |
-XX:G1RSetUpdatingPauseTimePercent | 5 | 5 | Share of the pause budget spent on remembered set updates, with the remainder deferred to concurrent work. |
-XX:+ParallelRefProcEnabled | active | active | Reference processing runs on multiple threads during a pause. |
-XX:ParallelGCThreads, -XX:ConcGCThreads | set to the core count by cassandra-env.sh when neither appears in the options files | not set | Equal values for both reduce stop-the-world duration. |
5.0.4 retuned G1 rather than changing collector (CASSANDRA-20296). Three of the rows above differ from 5.0.0 to 5.0.3: the young generation gained a floor of 50% of the heap, the tenuring threshold rose from 1 to 2, and both GC thread counts are set to the host core count. The same ticket changed the heap formula, which is covered in Heap Sizing.
-XX:MaxGCPauseMillis=300 is a target, not a guarantee. G1 sizes each collection against the goal from its own measurements of collection cost, and a collection that has more live data to copy than the prediction accounted for overruns the goal. 300ms sits above the JVM default of 200 for throughput and below the default request timeouts in cassandra.yaml, so a node meeting the goal does not fail requests because of a single collection.
-XX:InitiatingHeapOccupancyPercent=70 is a starting value rather than a fixed trigger. -XX:+G1UseAdaptiveIHOP is on by default from JDK 9 and no shipped option disables it, so the supplied value applies only until G1 has observed enough marking cycles to predict its own threshold, after which G1 sets the trigger itself.
-XX:-UseBiasedLocking is not part of the G1 block. It is a general setting, present in jvm11-server.options on 5.0 and in jvm-server.options on 4.1 and 4.0; see What the Release Already Sets.
The G1 Block on 4.1 and 4.0
Section titled “The G1 Block on 4.1 and 4.0”jvm11-server.options on both releases carries a ### G1 Settings section with every line commented out. The values are the project's, as are the comments explaining them: the file records -XX:MaxGCPauseMillis as the "main G1GC tunable", notes that "lowering the pause target will lower throughput and vise versa", instructs that it be kept "smaller than the timeouts in cassandra.yaml", and describes -XX:InitiatingHeapOccupancyPercent=70 as saving "CPU time on large (>= 16GB) heaps by delaying region scanning until the heap is 70% full".
The two releases do not carry the same block:
| Commented setting | 4.1 | 4.0 |
|---|---|---|
-XX:+UseG1GC | present | present |
-XX:+ParallelRefProcEnabled | present | present |
-XX:MaxTenuringThreshold | 1 | absent from the block |
-XX:G1HeapRegionSize | 16m | absent from the block |
-XX:G1RSetUpdatingPauseTimePercent | 5 | 5 |
-XX:MaxGCPauseMillis | 300 | 500 |
-XX:InitiatingHeapOccupancyPercent | 70, under "Optional G1 Settings" | 70, under "Optional G1 Settings" |
-XX:ParallelGCThreads | 16, optional | 16, optional |
-XX:ConcGCThreads | 16, optional | 16, optional |
Three of those rows change behaviour on 4.0 relative to 4.1. The pause goal on 4.0 is 500ms rather than the 300ms of 5.0 and 4.1, which trades a longer worst-case pause for throughput and leaves less margin below the default request timeouts. 4.0 fixes no region size, so G1 derives one from the heap, targeting about 2048 regions. 4.0 sets no tenuring threshold within the G1 block, so the JVM default of 15 applies once the CMS block is commented out, and objects age through the survivor spaces for longer before promotion than the value of 1 that 4.1 supplies.
The optional lines are commented separately from the rest of the block and are not required to run G1. -XX:ParallelGCThreads=16 and -XX:ConcGCThreads=16 are worked examples rather than values to apply unread: the shipped comment states that machines with more than 10 cores should try setting these at or below the full core count, and that setting both to the same value can reduce stop-the-world durations. On 4.1 and 4.0 cassandra-env.sh sets neither, so leaving both commented gives the JVM defaults, which derive -XX:ParallelGCThreads from the processors available and set -XX:ConcGCThreads to a quarter of it.
Enabling G1 on 4.1 and 4.0
Section titled “Enabling G1 on 4.1 and 4.0”Uncomment the G1 block and comment out the CMS block in the same file. Both steps are required. -XX:+UseConcMarkSweepGC left active selects CMS whatever the G1 lines below it say, and -XX:MaxTenuringThreshold left active in the CMS block continues to apply under G1, which on 4.0 silently replaces the JVM default the G1 block relies on.
# jvm11-server.options on 4.1# Comment out every line of the ### CMS Settings block first
-XX:+UseG1GC-XX:+ParallelRefProcEnabled-XX:MaxTenuringThreshold=1-XX:G1HeapRegionSize=16m-XX:G1RSetUpdatingPauseTimePercent=5-XX:MaxGCPauseMillis=300# jvm11-server.options on 4.0# Comment out every line of the ### CMS Settings block first# The block carries no region size and no tenuring threshold
-XX:+UseG1GC-XX:+ParallelRefProcEnabled-XX:G1RSetUpdatingPauseTimePercent=5-XX:MaxGCPauseMillis=500Two behaviours of cassandra-env.sh change with the collector on 4.1 and 4.0. -Xmn and HEAP_NEWSIZE stop applying: the script appends -Xmn only under CMS, refuses to start the node when -Xmn appears in an options file alongside G1, and prints a warning when HEAP_NEWSIZE is set under G1. MAX_HEAP_SIZE may then be set on its own, without HEAP_NEWSIZE.
The heap itself does not change. cassandra-env.sh on 4.1 and 4.0 computes the same max heap whatever the collector, capped at 8192M, unlike 5.0.4 and later where the cap depends on whether CMS is in use. A node that needs more heap than the computed value under G1 has to pin -Xmx and -Xms, which Changing the Heap covers.
Adjusting G1
Section titled “Adjusting G1”Where collections overrun the pause goal under representative load, change one setting at a time and re-measure.
Raise the young generation floor first on 5.0. 5.0.4 and later ship 50, and CASSANDRA-20296 records 66 as usable in production. The setting is experimental in the JVM, so -XX:+UnlockExperimentalVMOptions must precede it. 5.0.4 and later already set that flag, and 5.0.0 to 5.0.3 do not. Lower the occupancy trigger only once the first change has been measured.
# jvm11-server.options or jvm17-server.options on 5.0
# 1. Young generation floor; requires -XX:+UnlockExperimentalVMOptions-XX:G1NewSizePercent=66
# 2. Start the concurrent cycle earlier, at the cost of more concurrent GC CPU-XX:InitiatingHeapOccupancyPercent=60A larger young generation collects more of the short-lived allocation that Cassandra read and write paths produce, before it is promoted. A lower occupancy trigger starts the concurrent cycle sooner, which reduces the chance of a collection failing to keep up, and spends more CPU on collection while the node serves requests.
The occupancy trigger has a bounded effect, because adaptive sizing remains active. Lowering it therefore changes behaviour in the period after a restart more than in steady state. Pinning the value requires -XX:-G1UseAdaptiveIHOP, which no release ships and which gives up G1's own estimate of how long a concurrent cycle needs.
Where 5.0.4 and later ship a young generation floor of 50% of the heap, 5.0.0 to 5.0.3 do not set -XX:G1NewSizePercent at all, so the JVM default floor of 5% applies. Adding it, together with -XX:+UnlockExperimentalVMOptions, is the first change to try on those patch versions. The same applies to a node running G1 on 4.1 or 4.0, where the block supplies no young generation floor either.
On 4.0, the 500ms pause goal is the first value to examine rather than the young generation. Compare it against read_request_timeout_in_ms and write_request_timeout_in_ms in cassandra.yaml: a goal close to the timeout leaves no margin for a collection that overruns it. Lowering the goal toward the 300ms that 5.0 and 4.1 use costs throughput, which is the trade the shipped comment describes.
Do Not Set -Xmn Under G1
Problem: -Xmn fixes the young generation size, which overrides the pause goal G1 works to. cassandra-env.sh treats the combination as a configuration error rather than ignoring it.
Symptoms: The node does not start. cassandra-env.sh prints "It is not recommended to set -Xmn with the G1 garbage collector" and exits. This applies on 5.0, and on 4.1 and 4.0 once G1 has been selected.
Instead: Set a floor on the young generation as a percentage of the heap, which G1 honours while keeping its pause goal:
# BAD: stops the node starting under G1-Xmn8G
# GOOD: 5.0.4 and later ship 50; raise it to at most 66 against a measurement-XX:+UnlockExperimentalVMOptions-XX:G1NewSizePercent=50Under CMS on 4.1 and 4.0, -Xmn is valid. Setting it in jvm-server.options without also setting -Xmx stops the start, with a message asking for the pair.
Verifying the Running Configuration
Section titled “Verifying the Running Configuration”cassandra-env.sh appends values that appear in no options file, so the command line of the running process is the record of what took effect:
ps -o args= -p "$(pgrep -fn CassandraDaemon)" | tr ' ' '\n' \ | grep -E 'UseG1GC|MaxGCPauseMillis|G1NewSizePercent|InitiatingHeapOccupancyPercent|GCThreads'-XX:+UseG1GC in the output confirms the collector. On 4.1 and 4.0 its absence after an edit usually means the CMS block was left active, since -XX:+UseConcMarkSweepGC further up the same file selects CMS regardless of what follows. On 5.0, a node not reporting G1 has been configured by hand.
Related Documentation
Section titled “Related Documentation”- JVM Tuning - Heap sizing, direct memory, GC logging and the procedure for applying a change
- CMS - The collector 4.1 and 4.0 start with
- Shenandoah - The collector AxonOps recommends over G1, in no options file on any release
- ZGC - In no options file on any release, and not recommended by the project
- JVM Architecture - The pointer-compression boundary, where the heap sits in process memory, and why the heap formula changed
- GC Pause Issues - Long pauses without an allocation failure