Skip to content
AxonOps Documentation

AxonOps — AI-Native Control Plane for Open Source Data Platforms

ZDM Proxy & CDM Setup for DSE Migration

The recommended data-migration path for DSE (and the only supported path for DSE 6.0+) uses two tools together:

  • the Zero-Downtime Migration (ZDM) Proxy, which sits between the application and both clusters and manages traffic routing; and
  • the Cassandra Data Migrator (CDM), which bulk-copies existing data from the source cluster to the target and validates row-level consistency.

This page covers how the two tools fit together, how to size and configure them, and the schema and credential preparation that must be in place before traffic is routed through the proxy. The migration run itself is covered in Data Migration.

The ZDM Proxy is open source under the Apache License 2.0. AxonOps maintains a fork at github.com/axonops/zdm-proxy that adds migration metrics for monitoring the proxy layer and finer control over write consistency to the target (see Overriding the target write consistency level below); AxonOps can extend the fork further for specific environment requirements (contact AxonOps). The upstream DataStax ZDM Proxy targets migrations from Cassandra to DataStax Astra; migrating in the DSE-to-open-source direction benefits from the fork's adjustments.

How ZDM Proxy works

Section titled “How ZDM Proxy works”

The application connects to the ZDM Proxy instead of directly to a cluster. The only client change is the contact point. The proxy then connects to both the source (Origin, the DSE cluster) and the target (Target, the new open-source Apache Cassandra cluster):

  • Writes are sent to both clusters concurrently from the moment the proxy is deployed (dual-writes).
  • Reads initially go to the Origin only. Later in the migration, asynchronous dual reads are enabled, and primary reads are finally switched to the Target.
ZDM Proxy dual-write topology during migration ZDM Proxy: dual-write topology during migration App Application Proxy ZDM Proxy (≥ 3 instances) App->Proxy CQL (single contact point) Origin Origin DSE cluster Proxy->Origin reads (initially) + dual writes Target Target Apache Cassandra Proxy->Target dual writes (+ async reads later) CDM CDM bulk copy + validate Origin->CDM read existing data CDM->Target write existing data

CDM runs alongside the proxy to copy data that already existed before dual-writes began. Dual-writes keep new data in sync; CDM backfills the existing data.

Supported versions

Section titled “Supported versions”

The AxonOps fork of the ZDM Proxy negotiates native protocol versions v2, v3, v4, DSE_V1, and DSE_V2; it does not natively negotiate v5. Cassandra 5.0, which supports v5, connects as a Target over v4, as long as no v5-specific functionality is used. The Origin and Target must share at least one common protocol version.

In practice the proxy supports, as Origin and/or Target:

  • Apache Cassandra from 2.0 through 5.x (Cassandra 5.0 connects as a Target over CQL native protocol v4);
  • DataStax Enterprise 4.8+;
  • DataStax Astra DB.

Thrift is not supported

The ZDM Proxy does not support the legacy Thrift protocol. If a client or cluster only supports Thrift, the application must be moved to CQL (and the cluster potentially upgraded) before the migration begins.

Sizing the migration infrastructure

Section titled “Sizing the migration infrastructure”

The proxy and CDM run on dedicated migration infrastructure, separate from both clusters. For a production migration, plan for at least:

  • 3 ZDM Proxy instances (the minimum supported for production; a single instance is for local testing only);
  • 1 jumphost / monitoring instance; and
  • 1 instance to run CDM.

Size the instances against throughput and data volume, following the hardware recommendations for each tool.

Configuration

Section titled “Configuration”

The proxy is configured with environment variables, or with an equivalent YAML file (the YAML keys are the lower-cased variable names without the ZDM_ prefix). The most important settings:

Terminal window
ZDM_ORIGIN_CONTACT_POINTS=10.0.0.1   # required: DSE cluster
ZDM_ORIGIN_USERNAME=cassandra        # required
ZDM_ORIGIN_PASSWORD=********          # required
ZDM_ORIGIN_PORT=9042
ZDM_TARGET_CONTACT_POINTS=10.0.0.2   # required: Apache Cassandra cluster
ZDM_TARGET_USERNAME=cassandra        # required
ZDM_TARGET_PASSWORD=********          # required
ZDM_TARGET_PORT=9042
ZDM_PROXY_LISTEN_PORT=14002
ZDM_PROXY_LISTEN_ADDRESS=0.0.0.0
ZDM_PRIMARY_CLUSTER=ORIGIN           # reads go to Origin initially
ZDM_READ_MODE=PRIMARY_ONLY           # switch to dual/async reads during validation
ZDM_LOG_LEVEL=INFO
# ZDM_TARGET_CONSISTENCY_LEVEL=LOCAL_ONE  # optional: override write CL on the target

The equivalent YAML file is passed with --config:

origin_contact_points: 10.0.0.1
target_contact_points: 10.0.0.2
origin_username: cassandra
origin_password: "********"
target_username: cassandra
target_password: "********"

Overriding the target write consistency level

Section titled “Overriding the target write consistency level”

A feature of the AxonOps fork is ZDM_TARGET_CONSISTENCY_LEVEL, which sets the consistency level for writes to the target independently of what the application requests. An application issuing writes at LOCAL_QUORUM, for example, can be configured so the proxy writes to the target at LOCAL_ONE.

This is useful during a migration. The target cluster is brand new, and the operator will often perform actions on it that would not be done on a live cluster: adding or replacing nodes, running compactions, restarting for configuration changes, and generally settling it in. Decoupling the target's write consistency from the source means that work can proceed without a transient target hiccup turning into a failed write that propagates back to the application. It also makes dual-writes more tolerant of a slow or briefly unavailable target replica.

Relaxing the target consistency is safe here precisely because of how the migration works. During dual-writes, reads are not served from the target, so the target does not need strong read-your-writes guarantees yet. Any replicas that miss a write at the lower consistency level are reconciled by the full repair run on the target before cutover (see Validation). Strong write consistency is not needed during the copy; the end-of-migration repair restores full consistency before the target takes production reads.

Prepare the target schema, roles, and credentials before deploying the proxy

ZDM Proxy dual-writes begin immediately once the proxy is deployed. If the target keyspaces, tables, roles, permissions, and user passwords are not already in place, writes to the target will fail, and those failures can propagate back to clients. Before deploying the proxy, the following must be complete:

  1. Recreate all keyspaces and tables on the target with matching definitions.
  2. Recreate all roles, permissions, and user passwords. CDM does not migrate system tables, so authentication credentials are not copied automatically; they must be created manually.
  3. Confirm every target node reports UN (Up/Normal) in nodetool status.

About CDM

Section titled “About CDM”

The Cassandra Data Migrator copies user table data only (not system tables) and validates row-level consistency between source and target. It writes run metadata to cdm_run_info and cdm_run_details tables in each targeted keyspace, and prints progress to standard output. Redirect that output to a log file so any issues can be investigated later.

CDM has important behavioural caveats around collections, user-defined types (UDTs), TTLs, and counter tables. Review Operational Caveats before running it.

Cluster-level monitoring does not surface what happens inside the proxy layer. The AxonOps ZDM Proxy fork exposes additional migration metrics (primary-cluster reads, dual reads, and target-cluster reads) that capture degradations or failures visible only at the proxy; run these alongside cluster-level monitoring. See AxonOps for Cassandra.

Next: Phase 3: DSE to Cassandra data migration.