AxonOps Agent Set up

AxonOps Cloud Agent Network Requirements

axon-agent connects securely to the following AxonOps Cloud service endpoint:

https://agents.axonops.cloud

The TLS HTTPS connection initiated by the agent is upgraded to a WebSocket connection and thus requires WebSocket support in your corporate infrastructure, such as a secure web proxy service.

If you have a DNS-based security policy, you must allow outbound access to the following domain.

agents.axonops.cloud

If you have an IP address-based security policy, you must open access to the IP address ranges provided in the following links.

https://agents.axonops.cloud/ips-v4
https://agents.axonops.cloud/ips-v6

To test connectivity, execute the following command:

curl https://agents.axonops.cloud/test.html

You should expect the following response:

AxonOps Agent Test Page

Set up the AxonOps repository and install axon-agent

Select the OS Family

Execute the following commands to setup the AxonOps repository:

sudo apt-get update
sudo apt-get install -y curl gnupg ca-certificates

curl -L https://packages.axonops.com/apt/repo-signing-key.gpg \
  | sudo gpg --dearmor -o /usr/share/keyrings/axonops.gpg

echo "deb [arch=arm64,amd64 signed-by=/usr/share/keyrings/axonops.gpg]\
  https://packages.axonops.com/apt axonops-apt main" \
  | sudo tee /etc/apt/sources.list.d/axonops-apt.list
sudo apt-get update

Select the service you want to configure

Install Cassandra Agent

Select the Cassandra Version

Select the Java Version

Install the AxonOps Cassandra Agent and its dependency axon-agent:

sudo apt-get install axon-cassandra3.0-agent

Configuration File Locations

AxonOps Cassandra Agent

The AxonOps Cassandra Agent is the jar that is directly loaded by Cassandra. The AxonOps Cassandra Agent then reaches out directly to the AxonOps Agent binary which contacts the AxonOps Server directly.

Configuration File: /etc/axonops/axon-agent.yml
Binary: /usr/share/axonops/axon-cassandra{version}-agent.jar
Version number: /usr/share/axonops/axon-cassandra{version}-agent.version
Copyright: /usr/share/doc/axonops/axon-cassandra{version}-agent/copyright
Licenses: /usr/share/axonops/licenses/axon-cassandra{version}-agent/

AxonOps Agent

The axon-agent is a dependency of the AxonOps Cassandra Agent. This binary contacts the AxonOps Server directly while minimizing the memory footprint and CPU utilization of the Cassandra process.

Configuration File: /etc/axonops/axon-agent.yml
Binary: usr/share/axonops/axon-agent
Logs: /var/log/axonops/axon-agent.log
Systemd service: /usr/lib/systemd/system/axon-agent.service

Agent Configuration

Update the following highlighted lines from /etc/axonops/axon-agent.yml:

Please update the key and org values, they can be viewed by logging into console.axonops.cloud

Organization (org) name is next to the logo in the console
Agent Keys (key) found in Agent Setup

agent_keys

If there is a Dedicated NTP server in your Organization please uncomment and update the NTP section.

  axon-server:
      hosts: "agents.axonops.cloud"

  axon-agent:
      key: <THIS_IS_A_DUMMY_KEY_PLEASE_UPDATE>
      org: <THIS_IS_A_DUMMY_ORG_NAME_PLEASE_UPDATE>

  # Specify the NTP server IP addresses or hostnames configured for your Cassandra hosts
  # if using Cassandra deployed in Kubernetes or if auto-detection fails.
  # The port defaults to 123 if not specified.
  # NTP:
  #    hosts:
  #        - "x.x.x.x:123"
  # Optionally restrict which commands can be executed by axon-agent.
  # If "true", only scripts placed in scripts_location can be executed by axon-agent.
  # disable_command_exec: false
  # If disable_command_exec is true then axon-agent is only allowed to execute scripts
  # under this path
  # scripts_location: /var/lib/axonops/scripts/

Set file permissions on /etc/axonops/axon-agent.yml file by executing the following command

sudo chmod 0640 /etc/axonops/axon-agent.yml

Configure Cassandra

Add the following line at the end of /etc/cassandra/cassandra-env.sh:

JVM_OPTS="$JVM_OPTS -javaagent:/usr/share/axonops/axon-cassandra3.0-agent.jar=/etc/axonops/axon-agent.yml"

If Cassandra was installed using a tarball, the correct path for the cassandra-env.sh will be <Cassandra Installation Directory>/conf/cassandra-env.sh.

NB. Make sure this configuration is not overridden by automation tools.

Configure Cassandra user group

Configure the Linux user groups by:

Adding the axonops user to the cassandra user group.
Adding the cassandra user to the axonops user group.

CASSANDRA_GROUP=cassandra
CASSANDRA_USER=cassandra

sudo usermod -aG "$CASSANDRA_GROUP" axonops
sudo usermod -aG axonops "$CASSANDRA_USER"

If Cassandra was setup to use a non-default user or group, CASSANDRA_GROUP and/or CASSANDRA_USER will need be updated accordingly for the above commands to work properly.

Uninstall `needrestart` Package

Due to recent change with needrestart, as seen in Ubuntu 24.04, uninstalling the needrestart package is currently recommended.

Failure to uninstall the needrestart package may cause the Cassandra service to be restarted when updating the axon-agent package.

Start/Restart Cassandra

To load the AxonOps Java Agent and Cassandra config changes please,

Start the Cassandra service if stopped.
Restart the Cassandra service if already running.

Install Kafka Agent

Select the Kafka Version

Install the AxonOps Kafka Agent and its dependency axon-agent:

sudo apt-get install axon-kafka2-agent

sudo apt-get install axon-kafka3-agent

sudo apt-get install axon-kafka4-agent

sudo apt-get install axon-kafka2-agent

sudo apt-get install axon-kafka3-agent

sudo apt-get install axon-kafka4-agent

Configuration File Locations

AxonOps Kafka Agent

The AxonOps Kafka Agent is the jar that is directly loaded by Kafka. The AxonOps Kafka Agent then reaches out directly to the AxonOps Agent binary which contacts the AxonOps Server directly.

Configuration File: /etc/axonops/axon-agent.yml
Binary: /usr/share/axonops/axon-kafka{version}-agent.jar
Version number: /usr/share/axonops/axon-kafka{version}-agent.version
Copyright: /usr/share/doc/axonops/axon-kafka{version}-agent/copyright
Licenses: /usr/share/axonops/licenses/axon-kafka{version}-agent/

AxonOps Agent

The AxonOps Agent is a dependency of the AxonOps Kafka Agent. This binary contacts the AxonOps Server directly while minimizing the memory footprint and CPU utilization of the Kafka process.

Configuration File: /etc/axonops/axon-agent.yml
Binary: usr/share/axonops/axon-agent
Logs: /var/log/axonops/axon-agent.log
Systemd service: /usr/lib/systemd/system/axon-agent.service

Agent Configuration

Update the key and org values within the highlighted lines of /etc/axonops/axon-agent.yml below.

The values can be found by logging into console.axonops.cloud:

Organization (org) can be found next to the logo at the top of the console.
Agent Keys (key) can be found within Agent setup > Agent keys.

Console Screenshot

If there is a Dedicated NTP server in your Organization please uncomment and update the NTP section.

axon-server:
  hosts: "agents.axonops.cloud" # AxonOps SaaS
  # hosts: "${AXONOPS_SERVER_HOSTS}" # AxonOps Server On-Premise Endpoint
  # port: 1888 # AxonOps Server On-Premise Port (Default is 1888)

axon-agent:
  key: "<THIS_IS_A_DUMMY_KEY_PLEASE_UPDATE>"
  org: "<THIS_IS_A_DUMMY_ORG_NAME_PLEASE_UPDATE>"
  cluster_name: "<THIS_IS_A_DUMMY_CLUSTER_NAME_PLEASE_UPDATE>"
  tls:
    mode: "TLS" # disabled, TLS
    #skipVerify: false # Disables CA and Hostname verification
    #caFile: "path_to_certs_on_axon_agent_node.crt" # required if skipVerify is not set and you are using a self-signed cert
    #certFile: "path_to_certs_on_axon_agent_node.crt"
    #keyFile: "path_to_key_file_on_axon_agent_node.key"
# Specify the NTP server IP addresses or hostnames configured for your hosts
# The port defaults to 123 if not specified.
# NTP:
#    hosts:
#        - "x.x.x.x:123"
# Optionally restrict which commands can be executed by axon-agent.
# If "true", only scripts placed in scripts_location can be executed by axon-agent.
# disable_command_exec: false
# If disable_command_exec is true then axon-agent is only allowed to execute scripts
# under this path
# scripts_location: /var/lib/axonops/scripts/

kafka:
  node_type: "broker" # broker, kraft-broker, kraft-controller, zookeeper, connect
  # rack: "testrack" # Auto-detected from Kafka config, optionally override the rack to group nodes in AxonOps
  kafka_client:
    brokers: ["<host_listener_ip_address_or_fqdn>:<port>>"] # 10.0.0.2:9092 or 10.20.30.40:9094 or this_is_my_server.domain.com:9093
    # Authentication Settings
    sasl:
      username: <THIS_IS_A_DUMMY_USERNAME_PLEASE_UPDATE>
      password: <THIS_IS_A_DUMMY_PASSWORD_PLEASE_UPDATE>
      mechanism: PLAIN # SCRAM-SHA-256, SCRAM-SHA-512, OAUTHBEARER, GSSAPI
      # For oauth support
      # oauth:
      #   token:
      #   clientId:
      #   clientSecret:
      #   tokenEndpoint:
      #   scope:
      # For gssapi support
      # gssapi:
      #   authType:
      #   keyTabPath:
      #   kerberosConfigPath:
      #   serviceName:
      #   username:
      #   password:
      #   realm:
      #   enableFast: true
    # SSL settings for connection to Kafka
    tls:
      enabled: true
      caFilepath: <THIS_IS_A_DUMMY_CA_PATH_PLEASE_UPDATE>
      insecureSkipTlsVerify: false

axon-server:
  hosts: "agents.axonops.cloud" # AxonOps SaaS
  # hosts: "${AXONOPS_SERVER_HOSTS}" # AxonOps Server On-Premise Endpoint
  # port: 1888 # AxonOps Server On-Premise Port (Default is 1888)

axon-agent:
  key: "<THIS_IS_A_DUMMY_KEY_PLEASE_UPDATE>"
  org: "<THIS_IS_A_DUMMY_ORG_NAME_PLEASE_UPDATE>"
  cluster_name: "<THIS_IS_A_DUMMY_CLUSTER_NAME_PLEASE_UPDATE>"
  tls:
    mode: "TLS" # disabled, TLS
    #skipVerify: false # Disables CA and Hostname verification
    #caFile: "path_to_certs_on_axon_agent_node.crt" # required if skipVerify is not set and you are using a self-signed cert
    #certFile: "path_to_certs_on_axon_agent_node.crt"
    #keyFile: "path_to_key_file_on_axon_agent_node.key"

# Specify the NTP server IP addresses or hostnames configured for your hosts
# The port defaults to 123 if not specified.
# NTP:
#    hosts:
#        - "x.x.x.x:123"
# Optionally restrict which commands can be executed by axon-agent.
# If "true", only scripts placed in scripts_location can be executed by axon-agent.
# disable_command_exec: false
# If disable_command_exec is true then axon-agent is only allowed to execute scripts
# under this path
# scripts_location: /var/lib/axonops/scripts/

kafka:
  node_type: "zookeeper" # broker, kraft-broker, kraft-controller, zookeeper, connect
  # rack: "testrack" # Optionally specify a rack to group nodes in AxonOps

axon-server:
  hosts: "agents.axonops.cloud" # AxonOps SaaS
  # hosts: "${AXONOPS_SERVER_HOSTS}" # AxonOps Server On-Premise Endpoint
  # port: 1888 # AxonOps Server On-Premise Port (Default is 1888)

axon-agent:
  key: "<THIS_IS_A_DUMMY_KEY_PLEASE_UPDATE>"
  org: "<THIS_IS_A_DUMMY_ORG_NAME_PLEASE_UPDATE>"
  cluster_name: "<THIS_IS_A_DUMMY_CLUSTER_NAME_PLEASE_UPDATE>"
  tls:
    mode: "TLS" # disabled, TLS
    #skipVerify: false # Disables CA and Hostname verification
    #caFile: "path_to_certs_on_axon_agent_node.crt" # required if skipVerify is not set and you are using a self-signed cert
    #certFile: "path_to_certs_on_axon_agent_node.crt"
    #keyFile: "path_to_key_file_on_axon_agent_node.key"

# Specify the NTP server IP addresses or hostnames configured for your hosts
# The port defaults to 123 if not specified.
# NTP:
#    hosts:
#        - "x.x.x.x:123"
# Optionally restrict which commands can be executed by axon-agent.
# If "true", only scripts placed in scripts_location can be executed by axon-agent.
# disable_command_exec: false
# If disable_command_exec is true then axon-agent is only allowed to execute scripts
# under this path
# scripts_location: /var/lib/axonops/scripts/

kafka:
  node_type: "kraft-broker" # broker, kraft-broker, kraft-controller, zookeeper, connect
  # rack: "testrack" # Auto-detected from Kafka config, optionally override the rack to group nodes in AxonOps
  kafka_client:
    brokers: ["<host_listener_ip_address_or_fqdn>:<port>>"] # 10.0.0.2:9092 or 10.20.30.40:9094 or this_is_my_server.domain.com:9093
    # Authentication Settings
    sasl:
      username: <THIS_IS_A_DUMMY_USERNAME_PLEASE_UPDATE>
      password: <THIS_IS_A_DUMMY_PASSWORD_PLEASE_UPDATE>
      mechanism: PLAIN # SCRAM-SHA-256, SCRAM-SHA-512, OAUTHBEARER, GSSAPI
      # For oauth support
      # oauth:
      #   token:
      #   clientId:
      #   clientSecret:
      #   tokenEndpoint:
      #   scope:
      # For gssapi support
      # gssapi:
      #   authType:
      #   keyTabPath:
      #   kerberosConfigPath:
      #   serviceName:
      #   username:
      #   password:
      #   realm:
      #   enableFast: true
    # SSL settings for connection to Kafka
    tls:
      enabled: true
      caFilepath: <THIS_IS_A_DUMMY_CA_PATH_PLEASE_UPDATE>
      insecureSkipTlsVerify: false

axon-server:
  hosts: "agents.axonops.cloud" # AxonOps SaaS
  # hosts: "${AXONOPS_SERVER_HOSTS}" # AxonOps Server On-Premise Endpoint
  # port: 1888 # AxonOps Server On-Premise Port (Default is 1888)

axon-agent:
  key: "<THIS_IS_A_DUMMY_KEY_PLEASE_UPDATE>"
  org: "<THIS_IS_A_DUMMY_ORG_NAME_PLEASE_UPDATE>"
  cluster_name: "<THIS_IS_A_DUMMY_CLUSTER_NAME_PLEASE_UPDATE>"
  tls:
    mode: "TLS" # disabled, TLS
    #skipVerify: false # Disables CA and Hostname verification
    #caFile: "path_to_certs_on_axon_agent_node.crt" # required if skipVerify is not set and you are using a self-signed cert
    #certFile: "path_to_certs_on_axon_agent_node.crt"
    #keyFile: "path_to_key_file_on_axon_agent_node.key"

# Specify the NTP server IP addresses or hostnames configured for your hosts
# The port defaults to 123 if not specified.
# NTP:
#    hosts:
#        - "x.x.x.x:123"
# Optionally restrict which commands can be executed by axon-agent.
# If "true", only scripts placed in scripts_location can be executed by axon-agent.
# disable_command_exec: false
# If disable_command_exec is true then axon-agent is only allowed to execute scripts
# under this path
# scripts_location: /var/lib/axonops/scripts/

kafka:
  node_type: "kraft-controller" # broker, kraft-broker, kraft-controller, zookeeper, connect
  # rack: "testrack" # Auto-detected from Kafka config, optionally override the rack to group nodes in AxonOps
  kafka_client:
    brokers: ["<host_listener_ip_address_or_fqdn>:<port>>"] # 10.0.0.2:9092 or 10.20.30.40:9094 or this_is_my_server.domain.com:9093
    # Authentication Settings
    sasl:
      username: <THIS_IS_A_DUMMY_USERNAME_PLEASE_UPDATE>
      password: <THIS_IS_A_DUMMY_PASSWORD_PLEASE_UPDATE>
      mechanism: PLAIN # SCRAM-SHA-256, SCRAM-SHA-512, OAUTHBEARER, GSSAPI
      # For oauth support
      # oauth:
      #   token:
      #   clientId:
      #   clientSecret:
      #   tokenEndpoint:
      #   scope:
      # For gssapi support
      # gssapi:
      #   authType:
      #   keyTabPath:
      #   kerberosConfigPath:
      #   serviceName:
      #   username:
      #   password:
      #   realm:
      #   enableFast: true
    # SSL settings for connection to Kafka
    tls:
      enabled: true
      caFilepath: <THIS_IS_A_DUMMY_CA_PATH_PLEASE_UPDATE>
      insecureSkipTlsVerify: false

axon-server:
  hosts: "agents.axonops.cloud" # AxonOps SaaS
  # hosts: "${AXONOPS_SERVER_HOSTS}" # AxonOps Server On-Premise Endpoint
  # port: 1888 # AxonOps Server On-Premise Port (Default is 1888)

axon-agent:
  key: "<THIS_IS_A_DUMMY_KEY_PLEASE_UPDATE>"
  org: "<THIS_IS_A_DUMMY_ORG_NAME_PLEASE_UPDATE>"
  cluster_name: "<THIS_IS_A_DUMMY_CLUSTER_NAME_PLEASE_UPDATE>"
  tls:
    mode: "TLS" # disabled, TLS
    #skipVerify: false # Disables CA and Hostname verification
    #caFile: "path_to_certs_on_axon_agent_node.crt" # required if skipVerify is not set and you are using a self-signed cert
    #certFile: "path_to_certs_on_axon_agent_node.crt"
    #keyFile: "path_to_key_file_on_axon_agent_node.key"

# Specify the NTP server IP addresses or hostnames configured for your hosts
# The port defaults to 123 if not specified.
# NTP:
#    hosts:
#        - "x.x.x.x:123"
# Optionally restrict which commands can be executed by axon-agent.
# If "true", only scripts placed in scripts_location can be executed by axon-agent.
# disable_command_exec: false
# If disable_command_exec is true then axon-agent is only allowed to execute scripts
# under this path
# scripts_location: /var/lib/axonops/scripts/

kafka:
  node_type: "connect" # broker, kraft-broker, kraft-controller, zookeeper, connect
  # rack: "testrack" # Optionally specify a rack to group nodes in AxonOps
  connect_client:
    enabled: true
    connectTimeout: 5s
    readTimeout: 5s
    requestTimeout: 5s
    clusters:
      - name: "connect-prod"
        url: "https://connect.example.com:8083"
        username: "connect_user"
        password: "connect_password"
        tls:
          enabled: true
          caFilepath: "/etc/ssl/certs/ca.pem"
          certFilepath: "/etc/ssl/certs/client.pem"
          keyFilepath: "/etc/ssl/private/client.key"
          insecureSkipTlsVerify: false

Set file permissions on /etc/axonops/axon-agent.yml file by executing the following command

sudo chmod 0640 /etc/axonops/axon-agent.yml

Configure Kafka

Edit kafka-server-start.sh, usually located in your Kafka install path such as:

<Kafka_Home>/bin/kafka-server-start.sh

Configure Zookeeper

Edit zookeeper-server-start.sh, usually located in your Zookeeper install path such as:

<Zookeeper_Home>/bin/zookeeper-server-start.sh

Configure KRaft Broker

Edit kafka-server-start.sh, usually located in your Kafka install path such as:

<Kafka_Home>/bin/kafka-server-start.sh

Configure KRaft Controller

Edit kafka-server-start.sh, usually located in your Kafka install path such as:

<Kafka_Home>/bin/kafka-server-start.sh

Configure Connect

Edit connect-distributed.sh, usually located in your Kafka install path such as:

<Kafka_Home>/bin/connect-distributed.sh

Insert the following code block into the above file, right before the final line which contains the exec statement:

# Please note the period(.) at the beginning of the config line.
. /usr/share/axonops/axonops-jvm.options

The final state of the last lines of the file should look like this:

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS org.apache.zookeeper.server.quorum.QuorumPeerMain "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

. /usr/share/axonops/axonops-jvm.options

exec $(dirname $0)/kafka-run-class.sh $EXTRA_ARGS org.apache.kafka.connect.cli.ConnectDistributed "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS org.apache.zookeeper.server.quorum.QuorumPeerMain "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

. /usr/share/axonops/axonops-jvm.options

exec $(dirname $0)/kafka-run-class.sh $EXTRA_ARGS org.apache.kafka.connect.cli.ConnectDistributed "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS org.apache.zookeeper.server.quorum.QuorumPeerMain "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

. /usr/share/axonops/axonops-jvm.options

exec $(dirname $0)/kafka-run-class.sh $EXTRA_ARGS org.apache.kafka.connect.cli.ConnectDistributed "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS org.apache.zookeeper.server.quorum.QuorumPeerMain "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

. /usr/share/axonops/axonops-jvm.options

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

. /usr/share/axonops/axonops-jvm.options

exec $(dirname $0)/kafka-run-class.sh $EXTRA_ARGS org.apache.kafka.connect.cli.ConnectDistributed "$@"

NB. Make sure that this configuration will not get overridden by an automation tool.