Introduction
AxonOps is the only solution for the one-stop operation of Apache Cassandra®. Built by Cassandra experts and underpinned by a secure and efficient bi-directional protocol, it combines the essential functionality to enable Site Reliability Engineering to observe, maintain and protect their clusters. Key features include pre-configured dashboards, alerting with enterprise integration, performance-adaptive repairs and highly configurable backups with fine-grained retention management.
This blog looks at monitoring Cassandra on Kubernetes with AxonOps. Cassandra is still primarily run on bare metal or virtual machines. As more companies migrate to Kubernetes (and cloud providers) they are also moving some of the database servers such as Cassandra into Kubernetes. Kubernetes presents several additional challenges and that’s why we are adding support for Kubernetes to our AxonOps products.
k8ssandra
k8ssandra is a Kubernetes Operator for running Apache Cassandra®. With K8ssandra you can quickly deploy and run Apache Cassandra® and by default, it would add grafana and prometheus for monitoring. I’ll show you below how to use AxonOps instead for richer and more complete Cassandra observability.
AxonOps
Before you start, get onto https://axonops.com and sign up for a Free Plan. Then, if you browse to the AxonOps console you should be able to grab a copy of the agent’s config file you’ll require for the next step.
Install k8ssandra operator
The k8ssandra-operator is very easy to install from helm charts. It requires cert-manager as well, if you don’t have it yet, remember to install it first. The example below uses a helmfile to install both k8ssandra-operator and cert-manager.
Then save this as helmfile.yaml then just do:
AxonOps Agent
Take the AxonOps Agent config you previously obtained from the AxonOps console and convert it into a configmap to be mounted into the pods:
Make sure you have an active plan enabled on the AxonOps Console. You can get a Free Plan that allows you to connect up to 5 nodes.
Apache Cassandra Cluster
The sample config below should work on minikube but you should obviously adjust memory, CPU and disk settings according to your set-up. Make sure (if using minikube) that there are enough memory and CPU resources available.
I’m using a MacBook to run minikube and these are the options I require. You may need slightly different arguments depending on whether you use Mac, Windows or Linux.
Storage Class
Please pay special attention to the storage class for the storage. If you’re running it locally (lab environment like minikube) you use either the standard k8s.io/minikube-hostpath or install instead the rancher.io/local-path provisioner.
On cloud environments running managed Kubernetes such as AWS or Google, you can usually leave the storageClassName empty to use the default configuration.
Note: I’ve encountered a bug on the minikube storage provisioner when starting multiple nodes. You may want to use the rancher’s provisioner instead which does work as expected.
Config
The configuration below deploys an Apache Cassandra cluster using a k8ssandra resource definition. You can find some examples at the k8ssandra.io website. The important parts for adding AxonOps to the cluster are:
- initContainer: this is used to deploy the AxonOps Agent configuration to the Cassandra pods
- volumes and volumeMounts: the AxonOps Agent requires access to the Apache Cassandra logs and they also need to share a volume where the network socket and configuration are hosted.
- axon-agent: there is an additional container added to the pod running the AxonOps Agent.
- jvmOptions: we’re injecting the AxonOps Agent as a
javaagentto Apache Cassandra
Running
Once you have saved this configuration to a file, you only need to apply it. Remember to create the namespace first, if it does not exist already.
If everything worked, you should be able to list the running Cassandra pods:
As they get bootstrapped, they will start appearing in the AxonOps Dashboard.
Conclusion
Monitoring Cassandra on Kubernetes with AxonOps is very simple. This blog focuses on k8ssandra but if you don’t use the operator but a helm chart or a custom deployment, AxonOps can be added in a similar way. Get in touch if you need help.
About the Author
Sergio Rua is a Principal DevOps specialist at Digitalis.io working with many customers managing and advising on Kubernetes, Kafka, Cassandra, Elasticsearch and other cool technologies.
