Skip to content

AxonOps — AI-Native Control Plane for Open Source Data Platforms

Kafka Compliance: Authentication & Authorisation

The Access Denials page reports rejected access to a Kafka cluster from two angles. Failed broker authentications occur when a client cannot prove its identity. Authorisation denials occur when an authenticated principal (the identity Kafka assigns to a client, such as User:orders-service) attempts an operation that its access-control list (ACL) entries do not permit. Together they show where access is being refused, and why, across the selected time range.

Both sets of figures are parsed, aggregated, and time bucketed on the AxonOps server; the browser receives only the summarised result.

Every figure on this page leads to the log lines behind it. Selecting a row in a ranked list, such as a denied principal or a failure reason, opens Logs & Events filtered to the relevant broker log and that term. The bars on the two time series are not selectable. Instead, dragging across a range of either chart narrows the page time range to that range, and the Open in Logs & Events button below each chart opens the matching log lines for the whole selected time range. See Navigating from a signal to its data.

The page reads two event sources on the AxonOps server:

SignalEvent sourceLines counted
Authentication failuresA source path ending in server.logLines containing the phrase “Failed authentication”
Authorisation denialsA source path containing authorizer (case-insensitive)Lines containing “Denied”

Both signals depend on AxonOps log collectors for the Kafka brokers, configured as described in Setting up log collection. Each broker requires a collector for the server.log file in its log directory and, where an authorizer is configured, a collector for the kafka-authorizer.log file in the same directory. The page matches on the file name, not the directory, so the collector path follows wherever the broker writes its logs. On AxonOps Cloud deployments, log collectors are managed by AxonOps.

Once the collectors are in place, the following checks confirm that both sources are being read:

  1. In Logs & Events, filter on the server.log source and confirm that events are arriving from every broker, not only some of them.
  2. Confirm that authorizer.class.name is set on every broker and that the broker’s log4j configuration routes the kafka.authorizer.logger logger to kafka-authorizer.log. Kafka logs denials from this logger at INFO and allowed requests at DEBUG, so the default level records denials.
  3. On the Compliance Overview, confirm that the Authorisation Denials tile shows a count rather than Not measured.

Totals and ranked lists cover different populations

Section titled “Totals and ranked lists cover different populations”

The total count and the time series cover every matching line in the selected time range. The ranked lists (failure reasons, source hosts, denied principals, denied operations, and denied resources) do not. AxonOps tallies them from only the most recent matching events, up to the server’s event-query limit of 200 events by default. On a self-hosted deployment the limit is set by the max_results_events_query setting in the database section of the axon-server configuration or by the DATABASE_MAX_RESULT_EVENTS_PER_QUERY environment variable; raising it increases the cost of every event query the server runs. On AxonOps Cloud the limit is managed by AxonOps. When the range holds more matching events than the limit, the ranked counts are lower bounds over the most recent events, and the page does not indicate that the lists are truncated. A principal, host, or reason that appears only in older events in the range is absent from the lists. Narrowing the time range, or opening the lines in Logs & Events, gives a complete view.

Failed authentications indicate misconfigured clients, expired or incorrect credentials, and brute-force or credential-stuffing attempts against the brokers.

AxonOps derives these figures from the “Failed authentication” lines written to each broker’s server.log. The figures therefore depend on each broker writing these lines to its log and on AxonOps collecting that log from every broker. Kafka writes them under its default logging configuration in both 4.x and 3.x. No dedicated logger exists for authentication failures, so they fall through to the root logger and its server.log appender. In Kafka 4.x that appender is defined in the default log4j2.yaml; in Kafka 3.x it is defined in the default log4j.properties.

AxonOps does not detect whether server.log is being collected from every broker, and offers no equivalent to the authorizer-log warning described below. A zero authentication-failure count should therefore be confirmed against the cluster’s log collection configuration before it is read as a clean result. A broker whose logging is reconfigured to suppress the lines, or whose log is not collected, contributes no failures, and this page does not distinguish that case from a broker with none.

The headline figure on the Compliance Overview Authentication Failures tile is the total number of failed authentications in the selected time range. The tile reports Pass at zero, Warning from 1 to 9, and Fail at 10 or more; the same verdict colours the count below the Authentication Failures Over Time chart. A cluster with correctly configured clients and credentials reports few or no failures over a sustained period. A rising or sustained count indicates credentials or client configuration that need investigation, and a large number of failures from a single source over a short interval is consistent with a brute-force or credential-stuffing attempt.

A time series of the failed authentication count across the selected time range. AxonOps divides the range into 25 buckets and reports the total and the peak count for a single bucket below the chart. A sustained baseline points to a persistently misconfigured client. A sharp spike points to a discrete event, such as a credential rotation that was not applied to every client.

The distinct authentication failure reasons, ranked by frequency. Reasons originate from the broker authentication layer; a common example is an invalid username or password. A single reason accounting for most failures points to a widespread credential problem; a reason seen only occasionally points to an isolated mechanism or protocol misconfiguration.

The client source hosts producing the most failed authentications, ranked by count. A single host dominating the ranking localises the problem to one client deployment, or identifies the origin of a brute-force attempt. Failures spread evenly across many hosts point instead to a cluster-wide credential or configuration change.

An authorisation denial occurs when an authenticated principal attempts an operation that its ACL entries do not permit. Denials indicate over-broad access attempts, mis-scoped clients, or an ACL policy that does not match how applications use the cluster.

AxonOps derives these figures from the broker authorizer log (kafka-authorizer.log). These signals are meaningful only where an authorizer is configured on the brokers (authorizer.class.name set) and its log is collected. A cluster with no authorizer permits every request, so the denial count is permanently zero; a zero count on such a cluster reflects the absence of enforcement, not access that is correctly scoped.

Two further paths permit requests without producing a denial on a cluster that does have an authorizer. Principals listed in the broker’s super.users setting bypass ACL checks entirely, so they are never denied. Where allow.everyone.if.no.acl.found is set to true (the default is false), a request to a resource that has no ACLs is allowed rather than denied. The absence of denials is therefore not evidence that access is correctly scoped; the super.users list and the allow.everyone.if.no.acl.found setting should be reviewed alongside the count.

AxonOps detects the second of these two conditions and states it in the page. Where no log collector is pointed at the authorizer log, a warning inside each of the Top Denied Principals, Top Denied Operations, and Top Denied Resources panels records that denials are not being collected and that the figures stay at zero regardless of what the brokers deny. The Authorisation Denials Over Time chart carries no such warning and shows its empty state. On the Compliance Overview, the Authorisation Denials tile reports Not measured, with the reason “No kafka-authorizer.log collector configured”, in place of a count. The collector is matched on its file name rather than its display name, because the file name is what becomes the event source on the AxonOps server; a collector named for the authorizer log but pointed at another file therefore counts as absent. Administrators of a self-hosted deployment can follow the warning’s Configure log collectors link to the cluster’s log collector settings; log collectors are managed by AxonOps on Cloud deployments, so the link is not offered there.

The headline figure on the Compliance Overview Authorisation Denials tile is the total number of denials in the selected time range. The tile reports Pass at zero, Warning from 1 to 24, and Fail at 25 or more; the same verdict colours the count below the Authorisation Denials Over Time chart. A cluster whose ACLs match its client workload reports few denials. A sustained or rising count indicates clients operating outside their granted permissions, or ACLs that need correction.

A time series of the denial count across the selected time range, divided into 25 buckets, with the total and the peak count for a single bucket reported below the chart. A steady baseline indicates a persistent client misconfiguration. A spike indicates a new deployment or a permission change that removed access an application still depends on.

The principals accumulating the most denials, ranked by count over the most recent denial events (see Totals and ranked lists cover different populations). A single principal dominating the ranking identifies the client identity most often operating outside its granted permissions.

The Kafka operations, such as Read, Write, Describe, or Create, most often denied, ranked by count. The leading operations show whether the permission gap is on the read path, the write path, or an administrative operation.

The resources, such as topics, consumer groups, or the cluster, most often denied, ranked by count. The leading resources identify the specific topics or groups where the ACL policy does not match client use.

The authentication mechanisms that produce failed-authentication events, including SASL/SCRAM, SASL/PLAIN, mTLS, OAuth/OIDC, Kerberos, and delegation tokens, are documented in the Kafka Authentication reference. The ACL model, resource types, operations, and authorizer configuration behind denials are documented in the Kafka Authorization reference. ACL management within AxonOps is documented in Topic Access Control List (ACL).

The signals on this page support the following ISO/IEC 27002:2022 controls: 5.15 Access control, 5.16 Identity management, 5.17 Authentication information, 5.18 Access rights, 8.3 Information access restriction, 8.5 Secure authentication, 8.15 Logging, and 8.16 Monitoring activities.