Kafka Authentication

Authentication verifies the identity of clients connecting to Kafka brokers and brokers communicating with each other. Kafka supports multiple authentication mechanisms to integrate with different security infrastructures.

---

Authentication Mechanisms

Mechanism	Protocol	Description	Documentation
SASL/SCRAM	SASL	Salted challenge-response with SHA-256/512	SASL/SCRAM Guide
SASL/PLAIN	SASL	Simple username/password	SASL/PLAIN Guide
SASL/GSSAPI	SASL	Kerberos authentication	Kerberos Guide
SASL/OAUTHBEARER	SASL	OAuth 2.0 / OIDC tokens	OAuth Guide
mTLS	SSL/TLS	Mutual TLS certificates	mTLS Guide
Delegation Tokens	SASL	Lightweight token-based auth	Delegation Tokens

---

Mechanism Selection Guide

Decision Matrix

Requirement	SCRAM	PLAIN	Kerberos	OAuth	mTLS
No external infrastructure	✅	✅	❌	❌	✅
Enterprise SSO integration	❌	❌	✅	✅	❌
Cloud-native environments	✅	❌	❌	✅	✅
Certificate-based identity	❌	❌	❌	❌	✅
Password-based auth	✅	✅	❌	❌	❌
Token refresh support	❌	❌	✅	✅	❌
Simple to set up	✅	✅	❌	⚠️	⚠️

Legend: ✅ Supported | ⚠️ Partial/Experimental | ❌ Not Supported

Recommended Use Cases

Environment	Recommended Mechanism	Rationale
Development	SASL/PLAIN or SASL/SCRAM	Simple setup, no external dependencies
Production (standalone)	SASL/SCRAM-SHA-512	Secure, no external infrastructure needed
Enterprise (AD/Kerberos)	SASL/GSSAPI	Integrates with existing Kerberos KDC
Cloud-native	SASL/OAUTHBEARER	Integrates with cloud identity providers
PKI environment	mTLS	Certificate-based, no passwords
Managed Kafka	Provider-specific	Follow provider recommendations

---

Security Protocols

Kafka combines authentication mechanisms with transport security:

Security Protocol	Authentication	Encryption	Use Case
PLAINTEXT	None	None	Development only
SSL	mTLS (optional)	TLS	Certificate auth or encryption only
SASL_PLAINTEXT	SASL	None	Internal networks (not recommended)
SASL_SSL	SASL	TLS	Production recommended

Never Use PLAINTEXT in Production

PLAINTEXT and SASL_PLAINTEXT transmit data unencrypted. Always use SSL or SASL_SSL in production.

Protocol Selection

# Encryption only (no authentication)
security.protocol=SSL

# SASL authentication with encryption (recommended)
security.protocol=SASL_SSL

# mTLS authentication with encryption
security.protocol=SSL
ssl.client.auth=required

---

Listener Configuration

Kafka supports different authentication mechanisms on different listeners, enabling separate configurations for internal and external traffic.

Multiple Listeners Example

# Define listeners
listeners=INTERNAL://0.0.0.0:9092,EXTERNAL://0.0.0.0:9093,REPLICATION://0.0.0.0:9094

# Map listener names to security protocols
listener.security.protocol.map=INTERNAL:SASL_PLAINTEXT,EXTERNAL:SASL_SSL,REPLICATION:SASL_SSL

# Inter-broker communication
inter.broker.listener.name=REPLICATION

# Different mechanisms per listener
listener.name.internal.sasl.enabled.mechanisms=PLAIN
listener.name.external.sasl.enabled.mechanisms=SCRAM-SHA-512,OAUTHBEARER
listener.name.replication.sasl.enabled.mechanisms=SCRAM-SHA-512

Listener Architecture

---

JAAS Configuration

Java Authentication and Authorization Service (JAAS) provides the authentication framework for SASL mechanisms.

Configuration Methods

Method	Scope	Use Case
Broker property	Per-listener, per-mechanism	Recommended for brokers
Static JAAS file	JVM-wide	Legacy, complex setups
Programmatic	Per-client	Application code

Broker Property (Recommended)

# Per-listener, per-mechanism JAAS config
listener.name.sasl_ssl.scram-sha-512.sasl.jaas.config=\
  org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="broker" \
  password="broker-secret";

Static JAAS File

kafka_server_jaas.conf

KafkaServer {
  org.apache.kafka.common.security.scram.ScramLoginModule required
  username="broker"
  password="broker-secret";
};

# JVM parameter
-Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf

Client Programmatic

props.put("sasl.jaas.config",
    "org.apache.kafka.common.security.scram.ScramLoginModule required " +
    "username=\"app\" password=\"app-secret\";");

See individual mechanism guides for detailed JAAS configuration.

---

Authentication Handshake

SASL authentication proceeds at the connection level once the transport is established. The client first negotiates a mechanism with a SaslHandshakeRequest, and the broker responds with the mechanisms it has enabled on that listener. The client and broker then exchange one or more SaslAuthenticateRequest and SaslAuthenticateResponse pairs that carry the mechanism-specific authentication bytes. For SCRAM, these bytes carry the client-first, server-first, client-final, and server-final messages.

The SaslAuthenticateRequest and SaslAuthenticateResponse envelope that wraps the SASL bytes applies to Kafka 1.0.0+ (KIP-152). In earlier versions the client negotiated the mechanism with SaslHandshakeRequest but then exchanged the raw SASL bytes directly on the socket, without the request and response envelope, so authentication failures surfaced only as a dropped connection rather than a structured error.

If the requested mechanism is not enabled on the listener, the broker returns a SaslHandshakeResponse that lists the mechanisms it does accept, and the client must not proceed with a SaslAuthenticateRequest for the unsupported mechanism. If a SaslAuthenticateRequest fails, the broker returns a SaslAuthenticateResponse carrying an error code and closes the connection; the client must treat this as a terminal failure and must not retry on the same connection. KIP-152 (Kafka 1.0.0+) added these structured error responses, which earlier versions could not report.

The client opens a TCP connection, completes the TLS handshake when the security protocol includes encryption, negotiates API versions, performs the SASL handshake and authentication, and only then reaches the ready state in which it can issue produce and fetch requests. The post-authentication ApiVersions round trip relates to KIP-511 (Kafka 2.4.0+), which lets the client report its software name and version to the broker once the connection is authenticated.

---

Monitoring Authentication

Brokers expose authentication counters and rates that indicate whether clients are authenticating successfully. A sustained drop in the success rate or any non-zero failure rate indicates misconfigured credentials, an unsupported mechanism, or an expired token or ticket.

Key metrics

Metric	Description	Alert condition
successful-authentication-rate	Successful authentications per second	Below baseline
failed-authentication-rate	Failed authentications per second	Greater than 0
successful-authentication-total	Total successful authentications	Reference counter
failed-authentication-total	Total failed authentications	Increasing
reauthentication-latency-avg	Average re-authentication time (requires Kafka 2.2.0+)	Greater than 1000ms

On the broker, these metrics are exposed per listener and per network processor under the kafka.server:type=socket-server-metrics,listener=<listenerName>,networkProcessor=<processor-id> MBean (KIP-188). They are not exposed under kafka.network:type=SocketServer, which carries unrelated metrics such as NetworkProcessorAvgIdlePercent, nor under BrokerTopicMetrics. Because the metric is reported per listener and per processor, no single cluster-wide counter exists; values must be read from, or aggregated across, each listener and processor instance. The same metric names also appear client-side, for outbound and inter-broker connections, under kafka.[producer|consumer|connect]:type=[producer|consumer|connect]-metrics,client-id=....

Monitoring commands

# Check authentication errors in broker logs
grep -i "authentication" /var/log/kafka/server.log

# Read the failed-authentication rate for one listener and processor over JMX.
# The metric name is an attribute of the MBean, not a key property of the
# object name, so it is selected with --attributes, not appended to --object-name.
kafka-run-class.sh kafka.tools.JmxTool \
    --jmx-url service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmi \
    --object-name 'kafka.server:type=socket-server-metrics,listener=SASL_SSL,networkProcessor=0' \
    --attributes failed-authentication-rate

---

Credential Management

The mechanism guides cover mechanism-specific credential handling, such as SCRAM credential rotation and secrets management. The storage and rotation practices below apply across SASL mechanisms.

Secure credential storage

Method	Security	Use case
Environment variables	Medium	Containers
Secrets manager	High	Production
Keytab files	High	Kerberos
Hardware security module	Highest	Financial

Credentials should not be hardcoded in configuration files. Reading them from the environment keeps them out of source control and image layers:

export KAFKA_SASL_USERNAME="alice"
export KAFKA_SASL_PASSWORD="alice-secret"

String jaasConfig = String.format(
    "org.apache.kafka.common.security.scram.ScramLoginModule required " +
    "username=\"%s\" password=\"%s\";",
    System.getenv("KAFKA_SASL_USERNAME"),
    System.getenv("KAFKA_SASL_PASSWORD")
);
props.put("sasl.jaas.config", jaasConfig);

Credential rotation

Rotation replaces credentials while existing sessions remain valid, then removes the old credentials once all clients have migrated. Authentication metrics confirm whether any client is still using the previous credentials before they are removed.

---

Version Compatibility

Feature	Kafka Version
SASL/PLAIN	0.9.0+
SASL/SCRAM	0.10.2+
SASL/GSSAPI	0.9.0+
SASL/OAUTHBEARER	2.0.0+
OAUTHBEARER OIDC support	3.1.0+
OAUTHBEARER login callback (secured package)	3.1.0+
OAUTHBEARER login callback (top-level package)	3.4.0+
Delegation tokens	1.1.0+
mTLS	0.9.0+
Re-authentication	2.2.0+
KRaft SCRAM bootstrap	3.5.0+

The OAUTHBEARER login-callback classes in the secured package (org.apache.kafka.common.security.oauthbearer.secured.*) were deprecated in Kafka 3.4.0 (KAFKA-13725) and removed in 4.0.0 (KAFKA-18293). Configurations must reference the top-level org.apache.kafka.common.security.oauthbearer package instead.

---

Common Configuration

Connection Timeouts

# SASL handshake timeout
sasl.login.connect.timeout.ms=10000

# SASL login retry
sasl.login.retry.backoff.ms=100
sasl.login.retry.backoff.max.ms=10000

Re-authentication

Enable periodic re-authentication for long-running connections:

# Broker: force re-authentication every hour
connections.max.reauth.ms=3600000

DNS Performance

Use FQDNs

SASL authentication performs reverse DNS lookups. Use fully qualified domain names in bootstrap.servers and advertised.listeners to avoid slow handshakes.

---

Troubleshooting

Common Issues

Issue	Symptom	Solution
Authentication failed	SaslAuthenticationException	Verify credentials, check JAAS config
Mechanism not enabled	UnsupportedSaslMechanismException	Add mechanism to sasl.enabled.mechanisms
SSL handshake failed	SSLHandshakeException	Verify truststore contains broker CA
Principal not found	Authorization failures	Check principal mapping rules
Slow connections	High connection latency	Use FQDNs, check DNS resolution

Debug Logging

# Enable SASL debug logging
log4j.logger.org.apache.kafka.common.security=DEBUG

# Enable SSL debug (JVM parameter)
-Djavax.net.debug=ssl:handshake

Verify Configuration

# Test SASL authentication
kafka-broker-api-versions.sh --bootstrap-server kafka:9093 \
  --command-config client.properties

# List SCRAM users
kafka-configs.sh --bootstrap-server kafka:9092 \
  --describe --entity-type users

---

Related Documentation

• SASL/SCRAM Authentication - Password-based authentication
• SASL/PLAIN Authentication - Simple username/password
• Kerberos Authentication - Enterprise SSO
• OAuth Authentication - Cloud identity providers
• mTLS Authentication - Certificate-based authentication
• Delegation Tokens - Lightweight tokens
• Authorization - ACL configuration
• Encryption - TLS setup
• Client connection architecture - connection lifecycle and the TLS/SASL layers this handshake fits into
• Kafka protocol - request/response framing underlying the handshake