Kafka Share Group Protocol APIs
This document specifies the Kafka protocol APIs used for Share Groups, a consumer model introduced as early access by KIP-932 in Kafka 4.0. Kafka 4.1 removed the early-access v0 coordination and data APIs and established v1 as the supported baseline. Share Groups provide queue-like semantics where messages are distributed among consumers without client-side partition assignment.
Share Group Overview
Section titled “Share Group Overview”Share Groups differ fundamentally from traditional Consumer Groups:
| Aspect | Consumer Groups | Share Groups |
|---|---|---|
| Partition assignment | Each partition assigned to one consumer | No partition assignment |
| Message delivery | All messages in partition to assigned consumer | Messages distributed across consumers |
| Ordering | Per-partition ordering guaranteed | No ordering guarantee |
| Acknowledgment | Offset-based | Per-record acknowledgment |
| Use case | Stream processing | Work queue, task distribution |
Share Group API Reference
Section titled “Share Group API Reference”Group Coordination
Section titled “Group Coordination”| API Key | Name | Purpose |
|---|---|---|
| 76 | ShareGroupHeartbeat | Maintain membership and receive assignments |
| 77 | ShareGroupDescribe | Describe share group state |
Data Operations
Section titled “Data Operations”| API Key | Name | Purpose |
|---|---|---|
| 78 | ShareFetch | Fetch records from share group |
| 79 | ShareAcknowledge | Acknowledge record processing |
State Management
Section titled “State Management”| API Key | Name | Purpose |
|---|---|---|
| 83 | InitializeShareGroupState | Initialize share partition state |
| 84 | ReadShareGroupState | Read share partition state |
| 85 | WriteShareGroupState | Write share partition state |
| 86 | DeleteShareGroupState | Delete share partition state |
| 87 | ReadShareGroupStateSummary | Read share partition state summary |
Offset Management
Section titled “Offset Management”| API Key | Name | Purpose |
|---|---|---|
| 90 | DescribeShareGroupOffsets | Describe share group offsets |
| 91 | AlterShareGroupOffsets | Alter share group offsets |
| 92 | DeleteShareGroupOffsets | Delete share group offsets |
Kafka 4.4 API Versions
Section titled “Kafka 4.4 API Versions”| API | Valid versions |
|---|---|
| ShareGroupHeartbeat, ShareGroupDescribe | 1 |
| ShareFetch, ShareAcknowledge | 1-2 |
| Initialize, Read, Delete ShareGroupState | 0 |
| WriteShareGroupState | 0-1 |
| ReadShareGroupStateSummary | 0-1 |
| DescribeShareGroupOffsets | 0-1 |
| AlterShareGroupOffsets, DeleteShareGroupOffsets | 0 |
Kafka trunk (4.5.0-SNAPSHOT)
KIP-1313 adds request-header v3 variants for the internal state APIs: Initialize v1, Read v1, Write v2, Delete v1, and ReadSummary v2. These versions are not available in Kafka 4.4.
ShareGroupHeartbeat API (Key 76)
Section titled “ShareGroupHeartbeat API (Key 76)”Overview
Section titled “Overview”The ShareGroupHeartbeat API maintains share group membership. Unlike consumer groups which use separate JoinGroup/SyncGroup/Heartbeat APIs, share groups use a single heartbeat API for all coordination.
Only v1 is supported in Kafka 4.4. The early-access v0 protocol was removed in Kafka 4.1.
Request Schema
Section titled “Request Schema”ShareGroupHeartbeatRequest => group_id: STRING member_id: STRING member_epoch: INT32 rack_id: NULLABLE_STRING subscribed_topic_names: [STRING]| Field | Type | Description |
|---|---|---|
group_id | STRING | Share group identifier |
member_id | STRING | Member ID (client-generated) |
member_epoch | INT32 | Member epoch (0=join, -1=leave) |
rack_id | NULLABLE_STRING | Rack ID (null if unchanged) |
subscribed_topic_names | ARRAY | Subscribed topic names (null if unchanged) |
Response Schema
Section titled “Response Schema”ShareGroupHeartbeatResponse => throttle_time_ms: INT32 error_code: INT16 error_message: NULLABLE_STRING member_id: NULLABLE_STRING member_epoch: INT32 heartbeat_interval_ms: INT32 assignment: Assignment
Assignment => topic_partitions: [TopicPartition]
TopicPartition => topic_id: UUID partitions: [INT32]Behavioral Contract
Section titled “Behavioral Contract”| Aspect | Guarantee |
|---|---|
| Epoch fencing | Stale members rejected via epoch |
| Assignment | Server-side assignment, no leader election |
| Heartbeat interval | Server specifies next heartbeat timing |
| Versioning | v0 early access was removed in 4.1; v1 is the only supported version through Kafka 4.4 |
ShareGroupDescribe API (Key 77)
Section titled “ShareGroupDescribe API (Key 77)”Overview
Section titled “Overview”The ShareGroupDescribe API retrieves detailed information about share groups.
Request Schema
Section titled “Request Schema”ShareGroupDescribeRequest => group_ids: [STRING] include_authorized_operations: BOOLEANResponse Schema
Section titled “Response Schema”ShareGroupDescribeResponse => throttle_time_ms: INT32 groups: [Group]
Group => error_code: INT16 error_message: NULLABLE_STRING group_id: STRING group_state: STRING group_epoch: INT32 assignment_epoch: INT32 assignor_name: STRING members: [Member] authorized_operations: INT32
Member => member_id: STRING rack_id: NULLABLE_STRING member_epoch: INT32 client_id: STRING client_host: STRING subscribed_topic_names: [STRING] assignment: Assignment
Assignment => topic_partitions: [TopicPartition]
TopicPartition => topic_id: UUID topic_name: STRING partitions: [INT32]ShareFetch API (Key 78)
Section titled “ShareFetch API (Key 78)”Overview
Section titled “Overview”The ShareFetch API retrieves records from a share group. Unlike regular Fetch, records are distributed across consumers and require acknowledgment.
Version 1 is the stable baseline. Version 2, added in Kafka 4.2, introduces share_acquire_mode and RENEW acknowledgements.
Request Schema
Section titled “Request Schema”ShareFetchRequest => group_id: NULLABLE_STRING member_id: NULLABLE_STRING share_session_epoch: INT32 max_wait_ms: INT32 min_bytes: INT32 max_bytes: INT32 max_records: INT32 batch_size: INT32 share_acquire_mode: INT8 is_renew_ack: BOOLEAN topics: [Topic] forgotten_topics_data: [ForgottenTopic]
Topic => topic_id: UUID partitions: [Partition]
Partition => partition_index: INT32 partition_max_bytes: INT32 acknowledgement_batches: [AcknowledgementBatch]
AcknowledgementBatch => first_offset: INT64 last_offset: INT64 acknowledge_types: [INT8]| Field | Type | Description |
|---|---|---|
group_id | STRING | Share group identifier |
share_session_epoch | INT32 | Session epoch for incremental fetch |
share_acquire_mode | INT8 | 0=batch-optimized, 1=record-limit. Added in Kafka 4.2 (KIP-1206, v2+) |
is_renew_ack | BOOLEAN | True if renew acks are present. Added in Kafka 4.2 (KIP-1222, v2+) |
acknowledgement_batches | ARRAY | Inline acknowledgments (optional) |
Response Schema
Section titled “Response Schema”ShareFetchResponse => throttle_time_ms: INT32 error_code: INT16 error_message: NULLABLE_STRING acquisition_lock_timeout_ms: INT32 responses: [Response] node_endpoints: [NodeEndpoint]
Response => topic_id: UUID partitions: [Partition]
Partition => partition_index: INT32 error_code: INT16 error_message: NULLABLE_STRING acknowledge_error_code: INT16 acknowledge_error_message: NULLABLE_STRING current_leader: LeaderIdAndEpoch records: RECORDS acquired_records: [AcquiredRecords]
AcquiredRecords => first_offset: INT64 last_offset: INT64 delivery_count: INT16| Field | Type | Description |
|---|---|---|
acquired_records | ARRAY | Records acquired by this consumer |
delivery_count | INT16 | Number of delivery attempts |
Behavioral Contract
Section titled “Behavioral Contract”| Aspect | Guarantee |
|---|---|
| Record locking | Fetched records locked to consumer |
| Delivery tracking | Delivery count tracks retries |
| Timeout | Unacknowledged records released after timeout |
ShareAcknowledge API (Key 79)
Section titled “ShareAcknowledge API (Key 79)”Overview
Section titled “Overview”The ShareAcknowledge API acknowledges processing of records fetched via ShareFetch.
Version 1 is the stable baseline. Version 2, added in Kafka 4.2, introduces RENEW acknowledgements.
Request Schema
Section titled “Request Schema”ShareAcknowledgeRequest => group_id: NULLABLE_STRING member_id: NULLABLE_STRING share_session_epoch: INT32 is_renew_ack: BOOLEAN topics: [Topic]
Topic => topic_id: UUID partitions: [Partition]
Partition => partition_index: INT32 acknowledgement_batches: [AcknowledgementBatch]
AcknowledgementBatch => first_offset: INT64 last_offset: INT64 acknowledge_types: [INT8]Acknowledge Types
Section titled “Acknowledge Types”| Value | Type | Description |
|---|---|---|
| 0 | GAP | Placeholder for gaps |
| 1 | ACCEPT | Successfully processed |
| 2 | RELEASE | Release for redelivery |
| 3 | REJECT | Reject permanently (DLQ) |
| 4 | RENEW | Renew acquisition lock timeout, extending processing time without releasing the record. Added in Kafka 4.2 (KIP-1222, v2+) |
Response Schema
Section titled “Response Schema”ShareAcknowledgeResponse => throttle_time_ms: INT32 error_code: INT16 error_message: NULLABLE_STRING acquisition_lock_timeout_ms: INT32 responses: [Response] node_endpoints: [NodeEndpoint]
Response => topic_id: UUID partitions: [Partition]
Partition => partition_index: INT32 error_code: INT16 error_message: NULLABLE_STRING current_leader: LeaderIdAndEpochBehavioral Contract
Section titled “Behavioral Contract”| Aspect | Guarantee |
|---|---|
| ACCEPT | Record marked complete, not redelivered |
| RELEASE | Record released for another consumer |
| REJECT | Record is permanently rejected. Kafka 4.4 can route it to a configured share-group DLQ when share.version=2 |
State Management APIs
Section titled “State Management APIs”InitializeShareGroupState API (Key 83)
Section titled “InitializeShareGroupState API (Key 83)”Initializes the share partition state when a share group first accesses a partition.
InitializeShareGroupStateRequest => group_id: STRING topics: [Topic]
Topic => topic_id: UUID partitions: [Partition]
Partition => partition: INT32 state_epoch: INT32 start_offset: INT64ReadShareGroupState API (Key 84)
Section titled “ReadShareGroupState API (Key 84)”Reads the current state of share partitions.
ReadShareGroupStateRequest => group_id: STRING topics: [Topic]
Topic => topic_id: UUID partitions: [Partition]
Partition => partition: INT32 leader_epoch: INT32WriteShareGroupState API (Key 85)
Section titled “WriteShareGroupState API (Key 85)”Writes updated share partition state.
WriteShareGroupStateRequest => group_id: STRING topics: [Topic]
Topic => topic_id: UUID partitions: [Partition]
Partition => partition: INT32 state_epoch: INT32 leader_epoch: INT32 start_offset: INT64 delivery_complete_count: INT32 state_batches: [StateBatch]
StateBatch => first_offset: INT64 last_offset: INT64 delivery_state: INT8 delivery_count: INT16DeleteShareGroupState API (Key 86)
Section titled “DeleteShareGroupState API (Key 86)”Deletes share partition state when cleaning up a share group.
DeleteShareGroupStateRequest => group_id: STRING topics: [Topic]
Topic => topic_id: UUID partitions: [INT32]ReadShareGroupStateSummary API (Key 87)
Section titled “ReadShareGroupStateSummary API (Key 87)”Reads a summary of share partition state for monitoring.
ReadShareGroupStateSummaryRequest => group_id: STRING topics: [Topic]
Topic => topic_id: UUID partitions: [Partition]
Partition => partition: INT32 leader_epoch: INT32Offset Management APIs
Section titled “Offset Management APIs”DescribeShareGroupOffsets API (Key 90)
Section titled “DescribeShareGroupOffsets API (Key 90)”Describes the current offsets for a share group.
DescribeShareGroupOffsetsRequest => groups: [Group]
Group => group_id: STRING topics: [Topic]
Topic => topic_name: STRING partitions: [INT32]AlterShareGroupOffsets API (Key 91)
Section titled “AlterShareGroupOffsets API (Key 91)”Alters the offsets for a share group (administrative operation).
AlterShareGroupOffsetsRequest => group_id: STRING topics: [Topic]
Topic => topic_name: STRING partitions: [Partition]
Partition => partition_index: INT32 start_offset: INT64DeleteShareGroupOffsets API (Key 92)
Section titled “DeleteShareGroupOffsets API (Key 92)”Deletes offset information for a share group.
DeleteShareGroupOffsetsRequest => group_id: STRING topics: [Topic]
Topic => topic_name: STRINGVersion Compatibility
Section titled “Version Compatibility”| Feature | Minimum Kafka Version |
|---|---|
| Share Groups early access (v0) | 4.0.0 |
| Stable ShareGroupHeartbeat, ShareGroupDescribe, ShareFetch, and ShareAcknowledge v1 | 4.1.0 |
ShareAcquireMode (batch_optimized, record_limit) | 4.2.0 (KIP-1206) |
| RENEW acknowledgement type | 4.2.0 (KIP-1222) |
| Share partition lag metrics | 4.2.0 (KIP-1226) |
| Additional share-group configurations | 4.3.0 (KAFKA-20037) |
| Assignment epochs for consumer/share groups | 4.3.0 (KIP-1251) |
| Share-group dead-letter queues | 4.4.0 (KIP-1191; share.version=2) |
| Offset initialization from a file | 4.4.0 (KIP-1323) |
Share Group API Versions
Share Groups were introduced in Kafka 4.0 (KIP-932). Kafka 4.2 adds v2 of the ShareFetch and ShareAcknowledge APIs with acquisition mode control (KIP-1206) and lock timeout renewal (KIP-1222). Kafka 4.3 extends group-level configuration coverage and introduces assignment epochs (KIP-1251). Kafka 4.4 adds dead-letter queues and file-based offset initialization.
Related Documentation
Section titled “Related Documentation”- Protocol Consumer APIs - Traditional consumer group APIs
- Protocol Core APIs - Core Produce/Fetch APIs
- Protocol Errors - Error code reference