KRaft mode (experimental)
| The Kafka KRaft mode is currently experimental, and subject to change. |
Apache Kafka’s KRaft mode replaces Apache ZooKeeper with Kafka’s own built-in consensus mechanism based on the Raft protocol. This simplifies Kafka’s architecture, reducing operational complexity by consolidating cluster metadata management into Kafka itself.
| The Stackable Operator for Apache Kafka currently does not support automatic cluster upgrades from Apache ZooKeeper to KRaft. |
Overview
-
Introduced: Kafka 2.8.0 (early preview, not production-ready).
-
Matured: Kafka 3.3.x (production-ready, though ZooKeeper is still supported).
-
Default & Recommended: Kafka 3.5+ strongly recommends KRaft for new clusters.
-
Full Replacement: Kafka 4.0.0 (2025) removes ZooKeeper completely.
-
Migration: Tools exist to migrate from ZooKeeper to KRaft, but new deployments should start with KRaft.
Configuration
The Stackable Kafka operator introduces a new role in the KafkaCluster CRD called KRaft Controller.
Configuring the Controller will put Kafka into KRaft mode. Apache ZooKeeper will not be required anymore.
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
name: kafka
spec:
image:
productVersion: "3.9.1"
brokers:
roleGroups:
default:
replicas: 1
controllers:
roleGroups:
default:
replicas: 3
Using spec.controllers is mutually exclusive with spec.clusterConfig.zookeeperConfigMapName.
|
Recommendations
A minimal KRaft setup consisting of at least 3 Controllers has the following resource requirements:
-
600mCPU request -
3000mCPU limit -
3000Mimemory request and limit -
6Gipersistent storage
| The Controller replicas should sum up to an odd number for the Raft consensus. |
Resources
Corresponding to the values above, the operator uses the following resource defaults:
controllers:
config:
resources:
memory:
limit: 1Gi
cpu:
min: 250m
max: 1000m
storage:
logDirs:
capacity: 2Gi
Overrides
The configuration of overrides, JVM arguments etc. is similar to the Broker and documented on the concepts page.
Internal operator details
KRaft mode requires major configuration changes compared to ZooKeeper:
-
cluster-id: This is set to themetadata.nameof the KafkaCluster resource during initial formatting -
node.id: This is a calculated integer, hashed from theroleandrolegroupand addedreplicaid. -
process.roles: Will always only bebrokerorcontroller. Mixedbroker,controllerservers are not supported. -
The operator configures a static voter list containing the controller pods. Controllers are not dynamicaly managed.
Known Issues
-
Automatic migration from Apache ZooKeeper to KRaft is not supported.
-
Scaling controller replicas might lead to unstable clusters.
-
Kerberos is currently not supported for KRaft in all versions.
Troubleshooting
Frequent leader elections
Likely caused by controller resource starvation or unstable Kubernetes scheduling.
Migration issues (ZooKeeper to KRaft)
Ensure Kafka version 3.9.x and higher and follow the official migration documentation. The Stackable Kafka operator currently does not support the migration.
Scaling issues
The Dynamic scaling is only supported from Kafka version 3.9.0. If you are using older versions, automatic scaling may not work properly (e.g. adding or removing controller replicas).