KRaft mode (experimental)

The Kafka KRaft mode is currently experimental, and subject to change.

Apache Kafka’s KRaft mode replaces Apache ZooKeeper with Kafka’s own built-in consensus mechanism based on the Raft protocol. This simplifies Kafka’s architecture, reducing operational complexity by consolidating cluster metadata management into Kafka itself.

The Stackable Operator for Apache Kafka currently does not support automatic cluster upgrades from Apache ZooKeeper to KRaft.

Overview

  • Introduced: Kafka 2.8.0 (early preview, not production-ready).

  • Matured: Kafka 3.3.x (production-ready, though ZooKeeper is still supported).

  • Default & Recommended: Kafka 3.5+ strongly recommends KRaft for new clusters.

  • Full Replacement: Kafka 4.0.0 (2025) removes ZooKeeper completely.

  • Migration: Tools exist to migrate from ZooKeeper to KRaft, but new deployments should start with KRaft.

Configuration

The Stackable Kafka operator introduces a new role in the KafkaCluster CRD called KRaft Controller. Configuring the Controller will put Kafka into KRaft mode. Apache ZooKeeper will not be required anymore.

apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
  name: kafka
spec:
  image:
    productVersion: "3.9.1"
  brokers:
    roleGroups:
      default:
        replicas: 1
  controllers:
    roleGroups:
      default:
        replicas: 3
Using spec.controllers is mutually exclusive with spec.clusterConfig.zookeeperConfigMapName.

Recommendations

A minimal KRaft setup consisting of at least 3 Controllers has the following resource requirements:

  • 600m CPU request

  • 3000m CPU limit

  • 3000Mi memory request and limit

  • 6Gi persistent storage

The Controller replicas should sum up to an odd number for the Raft consensus.

Resources

Corresponding to the values above, the operator uses the following resource defaults:

controllers:
  config:
    resources:
      memory:
        limit: 1Gi
      cpu:
        min: 250m
        max: 1000m
      storage:
        logDirs:
          capacity: 2Gi

Overrides

The configuration of overrides, JVM arguments etc. is similar to the Broker and documented on the concepts page.

Internal operator details

KRaft mode requires major configuration changes compared to ZooKeeper:

  • cluster-id: This is set to the metadata.name of the KafkaCluster resource during initial formatting

  • node.id: This is a calculated integer, hashed from the role and rolegroup and added replica id.

  • process.roles: Will always only be broker or controller. Mixed broker,controller servers are not supported.

  • The operator configures a static voter list containing the controller pods. Controllers are not dynamicaly managed.

Known Issues

  • Automatic migration from Apache ZooKeeper to KRaft is not supported.

  • Scaling controller replicas might lead to unstable clusters.

  • Kerberos is currently not supported for KRaft in all versions.

Troubleshooting

Cluster does not start

Check that at least a quorum (majority) of controllers are reachable.

Frequent leader elections

Likely caused by controller resource starvation or unstable Kubernetes scheduling.

Migration issues (ZooKeeper to KRaft)

Ensure Kafka version 3.9.x and higher and follow the official migration documentation. The Stackable Kafka operator currently does not support the migration.

Scaling issues

The Dynamic scaling is only supported from Kafka version 3.9.0. If you are using older versions, automatic scaling may not work properly (e.g. adding or removing controller replicas).