Pod placement

You can configure Pod placement for HDFS nodes as described in Pod placement.

Default Pod placement constraints for name nodes:

affinity:
  podAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - podAffinityTerm:
        labelSelector:
          matchLabels:
            app.kubernetes.io/instance: cluster-name
            app.kubernetes.io/name: hdfs
        topologyKey: kubernetes.io/hostname
      weight: 20
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - podAffinityTerm:
        labelSelector:
          matchLabels:
            app.kubernetes.io/component: namenode
            app.kubernetes.io/instance: hdfs
            app.kubernetes.io/name: hdfs
        topologyKey: kubernetes.io/hostname
      weight: 70

If OPA authorization is configured, the name nodes additionally prefer to run on the same Kubernetes nodes as the OPA Pods (weight 50). Only the name nodes are configured with the OPA authorizer, so data nodes and journal nodes do not get this affinity.

affinity:
  podAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - podAffinityTerm:
        labelSelector:
          matchLabels:
            app.kubernetes.io/component: server
            app.kubernetes.io/instance: opa-cluster-name
            app.kubernetes.io/name: opa
        topologyKey: kubernetes.io/hostname
      weight: 50

opa-cluster-name is the configMapName from spec.clusterConfig.authorization.opa, which by convention is the name of the OpaCluster.

Default Pod placement constraints for data nodes:

affinity:
  podAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - podAffinityTerm:
        labelSelector:
          matchLabels:
            app.kubernetes.io/instance: cluster-name
            app.kubernetes.io/name: hdfs
        topologyKey: kubernetes.io/hostname
      weight: 20
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - podAffinityTerm:
        labelSelector:
          matchLabels:
            app.kubernetes.io/component: datanode
            app.kubernetes.io/instance: cluster-name
            app.kubernetes.io/name: hdfs
        topologyKey: kubernetes.io/hostname
      weight: 70

Default Pod placement constraints for journal nodes:

affinity:
  podAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - podAffinityTerm:
        labelSelector:
          matchLabels:
            app.kubernetes.io/instance: cluster-name
            app.kubernetes.io/name: hdfs
        topologyKey: kubernetes.io/hostname
      weight: 20
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - podAffinityTerm:
        labelSelector:
          matchLabels:
            app.kubernetes.io/component: journalnode
            app.kubernetes.io/instance: cluster-name
            app.kubernetes.io/name: hdfs
        topologyKey: kubernetes.io/hostname
      weight: 70

In the examples above cluster-name is the name of the HDFS custom resource that owns this Pod.