Configuration & environment overrides

The NiFi cluster definition supports overriding configuration properties, environment variables, and Pod attributes. The configuration overrides can be applied either per role, or per role group where the more specific override (role group) has precedence over the less specific one (role).

Do not override port numbers. This leads to cluster malfunction.

Configuration overrides

Apache NiFi runtime configuration is stored in several files listed below. The configOverrides block enables you to customize parameters in these files. The complete list of the configuration options can be found in the Apache NiFi documentation.

The following files can be edited directly via the configOverrides mechanism:

  • bootstrap.conf

  • nifi.properties

  • security.properties

Overrides are key-value pairs defined under one of the configuration files from the list above. They must match the property names as expected by NiFi. In the example below, a property nifi.flow.configuration.archive.enabled is being explicitly set to false, overriding the default value.

The following snippet shows how to disable workflow file backups in the NifiCluster definition:

  nodes:
    configOverrides:
      nifi.properties:
        nifi.flow.configuration.archive.enabled: false
Please be aware that by overriding config settings in this section you have a very high risk of breaking things, because the product does not behave the way the Stackable Operator for Apache NiFi expects it to behave anymore.

The security.properties file

The security.properties file is used to configure JVM security properties. It is very rare that you need to tweak any of these, but there is one use-case that stands out, and that users need to be aware of: the JVM DNS cache.

The JVM manages its own cache of successfully resolved host names as well as a cache of host names that cannot be resolved. Some products of the Stackable platform are very sensitive to the contents of these caches and their performance is heavily affected by them. As of version 1.21.0, Apache Nifi performs poorly if the positive cache is disabled. To cache resolved host names, and thus increase the performance your Nifi cluster, you can configure the TTL of entries in the positive cache like this:

  nodes:
    configOverrides:
      security.properties:
        networkaddress.cache.ttl: "30"
        networkaddress.cache.negative.ttl: "0"
The operator configures DNS caching by default as shown in the example above.

For details on the JVM security, consult the Java security overview.

Environment variables

Environment variables can be (over)written by adding the envOverrides property.

For example per role group:

nodes:
  roleGroups:
    default:
      config: {}
      replicas: 1
      envOverrides:
        MY_ENV_VAR: "MY_VALUE"

or per role:

nodes:
  envOverrides:
    MY_ENV_VAR: "MY_VALUE"
  roleGroups:
    default:
      config: {}
      replicas: 1

Pod overrides

Pod overrides allow you to configure any attributes that can be configured on a Pod, such as tolerations or labels on the Pod.

Read the Pod overrides concepts page to learn more.

Pod overrides on create-reporting-task Job

In addition to podOverrides on the created StatefulSet we also support podOverrides on the created Kubernetes Job, which enables the export of Prometheus metrics within NiFi.

spec:
  clusterConfig:
    createReportingTaskJob:
      # enabled: false # You can also turn off the Job entirely
      podOverrides: # podOverrides as usual
        spec:
          tolerations:
            - key: "key1"
              operator: "Equal"
              value: "value1"
              effect: "NoSchedule"