By default, a NiFi cluster will create five different persistent volume claims for flow files, provenance, database, content and state folders. The example below shows how to configure the sizes of the individual volumes. The volume sizes are the defaults used by the Operator. It is recommended that you configure these volume requests according to your needs.
Storage requests can be configured at role or group level, for one or more of the persistent volumes as follows:
Stackable operators handle resource requests in a sligtly different manner than Kubernetes. Resource requests are defined on role or group level. See Roles and role groups for details on these concepts. On a role level this means that e.g. all workers will use the same resource requests and limits. This can be further specified on role group level (which takes priority to the role level) to apply different resources.
This is an example on how to specify CPU and memory resources using the Stackable Custom Resources:
workers: # role-level
roleGroups: # role-group-level
resources-from-role: # role-group 1
resources-from-role-group: # role-group 2
In this case, the role group
resources-from-role will inherit the resources specified on the role level. Resulting in a maximum of
3Gi memory and
600m CPU resources.
The role group
resources-from-role-group has maximum of
4Gi memory and
800m CPU resources (which overrides the role CPU resources).
|For Java products the actual used Heap memory is lower than the specified memory limit due to other processes in the Container requiring memory to run as well. Currently, 80% of the specified memory limits is passed to the JVM.
For memory only a limit can be specified, which will be set as memory request and limit in the Container. This is to always guarantee a Container the full amount memory during Kubernetes scheduling.
A minimal HA setup consisting of 2 NiFi instances has the following resource requirements:
8448Mimemory request and limit