Resource management

The Stackable data platform and its operators deploy their products in Containers within Pods using StatefulSets or DaemonSets. In order for the Kubernetes scheduler to select a proper Node for a Pod, resource requests and limits for CPU and memory can be specified. The Kubernetes scheduler ensures that the sum of the CPU and memory requests does not exceed the capacity of a given Node.

Terminology

The most commonly defined resources are CPU and memory (RAM). Keep in mind that there are other resources as well. For more information have a look at the Kubernetes documentation on resources.

CPU

Requests

The CPU request defines which containers are allowed to have more CPU resources. Larger requests lead to more CPU resources than smaller requests if not enough resources are available.

Limits

The CPU limit is a hard bound on how much CPU resources the Container can use. The Linux Kernel checks to see if the limit is exceeded and waits if this is the case to resume the process.

Memory

Requests

The memory request is used during Kubernetes scheduling and checks which Nodes offer sufficient memory for a Pod to be scheduled.

Limits

The memory limit is a hard bound. If a Container tries to use more memory than specified, the Container is usually marked for restart. To avoid the restart it is critical to specify sufficient resources.

Storage

Some Stackable products require data storage. This is done using Persistent Volume Claims where the size of storage can be specified.

Kubernetes Resource Requests

In Kubernetes, you can specify resource requests and limits within Containers of Pods.

---
apiVersion: v1
kind: Pod
metadata:
  name: stackable
spec:
  containers:
  - name: stackable
    image: stackable:0.1.0
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"

This Pod/Container will be scheduled on a Node that has at least 64Mi of free memory. It is allowed to use a maximum amount of 128Mi of memory or will be restarted if this value is exceeded. It can not use more CPU resources than 500m (which is half of a physical or virtual core), but has a guaranteed CPU resources of 250m.

Stackable Resource Requests

CPU and Memory

Stackable operators handle resource requests in a sligtly different manner than Kubernetes. Resource requests are defined on role or group level. See Roles and role groups for details on these concepts. On a role level this means that e.g. all workers will use the same resource requests and limits. This can be further specified on role group level (which takes priority to the role level) to apply different resources.

This is an example on how to specify CPU and memory resources using the Stackable Custom Resources:

---
apiVersion: example.stackable.tech/v1alpha1
kind: ExampleCluster
metadata:
  name: example
spec:
  workers: # role-level
    config:
      resources:
        cpu:
          min: 300m
          max: 600m
        memory:
          limit: 3Gi
    roleGroups: # role-group-level
      resources-from-role: # role-group 1
        replicas: 1
      resources-from-role-group: # role-group 2
        replicas: 1
        config:
          resources:
            cpu:
              min: 400m
              max: 800m
            memory:
              limit: 4Gi

In this case, the role group resources-from-role will inherit the resources specified on the role level. Resulting in a maximum of 3Gi memory and 600m CPU resources.

The role group resources-from-role-group has maximum of 4Gi memory and 800m CPU resources (which overrides the role CPU resources).

For Java products the actual used Heap memory is lower than the specified memory limit due to other processes in the Container requiring memory to run as well. Currently, 80% of the specified memory limits is passed to the JVM.

For memory only a limit can be specified, which will be set as memory request and limit in the Container. This is to always guarantee a Container the full amount memory during Kubernetes scheduling.

Storage

This is an example on how to specify storage resources using the Stackable Custom Resources:

---
apiVersion: example.stackable.tech/v1alpha1
kind: ExampleCluster
metadata:
  name: example
spec:
  workers: # role-level
    config:
      resources:
        storage:
          data: # name of the storage
            capacity: 3Gi
    roleGroups: # role-group-level
      resources-from-role: # role-group 1
        replicas: 1
      resources-from-role-group: # role-group 2
        replicas: 1
        config:
          resources:
            storage:
              data: # name of the storage
                capacity: 4Gi

In this case, the role group resources-from-role will inherit the resources specified on the role level. Resulting in a 3Gi storage space for data.

The role group resources-from-role-group has maximum of 4Gi storage space for data (which overrides the role resources).

Stackable operators use different names (data in this example) for their storage specification. This is documented in the operator specific documentation.