Exposing NiFi processor ports

Some NiFi processors allow external tools to connect to them to trigger workflows or send data. For this to work from outside the Kubernetes cluster NiFi has been deployed to, it is necessary to create Service or Ingress objects to configure Kubernetes routes for these ports.

Stackable doesn’t place any restriction on the type of service that can be used here.

Example

An example for this is shown below. It consists of a NiFi cluster with three nodes called simple-nifi and has a processor that listens to a TCP port and forwards lines written into that port as flowfiles.

A ListenTCP processor

The processor has been configured to listen on port 8123, so this port is available on all NiFi Pods for processes that run inside the Kubernetes cluster.

In order to make this port available to external processes as well, a LoadBalancer type service can be used for example:

---
apiVersion: v1
kind: Service
metadata:
  name: nifi-lb
spec:
  type: LoadBalancer
  externalTrafficPolicy: Cluster
  selector:
    app.kubernetes.io/instance: simple-nifi
  ports:
    - name: tcp-port
      protocol: TCP
      port: 8080
      targetPort: 8123

Depending on the Kubernetes configuration this snippet requests an external IP address and opens port 8080 on that IP address. Any requests to this port are forwarded to port 8123 on the NiFi pods — and thus the processor.

If LoadBalancer Services or Ingress resources are not available for use, a NodePort Service would be an alternative:

---
apiVersion: v1
kind: Service
metadata:
  name: nifi-lb
spec:
  type: NodePort
  selector:
    app.kubernetes.io/instance: simple-nifi
  ports:
    - port: 8123
      protocol: TCP
      targetPort: 8123
      nodePort: 30007

This doesn’t require any external addresses and instead opens port 30007 on all Kubernetes nodes. Any requests to these ports are then forwarded to the NiFi processor.