ADR017: TLS authentication

Status: accepted
Deciders:
- Malte Sander
- Sebastian Bernauer
- Sönke Liebau
- Natalie Klestrup Röijezon
Date: 2022-05-04

Technical Story: https://github.com/stackabletech/zookeeper-operator/issues/466

Context and Problem Statement

Our products use TLS to encrypt network traffic and/or to authenticate themselves and their clients. We want to define a common way - across all the products - to configure these mechanisms.

Decision Drivers

Must support using no TLS usage, TLS only for encryption and TLS for encryption and authentication
Should fit into out existing authentication concept from ADR014: User Authentication for Products
Should be easy to understand and use for the user

Considered Options

Handle TLS as special case
Extend AuthenticationClass to support TLS
Split server-side and client-side authentication into separate config sections

Decision Outcome

Chosen option: "Split server-side and client-side authentication into separate config sections", because separating them allowed us to model the underlying structs the best way.

Pros and Cons of the Options

Handle TLS as special case

We don’t support TLS inside AuthenticationClass. We need a new tlsConfig attribute on every product CRD. An implementation could look something like this:

---
apiVersion: zookeeper.stackable.tech/v1alpha1
kind: ZookeeperCluster
metadata:
  name: ssl-zk
spec:
  config:
    tlsConfiguration: # optional
      quorum: # Communication between between different Zookeeper nodes
        tls: # commons tls config
          verification:
              none: {}
              # OR
              server:
                caCert:
                    webPki: {}
                    # OR
                    secretClass: tls # Reference to SecretClass below
              # OR
              mutual:
                certSecretClass: tls # Reference to SecretClass below
      clients: # Communication between Zookeeper clients and Zookeeper nodes
        tls: # commons tls config
          verification:
              none: {}
              # OR
              server:
                caCert:
                    webPki: {}
                    # OR
                    secretClass: tls # Reference to SecretClass below
              # OR
              mutual:
                certSecretClass: tls # Reference to SecretClass below
  servers:
    roleGroups:
      primary:
        replicas: 2
        config:
          myidOffset: 10
      secondary:
        replicas: 1
        config:
          myidOffset: 20
  version: 3.8.0
  stopped: false
---
apiVersion: secrets.stackable.tech/v1alpha1
kind: SecretClass
metadata:
  name: tls
spec:
  backend:
    autoTls:
      ca:
        secret:
          name: secret-provisioner-tls-ca
          namespace: default
        autoGenerate: true

Good, because simple (no indirection through AuthenticationClass object)
Bad, because TLS is a special case and does not fit in our AuthenticationClass mechanism.

Extend `AuthenticationClass` to support TLS

We already have a common AuthenticationClass structure discussed in ADR014: User Authentication for Products This option extends the AuthenticationClass so that it also support TLS authentication.

---
apiVersion: zookeeper.stackable.tech/v1alpha1
kind: ZookeeperCluster
metadata:
  name: ssl-zk
spec:
  config:
    tlsAuthenticationConfig:
      quorum:
        authenticationClass: zookeeper-tls-mutual
      clients:
        authenticationClass: zookeeper-tls
  servers:
    roleGroups:
      primary:
        replicas: 2
        config:
          myidOffset: 10
      secondary:
        replicas: 1
        config:
          myidOffset: 20
  version: 3.8.0
  stopped: false
---
apiVersion: authentication.stackable.tech/v1alpha1
kind: AuthenticationClass
metadata:
  name: zookeeper-tls-mutual
spec:
  provider:
    tls:
      verification:
          none: {}
          # OR
          server:
            caCert:
                webPki: {}
                # OR
                secretClass: tls # Reference to SecretClass below
          # OR
          mutual:
          certSecretClass: tls # Reference to SecretClass below
---
apiVersion: authentication.stackable.tech/v1alpha1
kind: AuthenticationClass
metadata:
  name: zookeeper-tls
spec:
  provider:
    tls:
      verification:
          none: {}
          # OR
          server:
            caCert:
                webPki: {}
                # OR
                secretClass: tls # Reference to SecretClass below
          # OR
          mutual:
          certSecretClass: tls # Reference to SecretClass below
      verification:
        mutual:
          caCert:
            secretClass: tls # Reference to SecretClass below
---
apiVersion: secrets.stackable.tech/v1alpha1
kind: SecretClass
metadata:
  name: tls
spec:
  backend:
    autoTls:
      ca:
        secret:
          name: secret-provisioner-tls-ca
          namespace: default
        autoGenerate: true

Good, because TLS is handled via the generic AuthenticationClass mechanism.
Bad, because an AuthenticationClass can express: Don’t do any authentication at all

Split server-side and client-side authentication into separate config sections

---
apiVersion: zookeeper.stackable.tech/v1alpha1
kind: ZookeeperCluster
metadata:
  name: ssl-zk
spec:
  config:
    # Only affects client connections
    # This setting controls
    # - If TLS encryption is used at all
    # - Which cert the servers should use to authenticate themselves against the client
    tls: # optional, defaults to "secretClass: tls"
      secretClass: tls # provides tls.crt, tls.key and ca.crt

    # Only affects client connections
    # This setting controls
    # - If clients need to authenticate themselves against the server via TLS
    # - Which ca.crt to use when validating the provided client certs
    clientAuthentication: # optional. Can only be provided if config.tls is provided
      authenticationClass: zookeeper-tls # provides ca.crt

    # Only affects quorum communication
    # Use mutual verification between Zookeeper Nodes (mandatory)
    # This setting controls
    # - Which cert the servers should use to authenticate themselves against other servers
    # - Which ca.crt to use when validating the other server
    quorumTlsSecretClass: tls # provides tls.crt, tls.key and ca.crt
  servers:
    roleGroups:
      primary:
        replicas: 2
        config:
          myidOffset: 10
      secondary:
        replicas: 1
        config:
          myidOffset: 20
  version: 3.8.0
  stopped: false
---
apiVersion: authentication.stackable.tech/v1alpha1
kind: AuthenticationClass
metadata:
  name: zookeeper-tls
spec:
  provider:
    # Expresses: Authenticate clients!
    # For this to work you must have an TLS-enabled endpoint which in turn requires a ca.crt (at least at Stackable ;))
    # So you already have a ca.crt, use that to validate the client certs!
    tls: {}
    # OR...
    tls:
      # Expresses: Authenticate clients using the following ca.crt!
      # This setting controls
      # - Which ca.crt to use when validating the provided client certs
      clientCertSecretClass: tls-client  # provides ca.crt
    # OR...
    # All the other authentication mechanisms out there still exist
    ldap: ldapStuff{...}
---
apiVersion: secrets.stackable.tech/v1alpha1
kind: SecretClass
metadata:
  name: tls
spec:
  backend:
    autoTls:
      ca:
        secret:
          name: secret-provisioner-tls-ca
          namespace: default
        autoGenerate: true

As a recap: This is how our products connect to TLS-secured services outside of the Stackable Cluster (e.g. Superset to LDAP). Mutual verification is not supported this way, as client-side authentication is handled via AuthenticationClass.

---
[ProductCRD...]
  tls:
    verification:
      none: {}
      # OR
      server:
        caCert:
          webPki: {}
          # OR
          secretClass: tls # Reference to a SecretClass providing the ca.crt

Good, because TLS is handled via the generic AuthenticationClass mechanism.
Good, because clients don’t need to know/understand AuthenticationClass objects. They only read the Discovery ConfigMap and the contained SecretClasses.
Bad, because it’s more complicated because of the indirection via AuthenticationClass.
Bad, because the client operator needs to read the Discovery ConfigMap rather than simply mounting it into the client product.
Bad, because doing so external clients (outside of Stackable) must need to take more effort to connect to Stackable services e.g. retrieve ca cert from SecretClass.