Troubleshooting

General

My secret-consuming Pods get stuck Pending!

  1. Does the Pod have any events relating to scheduling? (kubectl describe pod/$POD_NAME)

  2. Is the PersistentVolumeClaim being created? It should have the name $POD_NAME-$VOLUME_NAME.

  3. Is the PersistentVolumeClaim bound to a PersistentVolume? If not:

    1. Does the PersistentVolumeClaim have any relevant events? (kubectl describe pvc/$PVC_NAME)

    2. If the PersistentVolumeClaim has no relevant events (or only an event that it is waiting for a volume to be provisioned), check the logs of the secret-operator sidecar container named external-provisioner.

    3. Does the PersistentVolumeClaim have a StorageClass set?

    4. Is the StorageClass configured to use the provisioner named secrets.stackable.tech?

  4. Does the PersistentVolume have any relevant events? (kubectl describe pv/$PV_NAME)

  5. Is the CSIDriver object named secrets.stackable.tech configured correctly?

  6. Is the CSI driver registered on the relevant CSINode objects? (kubectl get csinode/$NODE_NAME -o yaml should contain the driver secrets.stackable.tech)

  7. Does the secret-operator sidecar container named node-driver-registrar have any relevant log entries?

  8. Does the kubelet have any relevant log entries?

  9. When running on OpenShift also have a look at the OpenShift documentation.

Active Directory

LDAP user already exists

The Stackable Secret Operator maintains a cache of Active Directory user credentials. This error occurs when a required user is missing from the cache but exists in AD.

This can be caused by a few different root issues:

  1. A race condition where multiple Pods require the same identity at the same time, leading to a stale cache. This is transient, and should resolve itself within a few seconds as the cache is updated and any failed attempts are retried.

  2. Trying to reuse a single AD domain across multiple Kubernetes clusters. This takes care to do safely, please see Principal Conflicts for more information.

  3. Deleting a user from the cache but not from AD (including deleting the whole cache). When deleting users from the cache, make sure to also delete the corresponding AD users. The Secret Operator should then automatically recreate them.

  4. The to-be-created principal conlicts with an existing unrelated principal. This has to be resolved manually by an administrator.