Implementation Notes

These notes may be of use when trying to understand why things are implemented the way that they are, but should not be required reading for regular use.

OPA replica per Node

We run an OPA on each node, because we want to avoid requiring network round trips for services making policy queries (which are often chained in serial, and block other tasks in the products).

We ensure local access via an InternalTrafficPolicy. This means that Pods accessing OPA via the service discovery will be routed to the OPA Pod on the same Node to reduce request latency and network traffic. This feature is only activated per default in Kubernetes versions 1.22 or higher.

OPA Bundle Builder

Users can manage policy rules by creating, updating and deleting ConfigMap resources.

The responsibility of the OPA Bundle Builder is to convert these resources to bundles (tar.gz files) and make them available via an HTTP endpoint. The OPA Bundle Builder runs in a side container of the OPA Pod as a simple HTTP server that OPA is querying regularly (every 20 to 30 seconds) for updates.

Kubernetes limits the size of `ConfigMap`s to 1MB. Users have to take this limit into consideration when manging policy rules.

Only ConfigMaps labeled with "true" are considered by the builder when updating bundles. The name of the data entries in the ConfigMap are used as file names when storing the rules in the bundle.

Currently, it is the user’s responsibility to make sure these names do not collide (as they will override each other).