Database connections

Airflow requires a metadata database for storing e.g. DAG, task and job data. The actual connection string is calculated by the operator so that the user does not need to remember the exact structure.

Typed connections

spec:
  clusterConfig:
    metadataDatabase:
      postgresql: (1)
        host: airflow-postgresql
        database: airflow
        credentialsSecretName: airflow-postgresql-credentials (2)
1 A reference to one of the supported database backends (e.g. postgresql).
2 A reference to a Secret which must contain the two fields username and password.

The queue/broker metadata and URL is only needed when running the celery executor. The resultBackend definition uses the same structure as metadataDatabase shown above. The broker definition requires Redis connection details.

spec:
  celeryExecutors:
    resultBackend:
      postgresql: (1)
        host: airflow-postgresql
        database: airflow
        credentialsSecretName: airflow-postgresql-credentials (2)
    broker:
      redis: (3)
        host: airflow-redis-master
        credentialsSecretName: airflow-redis-credentials (2)
1 A reference to one of the supported database backends (e.g. postgresql).
2 A reference to a secret which must contain the two fields username and password.
3 A reference to one of the supported queue brokers (e.g. redis).

Generic connections

Alternatively, these connections can also be defined in full in a referenced secret:

spec:
  clusterConfig:
    metadataDatabase:
      generic:
        connectionUrlSecretName: postgresql-metadata (1)
spec:
  resultBackend:
      generic:
        connectionUrlSecretName: postgresql-celery (2)
    broker:
      generic:
        connectionUrlSecretName: redis-celery (3)
1 A reference to a secret which must contain the single fields connectionUrl e.g. postgresql+psycopg2://airflow:airflow@airflow-postgresql/airflow
2 A reference to a secret which must contain the single fields connectionUrl e.g. db+postgresql://airflow:airflow@airflow-postgresql/airflow
3 A reference to a secret which must contain the single fields connectionUrl e.g. redis://:redis@airflow-redis-master:6379/0