The Spark-Kubernetes RBAC documentation describes what is needed for
spark-submit jobs to run successfully: minimally a role/cluster-role to allow the driver pod to create and manage executor pods.
However, to add security, each
spark-submit job launched by the spark-k8s operator will be assigned its own service account.
When the spark-k8s operator is installed via Helm, a cluster role named
spark-k8s-clusterrole is created with pre-defined permissions.
When a new Spark application is submitted, the operator creates a new service account with the same name as the application and binds this account to the cluster role
spark-k8s-clusterrole created by Helm.