First steps

After going through the Installation section and having installed all the operators, you will now deploy a Hive metastore cluster and it’s dependencies. Afterwards you can verify that it works.

Setup

Several requirements should have already been installed in the Installation guide:

S3Connection

In order to connect Hive to MinIO we need to create several files (or concat in one file).

An S3Connection to connect to MinIO (hive-minio-s3-connection.yaml)

---
apiVersion: s3.stackable.tech/v1alpha1
kind: S3Connection
metadata:
  name: minio
spec:
  host: minio
  port: 9000
  accessStyle: Path
  credentials:
    secretClass: hive-s3-secret-class

Credentials for the S3Connection to log into MinIO (hive-minio-credentials.yaml)

---
apiVersion: v1
kind: Secret
metadata:
  name: hive-s3-secret
  labels:
    secrets.stackable.tech/class: hive-s3-secret-class
stringData:
  accessKey: hive
  secretKey: hivehive

A SecretClass for the credentials to the Minio. The credentials were defined in the installation of Minio (hive-minio-credentials-secret-class.yaml)

---
apiVersion: secrets.stackable.tech/v1alpha1
kind: SecretClass
metadata:
  name: hive-s3-secret-class
spec:
  backend:
    k8sSearch:
      searchNamespace:
        pod: {}

And lastly the actual Apache Hive cluster definition (hive-postgres-s3.yaml). Note how it references the minio S3 object

---
apiVersion: hive.stackable.tech/v1alpha1
kind: HiveCluster
metadata:
  name: hive-postgres-s3
spec:
  image:
    productVersion: 3.1.3
    stackableVersion: 23.4.0
  clusterConfig:
    database:
      connString: jdbc:postgresql://postgresql:5432/hive
      user: hive
      password: hive
      dbType: postgres
    s3:
      reference: minio
  metastore:
    roleGroups:
      default:
        replicas: 1

Once the files are created apply them to the cluster:

kubectl apply -f hive-minio-credentials.yaml
kubectl apply -f hive-minio-credentials-secret-class.yaml
kubectl apply -f hive-minio-s3-connection.yaml
kubectl apply -f hive-postgres-s3.yaml

Verify that it works

Make sure that all the Pods in the StatefulSets are ready:

kubectl get statefulset

The output should show all pods ready:

NAME                                 READY   AGE
hive-postgres-s3-metastore-default   1/1     48s

For further testing we recommend to use e.g. the python hive-metastore-client in order to e.g. create tables or connect a product like Trino via the Stackable Trino Operator.

What’s next

Have a look at the Usage page to find out more about the features of the Operator.