stackablectl demo

A demo is an end-to-end demonstration of the usage of the Stackable data platform. It is tied to a specific stack of the Stackable data platform, which will provide the required products for the demo.

General Usage

Interact with demos, which are end-to-end usage demonstrations of the Stackable data platform

Usage: stackablectl demo [OPTIONS] <COMMAND>

Commands:
  list      List available demos
  describe  Print out detailed demo information
  install   Install a specific demo
  help      Print this message or the help of the given subcommand(s)

Options:
  -l, --log-level <LOG_LEVEL>
          Log level this application uses

      --no-cache
          Do not cache the remote (default) demo, stack and release files

          Cached files are saved at '$XDG_CACHE_HOME/stackablectl', which is usually
          '$HOME/.cache/stackablectl' when not explicitly set.

      --offline
          Do not request any remote files via the network

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version

File options:
  -d, --demo-file <DEMO_FILE>
          Provide one or more additional (custom) demo file(s)

          Demos are loaded in the following order: Remote (default) demo file, custom
          demo files provided via the 'STACKABLE_DEMO_FILES' environment variable, and
          lastly demo files provided via the '-d/--demo-file' argument(s). If there are
          demos with the same name, the last demo definition will be used.

          Use "stackablectl [OPTIONS] <COMMAND> -d path/to/demos1.yaml -d path/to/demos2.yaml"
          to provide multiple additional demo files.

  -s, --stack-file <STACK_FILE>
          Provide one or more additional (custom) stack file(s)

          Stacks are loaded in the following order: Remote (default) stack file, custom
          stack files provided via the 'STACKABLE_STACK_FILES' environment variable, and
          lastly demo files provided via the '-s/--stack-file' argument(s). If there are
          stacks with the same name, the last stack definition will be used.

          Use "stackablectl [OPTIONS] <COMMAND> -s path/to/stacks1.yaml -s path/to/stacks2.yaml"
          to provide multiple additional stack files.

  -r, --release-file <RELEASE_FILE>
          Provide one or more additional (custom) release file(s)

          Releases are loaded in the following order: Remote (default) release file,
          custom release files provided via the 'STACKABLE_RELEASE_FILES' environment
          variable, and lastly release files provided via the '-r/--release-file'
          argument(s). If there are releases with the same name, the last release
          definition will be used.

          Use "stackablectl [OPTIONS] <COMMAND> -r path/to/releases1.yaml -r path/to/releases2.yaml"
          to provide multiple additional release files.

Helm repository options:
      --helm-repo-stable <URL>
          Provide a custom Helm stable repository URL

          [default: https://repo.stackable.tech/repository/helm-stable/]

      --helm-repo-test <URL>
          Provide a custom Helm test repository URL

          [default: https://repo.stackable.tech/repository/helm-test/]

      --helm-repo-dev <URL>
          Provide a custom Helm dev repository URL

          [default: https://repo.stackable.tech/repository/helm-dev/]

Browse Available Demos

To list the available demos, run the following command:

$ stackablectl demo list
┌────┬───────────────────────┬─────────┬─────────────────────────────────────────────────────────┐
│ #  ┆ NAME                  ┆ STACK   ┆ DESCRIPTION                                             │
╞════╪═══════════════════════╪═════════╪═════════════════════════════════════════════════════════╡
│ 1  ┆ airflow-scheduled-job ┆ airflow ┆ Activate a simple Airflow DAG to run continuously at a  │
│    ┆                       ┆         ┆ set interval                                            │
├╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 2  ┆ ...                   ┆ ...     ┆ ...                                                     │
├╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 11 ┆ local-demo-test-env   ┆ test    ┆ Test demo. Provided with the STACKABLE_DEMO_FILES env   │
│    ┆                       ┆         ┆ var in the .env file                                    │
└────┴───────────────────────┴─────────┴─────────────────────────────────────────────────────────┘

Detailed information of a demo can be queried using the describe command:

$ stackablectl demo describe trino-taxi-data
 DEMO           trino-taxi-data
 DESCRIPTION    Demo loading 2.5 years of New York taxi data into S3 bucket, creating a Trino table and a Superset dashboard
 DOCUMENTATION  https://docs.stackable.tech/stackablectl/stable/demos/trino-taxi-data.html
 STACK          trino-superset-s3
 LABELS         trino, superset, minio, s3, ny-taxi-data

Installing a Demo

Using an Existing Kubernetes Cluster

If you want to access a Kubernetes cluster, make sure your kubectl Kubernetes client is configured to interact with the Kubernetes cluster. After that, run the following command:

$ stackablectl demo install trino-taxi-data

Using a Local Kubernetes Cluster

If you don’t have a Kubernetes cluster available, stackablectl can spin up a kind or minikube Kubernetes cluster for you. Based on the type of local cluster you want to use, ensure you have either kind or minikube installed on your system. stackablectl will perform a check to verify that these tools are available in your PATH and check if Docker is running.

Local Kind Kubernetes Cluster

$ stackablectl demo install trino-taxi-data -c kind
Creating cluster "stackable-data-platform" ...
 ✓ Ensuring node image (kindest/node:v1.26.3) 🖼
 ✓ Preparing nodes 📦 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹️
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
 ✓ Joining worker nodes 🚜
Set kubectl context to "kind-stackable-data-platform"
You can now use your cluster with:

kubectl cluster-info --context kind-stackable-data-platform

Have a nice day! 👋

Installed demo trino-taxi-data

Use "stackablectl operator installed" to display the installed operators
Use "stackablectl stacklet list" to display the installed stacklets

Local Minikube Kubernetes Cluster

$ stackablectl demo install trino-taxi-data -c minikube
😄  [stackable-data-platform] minikube v1.30.1 on Ubuntu 22.04.2
✨  Using the docker driver based on user configuration
🎉  minikube 1.31.2 is available! Download it: https://github.com/kubernetes/minikube/releases/tag/v1.31.2
💡  To disable this notice, run: 'minikube config set WantUpdateNotification false'

📌  Using Docker driver with root privileges
👍  Starting control plane node stackable-data-platform in cluster stackable-data-platform
🚜  Pulling base image ...
🔥  Creating docker container (CPUs=2, Memory=8000MB) ...
🐳  Preparing Kubernetes v1.26.3 on Docker 23.0.2 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔗  Configuring CNI (Container Networking Interface) ...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🔎  Verifying Kubernetes components...

👍  Starting worker node stackable-data-platform-m02 in cluster stackable-data-platform
🚜  Pulling base image ...
🔥  Creating docker container (CPUs=2, Memory=8000MB) ...
🌐  Found network options:
    ▪ NO_PROXY=192.168.58.2
🐳  Preparing Kubernetes v1.26.3 on Docker 23.0.2 ...
    ▪ env NO_PROXY=192.168.58.2
🔎  Verifying Kubernetes components...
🏄  Done! kubectl is now configured to use "stackable-data-platform" cluster and "default" namespace by default
Installed demo trino-taxi-data

Use "stackablectl operator installed" to display the installed operators
Use "stackablectl stacklet list" to display the installed stacklets

The demos create Kubernetes jobs that will populate test data and interact with the installed products to process the data. Until the products are ready, it is expected that the pods of these Jobs will fail with an error. They will get retried with an exponentially growing back-off time. After the products are ready, they should turn green, and everything should settle down.

Listing Deployed Stacklets

After installing your demo you can use the stackablectl stacklets command to list the installed stacklets as follows:

$ stackablectl stacklets list
┌──────────┬───────────────┬───────────┬─────────────────────────────────────────────┬────────────────────────────────────────────┐
│ PRODUCT  ┆ NAME          ┆ NAMESPACE ┆ ENDPOINTS                                   ┆ CONDITIONS                                 │
╞══════════╪═══════════════╪═══════════╪═════════════════════════════════════════════╪════════════════════════════════════════════╡
│ hive     ┆ hive          ┆ default   ┆                                             ┆ Available, Reconciling, Running            │
├╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ opa      ┆ opa           ┆ default   ┆                                             ┆ Available, Reconciling, Running            │
├╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ superset ┆ superset      ┆ default   ┆ external-superset http://192.168.58.2:30788 ┆ Available, Reconciling, Running            │
├╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ trino    ┆ trino         ┆ default   ┆                                             ┆ Unavailable: See [1], Reconciling, Running │
├╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ minio    ┆ minio-console ┆ default   ┆                                             ┆                                            │
└──────────┴───────────────┴───────────┴─────────────────────────────────────────────┴────────────────────────────────────────────┘

[1]: StatefulSet ["trino-coordinator-default", "trino-worker-default"] missing ready replicas.

Uninstalling a Demo

Currently, there is no support for uninstalling a demo again. However, this functionality will come soon.