stackablectl demo
A demo is an end-to-end demonstration of the usage of the Stackable data platform. It is tied to a specific stack of the Stackable data platform, which will provide the required products for the demo.
General Usage
Interact with demos, which are end-to-end usage demonstrations of the Stackable data platform
Usage: stackablectl demo [OPTIONS] <COMMAND>
Commands:
list List available demos
describe Print out detailed demo information
install Install a specific demo
help Print this message or the help of the given subcommand(s)
Options:
-l, --log-level <LOG_LEVEL>
Log level this application uses
--no-cache
Do not cache the remote (default) demo, stack and release files
Cached files are saved at '$XDG_CACHE_HOME/stackablectl', which is usually
'$HOME/.cache/stackablectl' when not explicitly set.
--release <RELEASE>
Target a specific Stackable release
-h, --help
Print help (see a summary with '-h')
-V, --version
Print version
File options:
-d, --demo-file <DEMO_FILE>
Provide one or more additional (custom) demo file(s)
Demos are loaded in the following order: Remote (default) demo file, custom
demo files provided via the 'STACKABLE_DEMO_FILES' environment variable, and
lastly demo files provided via the '-d/--demo-file' argument(s). If there are
demos with the same name, the last demo definition will be used.
Use "stackablectl [OPTIONS] <COMMAND> -d path/to/demos1.yaml -d path/to/demos2.yaml"
to provide multiple additional demo files.
-s, --stack-file <STACK_FILE>
Provide one or more additional (custom) stack file(s)
Stacks are loaded in the following order: Remote (default) stack file, custom
stack files provided via the 'STACKABLE_STACK_FILES' environment variable, and
lastly demo files provided via the '-s/--stack-file' argument(s). If there are
stacks with the same name, the last stack definition will be used.
Use "stackablectl [OPTIONS] <COMMAND> -s path/to/stacks1.yaml -s path/to/stacks2.yaml"
to provide multiple additional stack files.
-r, --release-file <RELEASE_FILE>
Provide one or more additional (custom) release file(s)
Releases are loaded in the following order: Remote (default) release file,
custom release files provided via the 'STACKABLE_RELEASE_FILES' environment
variable, and lastly release files provided via the '-r/--release-file'
argument(s). If there are releases with the same name, the last release
definition will be used.
Use "stackablectl [OPTIONS] <COMMAND> -r path/to/releases1.yaml -r path/to/releases2.yaml"
to provide multiple additional release files.
Helm repository options:
--helm-repo-stable <URL>
Provide a custom Helm stable repository URL
[default: https://repo.stackable.tech/repository/helm-stable/]
--helm-repo-test <URL>
Provide a custom Helm test repository URL
[default: https://repo.stackable.tech/repository/helm-test/]
--helm-repo-dev <URL>
Provide a custom Helm dev repository URL
[default: https://repo.stackable.tech/repository/helm-dev/]
Browse Available Demos
To list the available demos, run the following command:
$ stackablectl demo list
┌────┬───────────────────────┬─────────┬─────────────────────────────────────────────────────────┐
│ # ┆ NAME ┆ STACK ┆ DESCRIPTION │
╞════╪═══════════════════════╪═════════╪═════════════════════════════════════════════════════════╡
│ 1 ┆ airflow-scheduled-job ┆ airflow ┆ Activate a simple Airflow DAG to run continuously at a │
│ ┆ ┆ ┆ set interval │
├╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 2 ┆ ... ┆ ... ┆ ... │
├╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 11 ┆ local-demo-test-env ┆ test ┆ Test demo. Provided with the STACKABLE_DEMO_FILES env │
│ ┆ ┆ ┆ var in the .env file │
└────┴───────────────────────┴─────────┴─────────────────────────────────────────────────────────┘
Detailed information of a demo can be queried using the describe
command:
$ stackablectl demo describe trino-taxi-data
DEMO trino-taxi-data
DESCRIPTION Demo loading 2.5 years of New York taxi data into S3 bucket, creating a Trino table and a Superset dashboard
DOCUMENTATION https://docs.stackable.tech/stackablectl/stable/demos/trino-taxi-data.html
STACK trino-superset-s3
LABELS trino, superset, minio, s3, ny-taxi-data
Installing a Demo
Using an Existing Kubernetes Cluster
If you want to access a Kubernetes cluster, make sure your kubectl
Kubernetes client is configured to interact with the Kubernetes cluster. After that, run the following command:
$ stackablectl demo install trino-taxi-data
Using a Local Kubernetes Cluster
If you don’t have a Kubernetes cluster available, stackablectl
can spin up a kind or
minikube Kubernetes cluster for you. Based on the type of local cluster you want to
use, ensure you have either kind
or minikube
installed on your system. stackablectl
will perform a check to verify
that these tools are available in your PATH
and check if Docker is running.
Local Kind Kubernetes Cluster
$ stackablectl demo install trino-taxi-data -c kind
Creating cluster "stackable-data-platform" ...
✓ Ensuring node image (kindest/node:v1.26.3) 🖼
✓ Preparing nodes 📦 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
✓ Joining worker nodes 🚜
Set kubectl context to "kind-stackable-data-platform"
You can now use your cluster with:
kubectl cluster-info --context kind-stackable-data-platform
Have a nice day! 👋
Installed demo trino-taxi-data
Use "stackablectl operator installed" to display the installed operators
Use "stackablectl stacklet list" to display the installed stacklets
Local Minikube Kubernetes Cluster
$ stackablectl demo install trino-taxi-data -c minikube
😄 [stackable-data-platform] minikube v1.30.1 on Ubuntu 22.04.2
✨ Using the docker driver based on user configuration
🎉 minikube 1.31.2 is available! Download it: https://github.com/kubernetes/minikube/releases/tag/v1.31.2
💡 To disable this notice, run: 'minikube config set WantUpdateNotification false'
📌 Using Docker driver with root privileges
👍 Starting control plane node stackable-data-platform in cluster stackable-data-platform
🚜 Pulling base image ...
🔥 Creating docker container (CPUs=2, Memory=8000MB) ...
🐳 Preparing Kubernetes v1.26.3 on Docker 23.0.2 ...
▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...
🔗 Configuring CNI (Container Networking Interface) ...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟 Enabled addons: storage-provisioner, default-storageclass
🔎 Verifying Kubernetes components...
👍 Starting worker node stackable-data-platform-m02 in cluster stackable-data-platform
🚜 Pulling base image ...
🔥 Creating docker container (CPUs=2, Memory=8000MB) ...
🌐 Found network options:
▪ NO_PROXY=192.168.58.2
🐳 Preparing Kubernetes v1.26.3 on Docker 23.0.2 ...
▪ env NO_PROXY=192.168.58.2
🔎 Verifying Kubernetes components...
🏄 Done! kubectl is now configured to use "stackable-data-platform" cluster and "default" namespace by default
Installed demo trino-taxi-data
Use "stackablectl operator installed" to display the installed operators
Use "stackablectl stacklet list" to display the installed stacklets
The demos create Kubernetes jobs that will populate test data and interact with the installed products to process the data. Until the products are ready, it is expected that the pods of these Jobs will fail with an error. They will get retried with an exponentially growing back-off time. After the products are ready, they should turn green, and everything should settle down.
Listing Deployed Stacklets
After installing your demo you can use the stackablectl stacklets
command to list the
installed stacklets as follows:
$ stackablectl stacklets list
┌──────────┬───────────────┬───────────┬─────────────────────────────────────────────┬────────────────────────────────────────────┐
│ PRODUCT ┆ NAME ┆ NAMESPACE ┆ ENDPOINTS ┆ CONDITIONS │
╞══════════╪═══════════════╪═══════════╪═════════════════════════════════════════════╪════════════════════════════════════════════╡
│ hive ┆ hive ┆ default ┆ ┆ Available, Reconciling, Running │
├╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ opa ┆ opa ┆ default ┆ ┆ Available, Reconciling, Running │
├╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ superset ┆ superset ┆ default ┆ external-superset http://192.168.58.2:30788 ┆ Available, Reconciling, Running │
├╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ trino ┆ trino ┆ default ┆ ┆ Unavailable: See [1], Reconciling, Running │
├╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ minio ┆ minio-console ┆ default ┆ ┆ │
└──────────┴───────────────┴───────────┴─────────────────────────────────────────────┴────────────────────────────────────────────┘
[1]: StatefulSet ["trino-coordinator-default", "trino-worker-default"] missing ready replicas.