stackablectl operator

The stackable operator command allows the listing, installation and removal of Stackable operators. Operators manage the individual data products of the Stackable Data Platform.

This command manages individual operators. It is intended for people with experience working on the Stackable Data Platform. If you want an easy way to get started or don’t know which products and versions to install, using the stackablectl release command is recommended. This command will install a bundle of operators from an official Stackable release.

General Usage

Interact with single operator instead of the full platform

Usage: stackablectl operator [OPTIONS] <COMMAND>

Commands:
  list       List available operators
  describe   Print out detailed operator information
  install    Install one or more operators
  uninstall  Uninstall one or more operators
  installed  List installed operators
  help       Print this message or the help of the given subcommand(s)

Options:
  -l, --log-level <LOG_LEVEL>
          Log level this application uses

      --no-cache
          Do not cache the remote (default) demo, stack and release files

          Cached files are saved at '$XDG_CACHE_HOME/stackablectl', which is usually
          '$HOME/.cache/stackablectl' when not explicitly set.

      --offline
          Do not request any remote files via the network

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version

File options:
  -d, --demo-file <DEMO_FILE>
          Provide one or more additional (custom) demo file(s)

          Demos are loaded in the following order: Remote (default) demo file, custom
          demo files provided via the 'STACKABLE_DEMO_FILES' environment variable, and
          lastly demo files provided via the '-d/--demo-file' argument(s). If there are
          demos with the same name, the last demo definition will be used.

          Use "stackablectl [OPTIONS] <COMMAND> -d path/to/demos1.yaml -d path/to/demos2.yaml"
          to provide multiple additional demo files.

  -s, --stack-file <STACK_FILE>
          Provide one or more additional (custom) stack file(s)

          Stacks are loaded in the following order: Remote (default) stack file, custom
          stack files provided via the 'STACKABLE_STACK_FILES' environment variable, and
          lastly demo files provided via the '-s/--stack-file' argument(s). If there are
          stacks with the same name, the last stack definition will be used.

          Use "stackablectl [OPTIONS] <COMMAND> -s path/to/stacks1.yaml -s path/to/stacks2.yaml"
          to provide multiple additional stack files.

  -r, --release-file <RELEASE_FILE>
          Provide one or more additional (custom) release file(s)

          Releases are loaded in the following order: Remote (default) release file,
          custom release files provided via the 'STACKABLE_RELEASE_FILES' environment
          variable, and lastly release files provided via the '-r/--release-file'
          argument(s). If there are releases with the same name, the last release
          definition will be used.

          Use "stackablectl [OPTIONS] <COMMAND> -r path/to/releases1.yaml -r path/to/releases2.yaml"
          to provide multiple additional release files.

Helm repository options:
      --helm-repo-stable <URL>
          Provide a custom Helm stable repository URL

          [default: https://repo.stackable.tech/repository/helm-stable/]

      --helm-repo-test <URL>
          Provide a custom Helm test repository URL

          [default: https://repo.stackable.tech/repository/helm-test/]

      --helm-repo-dev <URL>
          Provide a custom Helm dev repository URL

          [default: https://repo.stackable.tech/repository/helm-dev/]

Listing Available Operators

To list the operators that are part of the Stackable Data Platform and their stable versions, run the following command:

$ stackablectl operator list
┌────┬───────────┬────────────────────────────────────────────────────────────────────────────────────────┐
│ #  ┆ OPERATOR  ┆ STABLE VERSIONS                                                                        │
╞════╪═══════════╪════════════════════════════════════════════════════════════════════════════════════════╡
│ 1  ┆ airflow   ┆ 0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 23.1.0, 23.4.0, 23.4.1, 23.7.0               │
├╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 2  ┆ ...       ┆ ...                                                                                    │
├╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 15 ┆ zookeeper ┆ 0.10.0, 0.11.0, 0.12.0, 0.6.0, 0.7.0, 0.8.0, 0.9.0, 23.1.0, 23.4.0, 23.4.1, 23.7.0     │
└────┴───────────┴────────────────────────────────────────────────────────────────────────────────────────┘

This command only includes the stable versions of every operator for clarity. If you’re interested in a particular version of an operator, you can use the describe command to get more details for a specific operator as follows:

$ stackablectl operator describe airflow
 OPERATOR         airflow
 STABLE VERSIONS  0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 23.1.0, 23.4.0, 23.4.1, 23.7.0
 TEST VERSIONS    0.0.0-pr303, 0.0.0-pr304, 0.0.0-pr305, 0.0.0-pr307, 0.0.0-pr308, 0.0.0-pr310, 0.0.0-pr311, 0.0.0-pr312, 0.0.0-pr314, 0.0.0-pr315,
                  0.0.0-pr316, 0.0.0-pr317
 DEV VERSIONS     0.0.0-dev

Installing Operators

If you want to access a Kubernetes cluster, make sure your kubectl Kubernetes client is configured to interact with the Kubernetes cluster. After that, run the following command, which will install the operators in their latest nightly version - built from the main branch of the operators.

$ stackablectl operator install airflow commons secret listener
Installing 4 operators
Installed airflow operator
Installed commons operator
Installed secret operator
Installed listener operator
Installed 4 operators

If you don’t have a Kubernetes cluster available, stackablectl can spin up a kind or minikube Kubernetes cluster for you. Based on the type of local cluster you want to use, ensure you have either kind or minikube installed on your system. See here for more information.

With this command, we installed the operator for Apache Airflow and three operators needed internally by the Stackable Data Platform (commons, secret and listener). As we didn’t specify a specific version to install, the operators were installed in the latest nightly version - built from the main branch of the operators. If you want to install a specific version, you can add the version to each operator to install as follows:

$ stackablectl operator install airflow=23.7 commons=23.7 secret=23.7 listener=23.7
Installing 4 operators
Installed airflow=23.7 operator
Installed commons=23.7 operator
Installed secret=23.7 operator
Installed listener=23.7 operator
Installed 4 operators

As you can see, the four operators were installed in the requested version.

Remember: If you want to install a recommended and tested set of operator versions, look at the stackablectl release command.

Listing Installed Operators

After installing some operators, you can list which operators are installed in your Kubernetes cluster:

$ stackablectl operator installed
┌────────────────────┬─────────┬─────────────────────┬──────────┬──────────────────────────────────────────┐
│ OPERATOR           ┆ VERSION ┆ NAMESPACE           ┆ STATUS   ┆ LAST UPDATED                             │
╞════════════════════╪═════════╪═════════════════════╪══════════╪══════════════════════════════════════════╡
│ airflow-operator   ┆ 23.7.0  ┆ stackable-operators ┆ deployed ┆ 2023-08-23 17:33:01.509777626 +0200 CEST │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ commons-operator   ┆ 23.7.0  ┆ stackable-operators ┆ deployed ┆ 2023-08-23 17:33:04.012698515 +0200 CEST │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ listener-operator  ┆ 23.7.0  ┆ stackable-operators ┆ deployed ┆ 2023-08-23 17:33:07.217309791 +0200 CEST │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ secret-operator    ┆ 23.7.0  ┆ stackable-operators ┆ deployed ┆ 2023-08-23 17:33:06.328410802 +0200 CEST │
└────────────────────┴─────────┴─────────────────────┴──────────┴──────────────────────────────────────────┘

Uninstalling Operators

You can use the stackablectl operator uninstall command to uninstall the operators again.

$ stackablectl operator uninstall airflow commons secret listener
The release airflow-operator was successfully uninstalled.
The release commons-operator was successfully uninstalled.
The release secret-operator was successfully uninstalled.
The release listener-operator was successfully uninstalled.
Uninstalled 4 operators