Obelisk Core 1.0.0 Help

Setup

Quick Start

Option 1: Local Deployment using `oblx_kind.py`

  1. Set up a Python virtual environment and install dependencies:

    cd kubernetes/python python3 -m venv .venv source .venv/bin/activate pip install -r requirements.txt

  2. Run the script to initialize the cluster:

    python oblx_kind.py --init

Option 2: Cluster deployment using Helmfile

  1. Ensure you have a Kubernetes cluster and `kubectl` configured.

  2. Set up a Python virtual environment and install dependencies:

    cd kubernetes/python python3 -m venv .venv source .venv/bin/activate pip install -r requirements.txt

  3. Run Helmfile to deploy from the `kubernetes` directory:

    cd kubernetes helmfile sync --state-values-set host=<your_host>

Detailed Instructions

Prerequisites

Ensure you have the following installed:

Option 1: Local Deployment using oblx_kind.py

This option sets up a local Kubernetes cluster using Kind and deploys the necessary components for Obelisk.

Steps

  1. Set Up Python Environment:

    We recommend using a virtual environment to isolate the dependencies for this project. The .venv directory is already included in the .gitignore file.

    cd kubernetes/python python3 -m venv .venv

    Make sure to use python3 when creating the virtual environment.

    Activating the virtual environment isolates your project's dependencies.

    source .venv/bin/activate
    .venv\Scripts\Activate.ps1
    .venv\Scripts\activate.bat

    Your shell prompt will change to indicate the active environment (e.g., (.venv)).

    pip install -r requirements.txt

  2. Run the Script:

    python oblx_kind.py [--cluster-name CLUSTER_NAME] [--init]

    When you want to repeat setup but already have the virtual environment, you can skip the first step, just activate the virtual environment and directly run the script.

    source .venv/bin/activate python oblx_kind.py [--cluster-name CLUSTER_NAME] [--init]

Examples

Create a cluster with the default name:

python oblx_kind.py

Create a cluster with a custom name:

python oblx_kind.py --cluster-name my-cluster

Create a cluster with the default name and trigger the init of the helmfile command, initializing Obelisk with namespaces and datasets defined in kubernetes/obelisk/init-demo.yaml:

python oblx_kind.py --init

Create a cluster with a custom name and trigger the init of the helmfile command:

python oblx_kind.py --cluster-name my-cluster --init

Limitations

  • Intended for local development and testing.

  • Performance and resource limitations of your local machine may affect the cluster.

  • Upgrading requires rerunning the script without the --init flag.

Option 2: Cluster Deployment using Helmfile

Steps

  1. Set Up Python Environment:

    cd kubernetes/python python3 -m venv .venv source .venv/bin/activate # Linux/macOS (Bash/Zsh/fish) .venv\Scripts\Activate.ps1 # Windows (PowerShell) pip install -r requirements.txt

    Make sure to use python3 when creating the virtual environment.

  2. Configure Environment Variables:

    export OBLX_HOST=<your_host> export OBLX_TLS_ENABLED=<true|false> export KEYCLOAK_ADMIN_USER=<your_keycloak_admin_user> export KEYCLOAK_ADMIN_PASSWORD=<your_keycloak_admin_password>

    Alternatively you can set these values in the helmfile.yaml file or override them as commandline arguments when running helmfile via --state-values-set.

  3. Run Helmfile:

    cd kubernetes helmfile sync

    Or if you want to set arguments on the command line:

    helmfile sync --state-values-set host=<your_host>,tlsEnabled=<true|false>,keycloakAdminUser=<your_keycloak_admin_user>,keycloakAdminPassword=<your_keycloak_admin_password>

    When you want to repeat setup but already have the virtual environment, you can skip the first step, just activate the virtual environment and directly run the script.

    source .venv/bin/activate cd kubernetes helmfile sync

Limitations

  • Requires an existing Kubernetes cluster.

  • Proper configuration of kubectl and environment variables is necessary.

  • Requires having a Host Domain Name (e.g. obelisk.example.org) forwarding to the Traefik Ingress Controller. See Proxy and hostname configuration for more details.

Scripts and Helmfile Explanation

oblx_kind.py

  • Purpose: Automates the setup of a local Kind cluster and deploys Obelisk components.

  • Overview:

    • Checks if Kind and Docker are installed.

    • Creates and configures a Kind cluster.

    • Ensures a local Docker registry is running and connected to the Kind network.

    • Deploys components using Helmfile.

    • Sets up necessary Kubernetes secrets and configurations.

oblx_bootstrap.py
python oblx_bootstrap.py --help

  • Purpose: Initializes Kubernetes resources for Obelisk.

  • Overview:

    • Checks if the namespace is already initialized using an annotation.

    • Initializes ClickHouse tables if they are missing.

    • Retrieves Keycloak client secrets and creates Kubernetes secrets.

    • Initializes OpenFGA with the authorization model.

    • Marks the namespace as initialized by setting an annotation.

helmfile.yaml

  • Purpose: Defines the Helm charts and configurations for deploying Obelisk components.

  • Overview:

    • Specifies environment-specific values.

    • Lists Helm chart repositories.

    • Defines Helm chart releases and their configurations.

    • Includes hooks for pre-deployment and post-deployment tasks.

Reinitializing the Namespace

The oblx_bootstrap.py script guards it initialization by setting an annotation on the namespace. If you need to reinitialize the namespace due to changes or issues, you can reset the annotation:

kubectl annotate namespace obelisk obelisk.discover.ilabt.imec.be/initialized=false --overwrite

Then run the bootstrap script again.

Proxy and hostname configuration

We use a reverse proxy to expose Obelisk and Keycloak to the outside world. The proxy should be configured to terminate SSL and forward traffic to Traefik, which is deployed as a dependency. We use Traefik as the ingress controller for Obelisk and Keycloak.

This proxy should be reachable through a DNS hostname such as obelisk.example.org. The hostname should point to the IP address of the proxy or load balancer.

Traefik is configured to set up a NodePort service on ports 32080 and 32443 for HTTP and HTTPS respectively. The proxy or load balancer should forward traffic to these ports by adding all cluster node IPs.

Alternative Setups

Alternative setups for exposing services include:

  • Load Balancer: Using a cloud provider's load balancer.

  • TLS Termination on Ingress Level: Configuring Traefik or another ingress controller to handle TLS termination.

We welcome contributions and documentation regarding these setups.

Common Issues and Troubleshooting

ImportError: No module named '*'

Ensure the virtual environment is activated and required packages are installed.

kubectl, helm, kustomize or helmfile commands not found:

Verify all requirements are installed and added to your system's PATH.

Kind cluster creation fails

Check Docker is running and you have permissions to create Docker containers. Verify you are using the latest versions of Kind and Docker.

Helmfile deployment fails

Review Helmfile logs for errors. Ensure correct Kubernetes configurations and Helm charts. Ensure you are in the kubernetes directory.

Kind cluster unreachable

Confirm the Kind cluster is running and your kubectl context is set to the Kind cluster.

nip.io address can't be resolved

Some DNS resolvers block *.nip.io addresses. Use a different DNS resolver, add nip.io to the allow list, or add an entry to your hosts file.

Conclusion

By following these steps, you should be able to set up and run Obelisk. If you encounter issues, refer to the troubleshooting section or open an issue.

15 July 2025
Obelisk CoreUsage