How to setup RWX storage

This Tutorial show you how to enable RWX (Read Write Many) storage on STACKIT Kubernetes Engine (SKE) by installing Longhorn.

Longhorn is an OpenSource Cloud-Native distributed storage engine built on and for Kubernetes and is developed by Rancher.

Requirements

• STACKIT Kubernetes Engine cluster
• Minimum Node Requirements
• Helm and kubectl installed
• (Optional) S3 Object Storage for the backup

Architecture

Longhorn is storing the volume data on the local node storage in order for Kubernetes to replace or upgrade nodes without loosing Longhorns data. It is necessary to at least add a minimum of three nodes to the cluster. The Data is synced to all nodes, on an event of a node going down a new node is added and automatically synced. In order for this to work properly it is very important that the max surge of the node pool is set to one otherwise this can lead to dataloss.

Installation

To install Longhorn to you cluster you need do the following steps:

Add the Longhorn Helm Chart repo

helm repo add longhorn https://charts.longhorn.io
helm repo update

Enable the SKE cluster to support Longhorn

To enable the SKE cluster to running Longhorn you need to setup an Daemonset that is starting the necessary services in order for Longhorn to work on the nodes

kubectl create namespace iscsi-node
kubectl apply -f https://raw.githubusercontent.com/stackitcloud/ske-longhorn-rwx/main/enableISCSI.yml

Setup Longhorn S3 backup (Optional)

To setup the Longhorn build in S3-Backup solution you need to pass the following values as file (values.yaml) with the Helm install command

defaultSettings:
 backupTarget: s3://<bucket>@<region>/<path>/ # s3://mybucket@eu01/object.storage.eu01.onstackit.cloud/
 backupTargetCredentialSecret: <secret-name>
 allowRecurringJobWhileVolumeDetached: true # optional to map volumes if deployment is scaled down during backup

Add a Kubernetes Secret to store the Access and secret key for the S3 bucket.

kubectl create secret generic <name> \
 --from-literal=AWS_ACCESS_KEY_ID=<s3-access-key> \
 --from-literal=AWS_SECRET_ACCESS_KEY=<s3 secret key> \
 --from-literal=AWS_ENDPOINTS='https://object.storage.eu01.onstackit.cloud' \
 -n longhorn-system

Install Longhorn via the Helm Chart

For more information please follow the official docs: Install with Helm.

helm install longhorn longhorn/longhorn --namespace longhorn-system --create-namespace -f values.yaml --version $(curl -s https://api.github.com/repos/longhorn/longhorn/releases/latest | jq -r '.tag_name')

• To use the install command provided here you need to have curl and jq installed
• Parameter -f is optional and only be needed if you enable the backup for an example values.yaml please take a look at the GitHub repository.

Configure Longhorn

After Longhorn in installed you then can configure a backup job (if previously enabled) and create a custom storage class in order to provision Longhorn based PVCs. For more backup options please take a look at the official documentation: Backup and Restore.
If your Longhorn cluster consists of more than three hosts, you need to change settings in the storageClass.yaml

### Create the Backup Job
kubectl apply -f https://raw.githubusercontent.com/stackitcloud/ske-longhorn-rwx/main/recurringBackupJob.yaml

### Create the Storage Class
kubectl apply -f https://raw.githubusercontent.com/stackitcloud/ske-longhorn-rwx/main/storageClass.yaml

Management

To manage the Longhorn installation you can enter the Management WebUI of Longhorn via tunnelling the port to your local machine. Refer to Accessing the UI.