Use NVIDIA GPUs

Zuletzt aktualisiert am 11. Aug. 2026

In this tutorial you will learn how to set up your STACKIT Kubernetes Engine (SKE) cluster to run NVIDIA GPU workloads. For that we will use the NVIDIA GPU Operator to install the GPU driver.

Cluster setup

Currently, the GPU Operator only supports node pools that use Ubuntu. During the creation of a cluster, choose a node pool using the Ubuntu image and select a machine flavor that supports GPUs. See the list of virtual machine flavors for specific flavor names.

If you already have a cluster, then add a new node pool using the Ubuntu image and a GPU machine flavor. Make sure that all GPU node pools use the same Ubuntu image. Once the cluster is created, setup access as described in access a Kubernetes cluster.

Operator Installation

Install Helm following the official installation instructions. Prefer a package manager such as Homebrew (brew install helm) or apt over downloading scripts directly.

Set up the GPU Operator repository:

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update

Install the GPU Operator in the cluster. Pin a tested version to ensure compatibility with your GPU driver, CUDA, and Kubernetes version. Check the GPU Operator Component Matrix for supported combinations. For example, to install version v26.3.3:

helm install --wait --generate-name \
    -n gpu-operator --create-namespace \
    nvidia/gpu-operator --version v26.3.3

To customize the GPU driver version, add --set driver.version=<driver-version> to the above command. See the GPU Operator Component Matrix for a list of supported NVIDIA GPU driver versions.

Wait until the driver installation is complete and all GPU Operator DaemonSets are ready. This will take a few minutes.

kubectl get daemonset -n gpu-operator

Verify driver installation

To test the driver installation, you could run the following CUDA test container:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  labels:
    run: vectoradd
  name: vectoradd
spec:
  restartPolicy: OnFailure
  containers:
  - name: vectoradd
    image: nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1-ubuntu20.04
    resources:
      limits:
        nvidia.com/gpu: 1
EOF

The nvidia.com/gpu resource limit is required to run containers that use GPUs. Such containers get scheduled to GPU nodes and are granted access to the GPUs. If you have already tainted your GPU node pool (see Optimize Pod scheduling below), add a corresponding toleration to the pod spec.

You can check the logs of the vectoradd Pod:

kubectl logs vectoradd

For further tests, you can for example deploy the Jupyter Notebook example from NVIDIA.

Optimize Pod scheduling

By default, Kubernetes schedules pods to all available nodes. This means that non-GPU workloads can also be scheduled to GPU nodes. To exclusively use GPU nodes for GPU workloads, use the node pool editing screen to add a nvidia.com/gpu taint with effect NoSchedule to the GPU node pool. When using such a taint, make sure that at least one other node pool without a taint exists.

Then add a corresponding toleration to pod specification of the GPU workloads:

spec:
  tolerations:
  - effect: NoSchedule
    key: nvidia.com/gpu
    operator: Exists

Operating system updates

During an operating system update, the nodes in a node pool are iteratively replaced with new nodes running an up-to-date operating system. The GPU Operator automatically installs the driver on the new nodes. This takes a few minutes, after which the new nodes are available for GPU workloads.

To minimize disruptions for your workload, you can set a PodDisruptionBudget for your application. This requires that the application runs at least two instances. When using the maxUnavailable setting, you must at least specify a value of 1. A value of 0 will be ignored and results in a forceful upgrade.

There is a small risk that the driver installation might fail after an operating system update. Make sure to configure the maintenance window of your cluster such that you are able to monitor updates.

Further information

Terraform

You can define a GPU node pool via Terraform. The following example shows a minimal GPU node pool with Ubuntu:

node_pools = [{
  name           = "gpu"
  machine_type   = "g1.medium"
  os_name        = "ubuntu"
  os_version_min = "24.04"
  minimum_nodes  = 1
  maximum_nodes  = 3
  taints = [{
    key    = "nvidia.com/gpu"
    effect = "NoSchedule"
  }]
}]

See the Terraform stackit_ske_cluster documentation for the full schema.

Cleanup

To remove the GPU Operator and test resources:

kubectl delete pod vectoradd
helm uninstall -n gpu-operator <release-name>

To find the release name, run helm list -n gpu-operator.