How to enhance the security of your cluster

This page contains helpful information on how to improve the security of your SKE cluster.

Workload security

The security of your workloads is central to the health of your Kubernetes environment. The following recommendations will help you secure your workloads optimally:

Container image security

• Only use trusted container registries
• Implement an automated scanning process for security vulnerabilities
• Use multi-stage builds to minimize image size
• Avoid using the "latest" tag
• Document the origin of your images (software supply chain / bill of materials)

Pod security standards

Pod security standards define different security levels for pods:

Restricted profile (recommended)

• Prevents privileged containers
• Forces non-root users
• Restricts Linux capabilities
• Blocks privilege escalation

Baseline profile

• Blocks known privilege escalation
• Allows standard Linux capabilities
• Requires container user/group

Privileged profile (only if absolutely necessary)

• No restrictions
• Only use for system components

Implementation:

PSS

apiVersion: pod-security.kubernetes.io/v1
kind: Namespace
metadata:
  name: mein-namespace
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/warn: restricted

Resource management

Resource and request limits are essential for the stability of your cluster. You can implement this as follows:

Resource Limits & Requests

spec:
  containers:
    - name: app
      resources:
        requests:
          memory: "64Mi"
          cpu: "250m"
        limits:
          memory: "128Mi"
          cpu: "500m"

Namespace resource quotas:

Resource Limits & Requests

apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-quota
  namespace: my-namespace
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 4Gi
    limits.cpu: "8"
    limits.memory: 8Gi

Network isolation

Api server access control (ACL)

Securing Kubernetes API server access is one of the most important aspects of Kubernetes cluster security. The ACL functionality allows you to restrict access to the API server of your cluster to certain IP ranges.

The following configurations are recommended:

Production environments

• Restrict access to known corporate IP ranges
• Use dedicated IP ranges for CI/CD systems
• Avoid using 0.0.0.0/0 (open access)

Example configurations

• Internal STACKIT access: 193.148.160.0/19 and 45.129.40.0/21 (you can find the current STACKIT public IP ranges here)
• Company network: your specific IP range (for example, 1.2.3.0/24)
• Individual systems: specific IP addresses (for example, 1.2.3.4/32)

Best practices

• Document all permitted IP ranges
• Check the ACL configuration regularly
• Implement a process for ACL changes
• Consider backup and disaster recovery (DR) locations in ACL configuration

Open access (0.0.0.0/0) should be avoided in production environments. For development and test environments, this could be acceptable, but should be documented and checked regularly.

Network policies

Network policies are your primary tool for controlling pod-to-pod traffic:

Default deny policy (starting point for each namespace):

Default Deny Policy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
  namespace: mein-namespace
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

Example of a granular policy:

Granular Policy (example)

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-allow
  namespace: mein-namespace
spec:
  podSelector:
    matchLabels:
      app: api
  policyTypes:
    - Ingress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              purpose: frontend
      ports:
        - protocol: TCP
          port: 8080

Ingress/egress control

• Use IngressController for external access
• Limit egress traffic to necessary destinations
• Use ExternalName services for external services

Role based access control (RBAC)

Role based access control (RBAC) is the primary means of access control in Kubernetes. Here are best practices for implementing in your cluster:

Roles and cluster roles

Example of a restricted developer role:

RBAC

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: mein-namespace
  name: developer
rules:
  - apiGroups: [""]
    resources: ["pods", "services"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["apps"]
    resources: ["deployments", "statefulsets"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]

Service accounts

Best practices for service accounts:

Service Accounts

apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-service-account
  namespace: mein-namespace
automountServiceAccountToken: false # deactivate if not necessary

Secret management

Kubernetes secrets

Basic secret creation:

Kubernetes Secret (example)

apiVersion: v1
kind: Secret
metadata:
  name: app-secrets
type: Opaque
data:
  username: BASE64_ENCODED_USERNAME
  password: BASE64_ENCODED_PASSWORD

Best Practices:

• Use external secret management systems where possible, such as the STACKIT Secrets Manager
• Implement secret rotation
• Restrict access through RBAC

Monitoring and logging

Monitoring

Example for Prometheus scrape configuration via annotations:

Kubernetes Secret (example)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: meine-app
spec:
  template:
    metadata:
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "9090"
        prometheus.io/path: "/metrics"
    spec:
      containers:
        - name: my-app
          #... further configuration

Logging

• Implement structured logging
• Centralize logs
• Set log retention policies
• Implement log rotation

Sources of information on known vulnerabilities

• Kubernetes security announcements
• CVE database

Update mechanisms

• Container image updates via CI/CD pipeline
• Kubernetes version updates (provider-controlled)
• Dependency updates in applications

Error handling and logging mechanisms

Health checks

Implement liveness and readiness probes:

Liveness & Readiness probes

spec:
  containers:
    - name: app
      livenessProbe:
        httpGet:
          path: /healthz
          port: 8080
        initialDelaySeconds: 3
        periodSeconds: 3
      readinessProbe:
        httpGet:
          path: /ready
          port: 8080
        initialDelaySeconds: 3
        periodSeconds: 3

Pod disruption budgets

Protect your workloads from unwanted downtime:

PDBs (example)

apiVersion: policy / v1
kind: PodDisruptionBudget
metadata:
  name: app-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: my-app

Tips for the role and rights concept

Best practices for roles

• Enforce namespace isolation
• Assign minimum authorizations
• Carry out regular audits

Risky combinations to avoid

• Cluster admin rights for service accounts
• Wildcards in RBAC Rules
• Excessive permissions in multiple namespaces

Appendix

Useful kubectl commands

Kubectl commands

## RBAC Checks
kubectl auth can-i --list --namespace=mein-namespace

### Pod Security Standards Check
kubectl get ns mein-namespace -o yaml | grep pod-security

### Network Policy Test
kubectl run nginx --image=nginx --restart=Never --namespace=my-namespace
kubectl exec nginx -n mein-namespace -- wget -qO- --timeout=2 http://another-service

Security checklist for you

• Pod security standards implemented
• Network policies defined
• RBAC configuration checked
• Resource limits set
• Monitoring activated
• Logging configured
• Secret management implemented
• Health checks configured