Setup cluster audit logging
Zuletzt aktualisiert am
Kubernetes audit logging provides a security-relevant chronological record of cluster activities, allowing administrators to trace who did what, when, and where. This visibility is essential for:
- detecting malicious activity
- troubleshooting unexpected state changes
- demonstrating regulatory compliance in a production environment
Audit logs track requests made to the kube-apiserver from the moment they are received until a response is sent. When a user, service account, or internal component interacts with the cluster, the API server evaluates the request against an audit policy, which dictates what data to log (ranging from simple metadata to full request and response bodies) and at what level.
For STACKIT Kubernetes Engine (SKE) clusters, you can configure your cluster to forward these logs to a STACKIT Telemetry Router. Once the logs reach the router, you can configure the appropriate telemetry links and destinations to enable long-term log storage.
Configuration
Section titled “Configuration”Currently, audit logging integration can only be enabled via STACKIT CLI:
-
Generate and fetch a basic cluster configuration using the STACKIT CLI.
-
Modify the following part of the configuration to enable the audit logging feature:
{"audit": {"enabled": true}} -
Use the following command to create or update a SKE cluster using audit logging:
Terminal window # create new clusterstackit ske cluster create -p <PROJECT_ID> <CLUSTER_NAME> --payload @cluster-configuration.json# update existing clusterstackit ske cluster update -p <PROJECT_ID> <CLUSTER_NAME> --payload @cluster-configuration.json
This activates audit log forwarding to the Telemetry Router. To disable the feature, set the enabled parameter to false or omit the audit section entirely in a subsequent configuration update.
Audit policy
Section titled “Audit policy”Audit logs for SKE clusters are generated based on the following audit policy:
apiVersion: audit.k8s.io/v1kind: PolicyomitStages:- "RequestReceived"
rules:# Don't log creates for calico tier CRD; the healthcheck tries to create it every iteration even though it already exists, which is noisy- level: None verbs: ["create"] resources: - group: "crd.projectcalico.org" resources: ["tiers"] users: - system:serviceaccount:kube-system:calico-kube-controllers
# Do not log watch operations performed by kube-proxy on endpoints and services- level: None users: ["system:kube-proxy"] verbs: ["watch"] resources: - group: "" # core resources: ["endpoints", "services", "services/status"]
# Do not log get operations performed by the system:nodes group (kubelet) on nodes and their statuses- level: None userGroups: ["system:nodes"] verbs: ["get"] resources: - group: "" # core resources: ["nodes", "nodes/status"]
# Do not log get and update operations performed by controller manager, scheduler, and endpoint-controller on endpoints in kube-system namespace- level: None users: - system:kube-controller-manager - system:kube-scheduler - system:serviceaccount:kube-system:endpoint-controller verbs: ["get", "update"] namespaces: ["kube-system"] resources: - group: "" # core resources: ["endpoints"]
# Do not log endpointslice updates from endpointslice service accounts because very noisy- level: None verbs: ["update"] resources: - group: "discovery.k8s.io" resources: ["endpointslices"] users: - system:serviceaccount:kube-system:endpointslicemirroring-controller - system:serviceaccount:kube-system:endpointslice-controller
# Do not log get operations performed by apiserver on namespaces and their statuses/finalizations- level: None users: ["system:apiserver"] verbs: ["get"] resources: - group: "" # core resources: ["namespaces", "namespaces/status", "namespaces/finalize"]
# Exclude kube-apiserver getting the 'kubernetes' endpoints/endpointslices- level: None users: ["system:apiserver"] verbs: ["get"] resources: - group: "" # Core API group resources: ["endpoints"] namespaces: ["default"] resourceNames: ["kubernetes"] - group: "discovery.k8s.io" resources: ["endpointslices"] namespaces: ["default"] resourceNames: ["kubernetes"]
# Do not log HPA fetching metrics- level: None users: - system:serviceaccount:kube-system:horizontal-pod-autoscaler verbs: ["get", "list"] resources: - group: "metrics.k8s.io"
# Do not log leases and events at all- level: None resources: - group: "" # core resources: ["events"] - group: "coordination.k8s.io" resources: ["leases"]
# Do not log non-security-relevant endpoints and resources- level: None nonResourceURLs: - "/healthz*" - "/livez*" - "/readyz*" - "/swagger*" - "/version" - "/"
# Secrets, ConfigMaps, TokenRequest and TokenReviews can contain sensitive & binary data, so only log at the Metadata level- level: Metadata resources: - group: "" # core resources: ["secrets", "configmaps", "serviceaccounts/token"] - group: authentication.k8s.io resources: ["tokenreviews"]
# Node and pod status calls from nodes are high-volume and can be large; do not log responses for expected updates from nodes- level: Request users: ["system:serviceaccount:kube-system:node-problem-detector"] verbs: ["update", "patch"] resources: - group: "" # core resources: ["nodes/status", "pods/status"]- level: Request userGroups: ["system:nodes"] verbs: ["update", "patch"] resources: - group: "" # core resources: ["nodes/status", "pods/status"]
# Deletecollection calls can be large; don't log responses for expected namespace deletions- level: Request users: ["system:serviceaccount:kube-system:namespace-controller"] verbs: ["deletecollection"]
# Get responses can be large; don't log response- level: Request verbs: ["get", "list", "watch"] resources: - group: "" - group: "admissionregistration.k8s.io" - group: "apiextensions.k8s.io" - group: "apiregistration.k8s.io" - group: "apps" - group: "authentication.k8s.io" - group: "authorization.k8s.io" - group: "autoscaling" - group: "batch" - group: "certificates.k8s.io" - group: "extensions" - group: "metrics.k8s.io" - group: "networking.k8s.io" - group: "policy" - group: "rbac.authorization.k8s.io" - group: "scheduling.k8s.io" - group: "settings.k8s.io" - group: "storage.k8s.io"
# Default level for known APIs- level: RequestResponse resources: - group: "" - group: "admissionregistration.k8s.io" - group: "apiextensions.k8s.io" - group: "apiregistration.k8s.io" - group: "apps" - group: "authentication.k8s.io" - group: "authorization.k8s.io" - group: "autoscaling" - group: "batch" - group: "certificates.k8s.io" - group: "extensions" - group: "metrics.k8s.io" - group: "networking.k8s.io" - group: "policy" - group: "rbac.authorization.k8s.io" - group: "scheduling.k8s.io" - group: "settings.k8s.io" - group: "storage.k8s.io"
# Catch-all rule but only metadata- level: Metadata