How to setup a private container registry

This tutorial guides you through the steps required to set up a private container registry on top of Kubernetes. By following this guide you will set up your own registry running on your cluster with a valid certificate by Let’s Encrypt and a proper DNS name. It will be protected by basic auth, so you have to add pull secrets in your cluster. You then have the option to pull images from docker, to retag them to point to your private registry and push them.
After these steps you will be able to point your Kubernetes pods to the private registry and pull the images from there.

Prerequisites

Ability to create DNS entries
Access to a running Kubernetes Cluster. For more information visit the cluster access documentation.

Setup a private container registry

Deploy the registry on Kubernetes

To install the latest version of Helm (read more: Helm Install Documentation), run the following command:

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

Install an ingress controller if you don’t have one already. For this tutorial nginx will be used:

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install ingress-nginx ingress-nginx/ingress-nginx

wait for the ingress’ load balancer to become ready. As soon as it’s ready create a DNS entry for the resulting external IP and save the resulting DNS name in $DNS_NAME.

## to get the ip
kubectl --namespace default get service ingress-nginx-controller -o jsonpath="{.status.loadBalancer.ingress[0].ip}"

### to set the DNS name
export DNS_NAME=<your-dns-name>

Install cert-manager:

kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.5.3/cert-manager.yaml

Configure a ClusterIssuer to point to Let’s Encrypt. Set email to receive certificate news:

export ACME_EMAIL=<your-email>

save the following file as clusterissuer.yaml:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    # You must replace this email address with your own.
    # Let's Encrypt will use this to contact you about expiring
    # certificates, and issues related to your account.
    email: $ACME_EMAIL
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      # Secret resource that will be used to store the account's private key.
      name: letsencrypt-issuer-key
    # Add a single challenge solver, HTTP01 using nginx
    solvers:
    - http01:
        ingress:
          class: nginx

Replace the dummy values and apply the file to create the cluster issuer:

cat clusterissuer.yaml | envsubst | kubectl apply -f

Create basic auth credentials to secure your registry:

export REGISTRY_USER=<username>
export REGISTRY_PASSWORD=<passord>
htpasswd -bc auth $REGISTRY_USER $REGISTRY_PASSWORD
kubectl create secret generic basic-auth --from-file=auth

Create the registry and save the following file as registry-template.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: docker-repo-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: registry
  labels:
    app: registry
spec:
  replicas: 1
  selector:
    matchLabels:
      app: registry
  template:
    metadata:
      name: registry
      labels:
        app: registry
    spec:
      containers:
        - name: registry
          image: registry:2.6.2
          volumeMounts:
            - name: repo-vol
              mountPath: "/var/lib/registry"
      volumes:
        - name: repo-vol
          persistentVolumeClaim:
            claimName: docker-repo-pvc

---

apiVersion: v1
kind: Service
metadata:
  name: registry
spec:
  selector:
    app: registry
  ports:
    - port: 5000
      targetPort: 5000

---

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: registry
  labels:
    app: registry
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: basic-auth
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required'
    nginx.ingress.kubernetes.io/proxy-body-size: 4G
spec:
  ingressClassName: nginx
  rules:
    - host: $DNS_NAME
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: registry
                port:
                  number: 5000
  tls:
    - hosts:
        - $DNS_NAME
      secretName: ske-executor-tls

Replace the dummy values and apply:

cat registry-template.yaml | envsubst | kubectl apply -f -

Connect to the registry from your local client

docker login $DNS_NAME -u $REGISTRY_USER -p $REGISTRY_PASSWORD

Push new images into the registry

## pull any image from official registry
docker pull nginx:latest

### retag
docker tag nginx:latest $DNS_NAME/nginx:latest

### push to private registry
docker push $DNS_NAME/nginx:latest

Use the newly pushed image in an example deployment

Create a pull secret

kubectl create secret docker-registry registry-pull-secret --docker-server=$DNS_NAME --docker-username=$REGISTRY_USER --docker-password=$REGISTRY_PASSWORD

Add the secret to the default service account

kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "registry-pull-secret"}]}'

Run a pod with an image from your registry

kubectl run nginx-pod --image=$DNS_NAME/nginx:latest

Watch the status of the created pod until its Running

k get pod nginx-pod -o wide -w

Now you’re ready to use the private registry for any image that you like.