Using Docker as an image building service in a containerd cluster

Last updated: 2021-01-11 10:20:30

    Overview

    In a Kubernetes cluster, some CI/CD workflow may need to use Docker to provide image packaging services. This can be implemented by the Docker of the host. Mount Docker's UNIX Socket (/var/run/docker.sock) as the hostPath to the CI/CD service Pod, and then call the Docker of the host through the UNIX Socket to build image in the container. This method is simple and can save more resources than running a Docker host inside of another Docker host (Docker in Docker). However, this method may encounter the following problems:

    • It cannot be performed in a cluster whose Runtime is containerd.
    • If it is not controlled, it may overwrite the existing image on the node.
    • When you need to modify the Docker Daemon configuration file, it may affect other services.
    • It is not safe in the multi-tenancy scenario. After the privileged Pod obtains the UNIX Socket of Docker, the container in the Pod can not only call the host's Docker to build the image, delete the existing image or container, or even operate other containers through docker exec interface.

    For the first problem above, Kubernetes has officially announced that Docker will be disused after version 1.22. These users may switch their service to containerd. For some clusters that require a containerd, and still use Docker to build the image without changing the CI/CD service process, you can add the DinD container to the original Pod as a sidecar or use DaemonSet to deploy the Docker service dedicated to building the image on the node.
    This document describes the following two ways to use Docker to build images on the CI/CD workflow:

    Directions

    Using DinD as the Sidecar of Pod

    For the implementation principle of DinD (Docker in Docker), see DinD Official Document. The following example shows that adding a Sidecar to clean-ci container, and combined with emptyDir, making the clean-ci container can access the DinD container through UNIX sockets.

    apiVersion: v1
    kind: Pod
    metadata:
       name: clean-ci
    spec:
       containers:
       - name: dind
         image: 'docker:stable-dind'
         command:
         - dockerd
         - --host=unix:///var/run/docker.sock
         - --host=tcp://0.0.0.0:8000
         securityContext:
           privileged: true
         volumeMounts:
         - mountPath: /var/run
           name: cache-dir
       - name: clean-ci
         image: 'docker:stable'
         command: ["/bin/sh"]
         args: ["-c", "docker info >/dev/null 2>&1; while [ $? -ne 0 ] ; do sleep 3; docker info >/dev/null 2>&1; done; docker pull library/busybox:latest; docker save -o busybox-latest.tar library/busybox:latest; docker rmi library/busybox:latest; while true; do sleep 86400; done"]
         volumeMounts:
         - mountPath: /var/run
           name: cache-dir
       volumes:
       - name: cache-dir
         emptyDir: {}

    Using DaemOnset to deploy Docker on each containerd node

    This method is simple. You just need to directly forward the DaemonSet in the containerd cluster (mounting hostPath). In order not to affect the /var/run path on the node, you can specify other paths.

    1. Use the following YAML to deploy DaemonSet, as shown below:
      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
         name: docker-ci
      spec:
         selector:
           matchLabels:
             app: docker-ci
         template:
           metadata:
             labels:
               app: docker-ci
           spec:
             containers:
             - name: docker-ci
               image: 'docker:stable-dind'
               command:
               - dockerd
               - --host=unix:///var/run/docker.sock
               - --host=tcp://0.0.0.0:8000
               securityContext:
                 privileged: true
               volumeMounts:
               - mountPath: /var/run
                 name: host
             volumes:
             - name: host
               hostPath:
                 path: /var/run
    2. Share the same hostPath between the service Pod and DaemonSet, as shown below:
      apiVersion: v1
      kind: Pod
      metadata:
         name: clean-ci
      spec:
         containers:
         - name: clean-ci
           image: 'docker:stable'
           command: ["/bin/sh"]
           args: ["-c", "docker info >/dev/null 2>&1; while [ $? -ne 0 ] ; do sleep 3; docker info >/dev/null 2>&1; done; docker pull library/busybox:latest; docker save -o busybox-latest.tar library/busybox:latest; docker rmi library/busybox:latest; while true; do sleep 86400; done"]
           volumeMounts:
           - mountPath: /var/run
             name: host
         volumes:
         - name: host
           hostPath:
             path: /var/run

    Was this page helpful?

    Was this page helpful?

    • Not at all
    • Not very helpful
    • Somewhat helpful
    • Very helpful
    • Extremely helpful
    Send Feedback
    Help