[elbe-devel] [PATCH] dockerfile: make elbe-testing run in docker
John Ogness
john.ogness at linutronix.de
Thu Jul 19 17:36:01 CEST 2018
Hi Manuel,
See my comments inline...
On 2018-07-12, Manuel Traut <manut at linutronix.de> wrote:
> a running libvirtd is needed in the container for elbe-testing
> this also drops ssh inside the container, because it should be
> ok, to just attach a bash to the container.
>
> also the hard coded kvm-gids are replaced by a solution that
> should work on all distros.
>
> Signed-off-by: Manuel Traut <manut at linutronix.de>
> ---
> contrib/dockerfile/Dockerfile | 73 ------------------
> contrib/dockerfile/Dockerfile.in | 94 ++++++++++++++++++++++++
> contrib/dockerfile/Makefile | 62 +++++++---------
> contrib/dockerfile/README.md | 11 +--
> contrib/dockerfile/adds/supervisord.conf | 26 -------
> 5 files changed, 123 insertions(+), 143 deletions(-)
> delete mode 100644 contrib/dockerfile/Dockerfile
> create mode 100644 contrib/dockerfile/Dockerfile.in
> delete mode 100644 contrib/dockerfile/adds/supervisord.conf
>
> diff --git a/contrib/dockerfile/Dockerfile b/contrib/dockerfile/Dockerfile
> deleted file mode 100644
> index 705b3ddd..00000000
> --- a/contrib/dockerfile/Dockerfile
> +++ /dev/null
> @@ -1,73 +0,0 @@
> -#
> -# ELBE - Debian Based Embedded Rootfilesystem Builder
> -# Copyright (c) 2014-2015 Silvio Fricke <silvio.fricke at gmail.com>
> -# Copyright (c) 2018 Manuel Traut <manut at linutronix.de>
> -#
> -# SPDX-License-Identifier: GPL-3.0-or-later
> -
> -# This Dockefile generate a image for the elbe buildsystem
> -FROM debian:stretch
> -
> -ENV LANG C.UTF-8
> -
> -# update and upgrade
> -RUN export DEBIAN_FRONTEND noninteractive ;\
> - apt-get update -y ;\
> - apt-get install -y --no-install-recommends \
> - build-essential \
> - ca-certificates \
> - cpio \
> - e2tools \
> - git \
> - kvm \
> - libvirt-daemon \
> - libvirt-daemon-system \
> - make \
> - openssh-server \
> - p7zip-full \
> - python \
> - python-apt \
> - python-debian \
> - python-lxml \
> - python-mako \
> - python-parted \
> - python-suds \
> - python-libvirt \
> - qemu \
> - qemu-keymaps \
> - qemu-kvm \
> - qemu-system \
> - qemu-user-static \
> - qemu-utils \
> - sudo \
> - supervisor \
> - tmux \
> - vim \
> - wget \
> - ; \
> - apt-get clean -y ;\
> - rm -rf /var/lib/apt/lists/*
> -
> -# additions
> -ADD adds/supervisord.conf /etc/supervisord.conf
> -
> -# create elbe user
> -RUN groupadd -g 78 -o -r kvm78 # archlinux
> -RUN groupadd -g 124 -o -r kvm124 # debian-sid
> -RUN groupadd -g 232 -o -r kvm232 # ubuntu 14.04
> -RUN useradd -d /home/elbe -U -G libvirt,kvm,kvm78,kvm124,kvm232,libvirt-qemu -m -s /bin/bash -u 1000 elbe
> -RUN echo "root:elbe" | chpasswd
> -RUN echo "elbe:elbe" | chpasswd
> -
> -# sudo for elbe
> -RUN echo "%elbe ALL=(ALL:ALL) NOPASSWD: ALL" > /etc/sudoers.d/elbegrp
> -RUN chmod 0440 /etc/sudoers.d/elbegrp
> -
> -# add sbc (https://github.com/turicas/sbc)
> -ADD https://raw.githubusercontent.com/turicas/sbc/develop/sbc /usr/bin/sbc
> -RUN chmod a+rx /usr/bin/sbc
> -
> -# ssh and startup configuration
> -RUN mkdir -v /var/run/sshd
> -CMD [ "/lib/systemd/systemd" ]
> -EXPOSE 22
> diff --git a/contrib/dockerfile/Dockerfile.in b/contrib/dockerfile/Dockerfile.in
> new file mode 100644
> index 00000000..2cc815b2
> --- /dev/null
> +++ b/contrib/dockerfile/Dockerfile.in
> @@ -0,0 +1,94 @@
> +#
> +# ELBE - Debian Based Embedded Rootfilesystem Builder
> +# Copyright (c) 2014-2015 Silvio Fricke <silvio.fricke at gmail.com>
> +# Copyright (c) 2018 Manuel Traut <manut at linutronix.de>
> +#
> +# SPDX-License-Identifier: GPL-3.0-or-later
> +
> +# This Dockefile generate a image for the elbe buildsystem
> +FROM dramaturg/debian-systemd
> +
> +USER root
> +
> +# use a sources.list including security and backports
> +RUN echo "deb http://ftp.de.debian.org/debian stretch main" > /etc/apt/sources.list; \
> + echo "deb http://ftp.de.debian.org/debian stretch-backports main" >> /etc/apt/sources.list; \
> + echo "deb http://security.debian.org/debian-security stretch/updates main" >> /etc/apt/sources.list
> +
> +# update, upgrade and install elbe runtime-dependencies
> +RUN export DEBIAN_FRONTEND noninteractive ;\
> + apt-get update -y ;\
> + apt-get install -y --no-install-recommends \
> + systemd \
> + ca-certificates \
> + sudo \
> + vim-nox \
> + wget \
> + software-properties-common \
> + gnupg \
> + python3-setuptools \
> + python3-yaml \
> + python3-jsonschema \
> + locales \
> + gcc \
> + g++ \
> + diffstat \
> + texinfo \
> + gawk \
> + chrpath \
> + python3-mako \
> + fuseiso9660 \
> + aptly \
> + qemu-system-x86
> +
> +RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen && locale-gen
> +
> +ENV LANG en_US.UTF-8
> +ENV LANGUAGE en_US:en
> +ENV LC_ALL en_US.UTF-8
> +
> +# install current elbe-testing
> +RUN apt-add-repository 'deb http://debian.linutronix.de/elbe-testing stretch main'
> +RUN wget http://debian.linutronix.de/elbe-testing/elbe-repo.pub
> +RUN apt-key add elbe-repo.pub
> +RUN export DEBIAN_FRONTEND noninteractive ;\
> + apt-get update -y
> +RUN export DEBIAN_FRONTEND noninteractive ;\
> + apt-get install -y --no-install-recommends \
> + elbe \
> + elbe-doc
> +RUN export DEBIAN_FRONTEND noninteractive ;\
> + apt-get clean -y
> +RUN rm -rf /var/lib/apt/lists/*
> +
> +# create elbe user
> +RUN groupadd -g @KVMGID@ -o -r kvm-elbe
> +RUN groupadd -g @FUSEGID@ -o -r fuse-elbe
> +RUN useradd -d /home/elbe -U -G fuse-elbe,kvm-elbe,libvirt -m -s /bin/bash -u @USERID@ elbe
> +RUN echo "root:elbe" | chpasswd
> +RUN echo "elbe:elbe" | chpasswd
> +
> +RUN rm -f /lib/systemd/system/multi-user.target.wants/*;\
> + rm -f /etc/systemd/system/*.wants/*;\
> + rm -f /lib/systemd/system/local-fs.target.wants/*; \
> + rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
> + rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
> + rm -f /lib/systemd/system/basic.target.wants/*;\
> + rm -f /lib/systemd/system/anaconda.target.wants/*;
> +
> +VOLUME [ "/sys/fs/cgroup" ]
> +VOLUME [ "/home" ]
> +VOLUME [ "/var/cache/elbe" ]
> +
> +# sudo for elbe
> +RUN echo "%elbe ALL=(ALL:ALL) NOPASSWD: ALL" > /etc/sudoers.d/elbegrp
> +RUN chmod 0440 /etc/sudoers.d/elbegrp
> +
> +# run qemu as root
> +RUN echo 'user = "root"' >> /etc/libvirt/qemu.conf
> +RUN echo 'group = "root"' >> /etc/libvirt/qemu.conf
> +
> +# run libvirt in systemd on startup
> +RUN systemctl enable libvirtd
> +
> +CMD [ "/lib/systemd/systemd" ]
> diff --git a/contrib/dockerfile/Makefile b/contrib/dockerfile/Makefile
> index 1a7c48fa..569deda3 100644
> --- a/contrib/dockerfile/Makefile
> +++ b/contrib/dockerfile/Makefile
> @@ -4,61 +4,49 @@
> #
> # SPDX-License-Identifier: GPL-3.0-or-later
>
> -IMAGENAME ?= elbe-image
> -CONTAINERNAME ?= elbe
> -
> -ifdef SSH_AUTH_SOCK
> - sshauth = -e SSH_AUTH_SOCK=/ssh-auth-sock -v $(SSH_AUTH_SOCK):/ssh-auth-sock
> -endif
> +IMAGENAME ?= elbe-devel-image
> +CONTAINERNAME ?= elbe-devel
> +KVMGID ?= `ls -n /dev/kvm | awk '{ print $$4 }'`
> +FUSEGID ?= `ls -n /dev/fuse | awk '{ print $$4 }'`
I recommend evaluating the expressions during initial Makefile
expansion, rather than as inline build commands. This makes it a little
easier to debug because you can see the values expanded in the build
commands. And while at it, I would add UID to the list:
KVMGID ?= $(shell ls -n /dev/kvm | awk '{ print $$4 }')
FUSEGID ?= $(shell ls -n /dev/fuse | awk '{ print $$4 }')
UID = $(shell id -u)
>
> # docker commands
> build:
On my test system, there was initially no /dev/fuse. Debugging this took
longer than it should have because FUSEGID got set to "", which
eventually caused the docker build to explode. I recommend adding a
couple tests here:
test -n "$(KVMGID)" || ( echo "/dev/kvm not found" && false )
test -n "$(FUSEGID)" || ( echo "/dev/fuse not found" && false )
> + sed -e "s#@KVMGID@#$(KVMGID)#g" Dockerfile.in > Dockerfile.in2
> + sed -e "s#@FUSEGID@#$(FUSEGID)#g" Dockerfile.in2 > Dockerfile.in3
> + sed -e "s#@USERID@#`id -u`#g" Dockerfile.in3 > Dockerfile
sed can evaluate multiple expressions in 1 command. It looks a little
nicer IMHO:
sed -e "s#@KVMGID@#$(KVMGID)#g" \
-e "s#@FUSEGID@#$(FUSEGID)#g" \
-e "s#@USERID@#$(UID)#g" \
Dockerfile.in > Dockerfile
> docker build --build-arg http_proxy=$(http_proxy) \
> --build-arg https_proxy=$(https_proxy) \
> --build-arg no_proxy=$(no_proxy) \
> -t $(IMAGENAME) .
> + rm Dockerfile Dockerfile.in?
... which then reduces the cleanup to:
rm Dockerfile
>
> -run:
> - docker run \
> +start:
> + docker run --name $(CONTAINERNAME) -d \
> + -e container=docker \
> + -e http_proxy=$(http_proxy) \
> + -e https_proxy=$(https_proxy) \
> + -e no_proxy=$(no_proxy) \
> -v $(realpath ../../.):/elbe \
> - --env http_proxy=$(http_proxy) \
> - --env https_proxy=$(https_proxy) \
> - --env no_proxy=$(no_proxy) \
> - --device /dev/kvm:/dev/kvm \
> + -v `pwd`/../..:/home -w /home \
I do not understand what you want here. I suppose it should look like
this:
-v $(realpath ../..):/elbe \
-v $(realpath ../..):/home -w /home \
But what is the point of that? You won't have a /home/elbe
directory. Maybe you meant:
-v $(realpath ../..):/elbe \
-v $(realpath ../../..):/home -w /home \
with the assumption that the source directory is called elbe. Then there
would be a /home/elbe. But that is a big assumption.
> -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
> --cap-add SYS_ADMIN \
> --security-opt seccomp:unconfined \
> + --group-add kvm \
> + --device /dev/kvm:/dev/kvm \
> + --device /dev/fuse:/dev/fuse \
> --tmpfs /tmp \
> --tmpfs /run \
> --tmpfs /run/lock \
> - -e container=docker \
> - $(sshauth) \
> - -d \
> - -ti \
> - --group-add kvm \
> - --name \
> - $(CONTAINERNAME) \
> - $(IMAGENAME) \
> - /lib/systemd/systemd
> -
> -getip:
> - docker inspect -f '{{ .NetworkSettings.IPAddress }}' $(CONTAINERNAME)
> -
> -start: run getip
> + $(IMAGENAME)
>
> stop:
> - docker stop $(CONTAINERNAME)
> + -docker stop $(CONTAINERNAME)
>
> stoprm: stop
> - docker rm $(CONTAINERNAME)
> + -docker rm $(CONTAINERNAME)
>
> -# ssh related functions
> -cleanssh:
> - IP=$(shell docker inspect -f '{{ .NetworkSettings.IPAddress }}' ${CONTAINERNAME}) ;\
> - [[ -n "$${IP}" ]] && ssh-keygen -R $${IP}
> +clean: stoprm
> + -docker rmi $(IMAGENAME)
>
> -connect:
> - SBC=$(shell which sbc) ;\
> - IP=$(shell docker inspect -f '{{ .NetworkSettings.IPAddress }}' ${CONTAINERNAME}) ;\
> - ssh-copy-id elbe@$${IP} ;\
> - $$SBC ssh -XA elbe@$${IP}
> +connect: start
start as a dependency seems really strange. It means connect will fail
if it is already started. I would expect the following sequece of
command to work:
make build
make start
make connect
make connect
make connect
make stop
> + docker exec -tiu `id -u` $(CONTAINERNAME) /bin/bash
And if UID is set during expansion, this can be:
docker exec -tiu $(UID) $(CONTAINERNAME) /bin/bash
Also, please add .PHONY targets.
> diff --git a/contrib/dockerfile/README.md b/contrib/dockerfile/README.md
> index 58219a4f..20407502 100644
> --- a/contrib/dockerfile/README.md
> +++ b/contrib/dockerfile/README.md
> @@ -12,8 +12,8 @@ devices.
> [docker][doc] is an open-source project to easily create lightweight, portable,
> self-sufficient containers from any application.
>
> -This is a Dockerfile to generate a elbe development environment for systems
> -other than debian based.
> +This is a Dockerfile to generate a elbe development and runtime environment for
> +systems other than debian based.
>
> [doc]: https://www.docker.io "Docker Homepage"
> [elb]: http://elbe-rfs.org "ELBE Homepage"
> @@ -30,13 +30,10 @@ is `elbe-image` and a started container name is `elbe`. This names are
> changeable via `IMAGENAME` and `CONTAINERNAME` environment variables.
>
> * `build`: build the image
> -* `start` start a container, mounts the elbe git-archive to `/elbe` and gives
> - back the ip address
> +* `start` start a container, mounts the elbe git-archive to `/elbe`
> * `stop`: stop a running container
> * `stoprm`: stop and remove the container
> -* `getip`: return ip address of a running container
> -* `connect`: connect via ssh to the container
> -* `cleanssh`: remove the used ip address (see `getip`) from your `${HOME}/.ssh/known_host`
> +* `connect`: attach to a running container
>
> After `connect` you can find the elbe git repository under `/elbe`.
>
> diff --git a/contrib/dockerfile/adds/supervisord.conf b/contrib/dockerfile/adds/supervisord.conf
> deleted file mode 100644
> index f59ce051..00000000
> --- a/contrib/dockerfile/adds/supervisord.conf
> +++ /dev/null
> @@ -1,26 +0,0 @@
> -; taken by https://github.com/sullof/docker-sshd
> -; SPDX-License-Identifier: GPL-3.0-or-later
> -[unix_http_server]
> -file=/tmp/supervisor.sock ; (the path to the socket file)
> -
> -[supervisord]
> -logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
> -logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
> -logfile_backups=10 ; (num of main logfile rotation backups;default 10)
> -loglevel=info ; (log level;default info; others: debug,warn,trace)
> -pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
> -nodaemon=false ; (start in foreground if true;default false)
> -minfds=1024 ; (min. avail startup file descriptors;default 1024)
> -minprocs=200 ; (min. avail process descriptors;default 200)
> -
> -; the below section must remain in the config file for RPC
> -; (supervisorctl/web interface) to work, additional interfaces may be
> -; added by defining them in separate rpcinterface: sections
> -[rpcinterface:supervisor]
> -supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
> -
> -[supervisorctl]
> -serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket
> -
> -[program:openssh]
> -command=/usr/sbin/sshd
John Ogness
More information about the elbe-devel
mailing list