[elbe-devel] [PATCH v2] dockerfile: make elbe-testing run in docker

Tue Jul 31 11:36:34 CEST 2018

Hi John,

On Mon, Jul 30, 2018 at 10:22:29AM +0200, John Ogness wrote:
> Hi Manuel,
> 
> I have added my Reviewed-by because I believe the Makefile is now doing
> what is intended.
> 
> However, as a user, I am confused about the usefulness of the "stop"
> target. After a "make stop", there are no targets that will get the
> instance running again. The only thing I can do is "make stoprm" and
> then "make start".

I tried to keep the existing make targets and the behaviour as it was before
wherever possible. To avoid breaking existing scripts.

> I am not a docker-user, so maybe this is normal. But I would expect that
> "stop" does what "stoprm" is doing. And the current "stop"
> implementation renamed to something like "pause" with a useful
> "continue" target implemented to get the paused instance running
> again. The motivation for my logic is that targets always have a useful
> counterpart:
> 
>     start <-> stop
>     pause <-> continue
>     build <-> clean

There is nothing like pause/continue in docker. There is no counterpart for
connect. Docker makes a difference between image creation and container
creation. You can create multiple containers from the same image. Youc can
start/stop/rm containers and you can remove images. 'docker run' creates a
container from an image an starts it. I think we should avoid this and use
'docker create' for container creation and 'docker start' for running the
container. Than we should be able to have those parallel targets.

However i'm not sure if i want to brake existing scripts more than necessary.

Anyone opionions about that?

  Manuel

> Feel free to commit this in its current form and maybe open a feature
> request on github if my non-docker-user comments make any sense.
> 
> John Ogness
> 
> On 2018-07-23, Manuel Traut <manut at linutronix.de> wrote:
> > a running libvirtd is needed in the container for elbe-testing
> > this also drops ssh inside the container, because it should be
> > ok, to just attach a bash to the container.
> >
> > also the hard coded kvm-gids are replaced by a solution that
> > should work on all distros.
> >
> > Signed-off-by: Manuel Traut <manut at linutronix.de>
> 
> Reviewed-by: John Ogness <john.ogness at linutronix.de>
> 
> > ---
> >
> > changes since v1:
> >
> >  * mount elbe src in /elbe
> >  * expanse KVMGID and UID during initialization
> >  * add sanity checks if /dev/kvm and /dev/fuse exist
> >  * add sanity check that KVMGID could be determined
> >  * simplify sed call
> >  * add security-opt apparmor:unconfied needed to allow fuse usage inside
> >    the container
> >  * remove --tmpfs /tmp --tmpfs /run --tmpfs /run/lock it might lead to errors
> >  * add a .PHONY target for build start stop stoprm clean and connect
> >
> >  contrib/dockerfile/Dockerfile            | 73 -------------------
> >  contrib/dockerfile/Dockerfile.in         | 93 ++++++++++++++++++++++++
> >  contrib/dockerfile/Makefile              | 71 ++++++++----------
> >  contrib/dockerfile/README.md             | 11 +--
> >  contrib/dockerfile/adds/supervisord.conf | 26 -------
> >  5 files changed, 128 insertions(+), 146 deletions(-)
> >  delete mode 100644 contrib/dockerfile/Dockerfile
> >  create mode 100644 contrib/dockerfile/Dockerfile.in
> >  delete mode 100644 contrib/dockerfile/adds/supervisord.conf
> >
> > diff --git a/contrib/dockerfile/Dockerfile b/contrib/dockerfile/Dockerfile
> > deleted file mode 100644
> > index 705b3ddd..00000000
> > --- a/contrib/dockerfile/Dockerfile
> > +++ /dev/null
> > @@ -1,73 +0,0 @@
> > -#
> > -# ELBE - Debian Based Embedded Rootfilesystem Builder
> > -# Copyright (c) 2014-2015 Silvio Fricke <silvio.fricke at gmail.com>
> > -# Copyright (c) 2018 Manuel Traut <manut at linutronix.de>
> > -#
> > -# SPDX-License-Identifier: GPL-3.0-or-later
> > -
> > -# This Dockefile generate a image for the elbe buildsystem
> > -FROM debian:stretch
> > -
> > -ENV LANG C.UTF-8
> > -
> > -# update and upgrade
> > -RUN export DEBIAN_FRONTEND noninteractive ;\
> > -    apt-get update -y ;\
> > -    apt-get install -y --no-install-recommends \
> > -        build-essential \
> > -        ca-certificates \
> > -        cpio \
> > -        e2tools \
> > -        git \
> > -        kvm \
> > -        libvirt-daemon \
> > -        libvirt-daemon-system \
> > -        make \
> > -        openssh-server \
> > -        p7zip-full \
> > -        python \
> > -        python-apt \
> > -        python-debian \
> > -        python-lxml \
> > -        python-mako \
> > -        python-parted \
> > -        python-suds \
> > -        python-libvirt \
> > -        qemu \
> > -        qemu-keymaps \
> > -        qemu-kvm \
> > -        qemu-system \
> > -        qemu-user-static \
> > -        qemu-utils \
> > -        sudo \
> > -        supervisor \
> > -        tmux \
> > -        vim \
> > -        wget \
> > -        ; \
> > -    apt-get clean -y ;\
> > -    rm -rf /var/lib/apt/lists/*
> > -
> > -# additions
> > -ADD adds/supervisord.conf /etc/supervisord.conf
> > -
> > -# create elbe user
> > -RUN groupadd -g 78 -o -r kvm78          # archlinux
> > -RUN groupadd -g 124 -o -r kvm124        # debian-sid
> > -RUN groupadd -g 232 -o -r kvm232        # ubuntu 14.04
> > -RUN useradd -d /home/elbe -U -G libvirt,kvm,kvm78,kvm124,kvm232,libvirt-qemu -m -s /bin/bash -u 1000 elbe
> > -RUN echo "root:elbe" | chpasswd
> > -RUN echo "elbe:elbe" | chpasswd
> > -
> > -# sudo for elbe
> > -RUN echo "%elbe  ALL=(ALL:ALL) NOPASSWD: ALL" > /etc/sudoers.d/elbegrp
> > -RUN chmod 0440 /etc/sudoers.d/elbegrp
> > -
> > -# add sbc (https://github.com/turicas/sbc)
> > -ADD https://raw.githubusercontent.com/turicas/sbc/develop/sbc /usr/bin/sbc
> > -RUN chmod a+rx /usr/bin/sbc
> > -
> > -# ssh and startup configuration
> > -RUN mkdir -v /var/run/sshd
> > -CMD [ "/lib/systemd/systemd" ]
> > -EXPOSE 22
> > diff --git a/contrib/dockerfile/Dockerfile.in b/contrib/dockerfile/Dockerfile.in
> > new file mode 100644
> > index 00000000..dbfcdfb4
> > --- /dev/null
> > +++ b/contrib/dockerfile/Dockerfile.in
> > @@ -0,0 +1,93 @@
> > +#
> > +# ELBE - Debian Based Embedded Rootfilesystem Builder
> > +# Copyright (c) 2014-2015 Silvio Fricke <silvio.fricke at gmail.com>
> > +# Copyright (c) 2018 Manuel Traut <manut at linutronix.de>
> > +#
> > +# SPDX-License-Identifier: GPL-3.0-or-later
> > +
> > +# This Dockefile generate a image for the elbe buildsystem
> > +FROM dramaturg/debian-systemd
> > +
> > +USER root
> > +
> > +# use a sources.list including security and backports
> > +RUN echo "deb http://ftp.de.debian.org/debian stretch main" > /etc/apt/sources.list; \
> > +    echo "deb http://ftp.de.debian.org/debian stretch-backports main" >> /etc/apt/sources.list; \
> > +    echo "deb http://security.debian.org/debian-security stretch/updates main" >> /etc/apt/sources.list
> > +
> > +# update, upgrade and install elbe runtime-dependencies
> > +RUN export DEBIAN_FRONTEND noninteractive ;\
> > +    apt-get update -y ;\
> > +    apt-get install -y --no-install-recommends \
> > +        systemd \
> > +        ca-certificates \
> > +        sudo \
> > +        vim-nox \
> > +        wget \
> > +        software-properties-common \
> > +        gnupg \
> > +        python3-setuptools \
> > +        python3-yaml \
> > +        python3-jsonschema \
> > +        locales \
> > +        gcc \
> > +        g++ \
> > +        diffstat \
> > +        texinfo \
> > +        gawk \
> > +        chrpath \
> > +        python3-mako \
> > +        fuseiso9660 \
> > +        aptly \
> > +        qemu-system-x86
> > +
> > +RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen && locale-gen
> > +
> > +ENV LANG en_US.UTF-8
> > +ENV LANGUAGE en_US:en
> > +ENV LC_ALL en_US.UTF-8
> > +
> > +# install current elbe-testing
> > +RUN apt-add-repository 'deb http://debian.linutronix.de/elbe-testing stretch main'
> > +RUN wget http://debian.linutronix.de/elbe-testing/elbe-repo.pub
> > +RUN apt-key add elbe-repo.pub
> > +RUN export DEBIAN_FRONTEND noninteractive ;\
> > +    apt-get update -y
> > +RUN export DEBIAN_FRONTEND noninteractive ;\
> > +    apt-get install -y --no-install-recommends \
> > +        elbe \
> > +        elbe-doc
> > +RUN export DEBIAN_FRONTEND noninteractive ;\
> > +    apt-get clean -y
> > +RUN rm -rf /var/lib/apt/lists/*
> > +
> > +# create elbe user
> > +RUN groupadd -g @KVMGID@ -o -r kvm-elbe
> > +RUN useradd -d /home/elbe -U -G kvm-elbe,libvirt -m -s /bin/bash -u @USERID@ elbe
> > +RUN echo "root:elbe" | chpasswd
> > +RUN echo "elbe:elbe" | chpasswd
> > +
> > +RUN rm -f /lib/systemd/system/multi-user.target.wants/*;\
> > +    rm -f /etc/systemd/system/*.wants/*;\
> > +    rm -f /lib/systemd/system/local-fs.target.wants/*; \
> > +    rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
> > +    rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
> > +    rm -f /lib/systemd/system/basic.target.wants/*;\
> > +    rm -f /lib/systemd/system/anaconda.target.wants/*;
> > +
> > +VOLUME [ "/sys/fs/cgroup" ]
> > +VOLUME [ "/elbe" ]
> > +VOLUME [ "/var/cache/elbe" ]
> > +
> > +# sudo for elbe
> > +RUN echo "%elbe  ALL=(ALL:ALL) NOPASSWD: ALL" > /etc/sudoers.d/elbegrp
> > +RUN chmod 0440 /etc/sudoers.d/elbegrp
> > +
> > +# run qemu as root
> > +RUN echo 'user = "root"' >> /etc/libvirt/qemu.conf
> > +RUN echo 'group = "root"' >> /etc/libvirt/qemu.conf
> > +
> > +# run libvirt in systemd on startup
> > +RUN systemctl enable libvirtd
> > +
> > +CMD [ "/lib/systemd/systemd" ]
> > diff --git a/contrib/dockerfile/Makefile b/contrib/dockerfile/Makefile
> > index 1a7c48fa..3400626b 100644
> > --- a/contrib/dockerfile/Makefile
> > +++ b/contrib/dockerfile/Makefile
> > @@ -4,61 +4,52 @@
> >  #
> >  # SPDX-License-Identifier: GPL-3.0-or-later
> >  
> > -IMAGENAME ?= elbe-image
> > -CONTAINERNAME ?= elbe
> > -
> > -ifdef SSH_AUTH_SOCK
> > -	sshauth = -e SSH_AUTH_SOCK=/ssh-auth-sock -v $(SSH_AUTH_SOCK):/ssh-auth-sock
> > -endif
> > +IMAGENAME ?= elbe-devel-image
> > +CONTAINERNAME ?= elbe-devel
> > +KVMGID ?= $(shell ls -n /dev/kvm | awk '{ print $$4 }')
> > +UID ?= $(shell id -u)
> >  
> >  # docker commands
> >  build:
> > +	test -c /dev/kvm  || ( echo "/dev/kvm not found" && false )
> > +	test -c /dev/fuse || ( echo "/dev/fuse not found" && false )
> > +	test -n "$(KVMGID)" || ( echo "detecting groupid of /dev/kvm failed" && false )
> > +	sed -e "s#@KVMGID@#$(KVMGID)#g" \
> > +	    -e "s#@USERID@#$(UID)#g" \
> > +	    Dockerfile.in > Dockerfile
> >  	docker build --build-arg http_proxy=$(http_proxy) \
> >  	             --build-arg https_proxy=$(https_proxy) \
> >  	             --build-arg no_proxy=$(no_proxy) \
> >  	             -t $(IMAGENAME) .
> > +	rm Dockerfile
> >  
> > -run:
> > -	docker run \
> > -		-v $(realpath ../../.):/elbe \
> > -		--env http_proxy=$(http_proxy) \
> > -		--env https_proxy=$(https_proxy) \
> > -		--env no_proxy=$(no_proxy) \
> > -		--device /dev/kvm:/dev/kvm \
> > +start:
> > +	docker ps | grep $(CONTAINERNAME)$$ || \
> > +	docker run --name $(CONTAINERNAME) -d \
> > +		-e container=docker \
> > +		-e http_proxy=$(http_proxy) \
> > +		-e https_proxy=$(https_proxy) \
> > +		-e no_proxy=$(no_proxy) \
> > +		-v $(realpath ../../.):/elbe -w /elbe\
> >  		-v /sys/fs/cgroup:/sys/fs/cgroup:ro \
> >  		--cap-add SYS_ADMIN \
> >  		--security-opt seccomp:unconfined \
> > -		--tmpfs /tmp \
> > -		--tmpfs /run \
> > -		--tmpfs /run/lock \
> > -		-e container=docker \
> > -		$(sshauth) \
> > -		-d \
> > -		-ti \
> > +		--security-opt apparmor:unconfined \
> >  		--group-add kvm \
> > -		--name \
> > -		$(CONTAINERNAME) \
> > -		$(IMAGENAME) \
> > -		/lib/systemd/systemd
> > -
> > -getip:
> > -	docker inspect -f '{{ .NetworkSettings.IPAddress }}' $(CONTAINERNAME)
> > -
> > -start: run getip
> > +		--device /dev/kvm \
> > +		--device /dev/fuse \
> > +		$(IMAGENAME)
> >  
> >  stop:
> > -	docker stop $(CONTAINERNAME)
> > +	-docker stop $(CONTAINERNAME)
> >  
> >  stoprm: stop
> > -	docker rm $(CONTAINERNAME)
> > +	-docker rm $(CONTAINERNAME)
> > +
> > +clean: stoprm
> > +	-docker rmi $(IMAGENAME)
> >  
> > -# ssh related functions
> > -cleanssh:
> > -	IP=$(shell docker inspect -f '{{ .NetworkSettings.IPAddress }}' ${CONTAINERNAME}) ;\
> > -	[[ -n "$${IP}" ]] && ssh-keygen -R $${IP}
> > +connect: start
> > +	docker exec -tiu $(UID) $(CONTAINERNAME) /bin/bash
> >  
> > -connect:
> > -	SBC=$(shell which sbc) ;\
> > -	IP=$(shell docker inspect -f '{{ .NetworkSettings.IPAddress }}' ${CONTAINERNAME}) ;\
> > -	ssh-copy-id elbe@$${IP} ;\
> > -	$$SBC ssh -XA elbe@$${IP}
> > +.PHONY: build start stop stoprm clean connect
> > diff --git a/contrib/dockerfile/README.md b/contrib/dockerfile/README.md
> > index 58219a4f..20407502 100644
> > --- a/contrib/dockerfile/README.md
> > +++ b/contrib/dockerfile/README.md
> > @@ -12,8 +12,8 @@ devices.
> >  [docker][doc] is an open-source project to easily create lightweight, portable,
> >  self-sufficient containers from any application.
> >  
> > -This is a Dockerfile to generate a elbe development environment for systems
> > -other than debian based.
> > +This is a Dockerfile to generate a elbe development and runtime environment for
> > +systems other than debian based.
> >  
> >  [doc]: https://www.docker.io "Docker Homepage"
> >  [elb]: http://elbe-rfs.org   "ELBE Homepage"
> > @@ -30,13 +30,10 @@ is `elbe-image` and a started container name is `elbe`. This names are
> >  changeable via `IMAGENAME` and `CONTAINERNAME` environment variables.
> >  
> >  * `build`: build the image
> > -* `start` start a container, mounts the elbe git-archive to `/elbe` and gives
> > -  back the ip address
> > +* `start` start a container, mounts the elbe git-archive to `/elbe`
> >  * `stop`: stop a running container
> >  * `stoprm`: stop and remove the container
> > -* `getip`: return ip address of a running container
> > -* `connect`: connect via ssh to the container
> > -* `cleanssh`: remove the used ip address (see `getip`) from your `${HOME}/.ssh/known_host`
> > +* `connect`: attach to a running container
> >  
> >  After `connect` you can find the elbe git repository under `/elbe`.
> >  
> > diff --git a/contrib/dockerfile/adds/supervisord.conf b/contrib/dockerfile/adds/supervisord.conf
> > deleted file mode 100644
> > index f59ce051..00000000
> > --- a/contrib/dockerfile/adds/supervisord.conf
> > +++ /dev/null
> > @@ -1,26 +0,0 @@
> > -; taken by https://github.com/sullof/docker-sshd
> > -; SPDX-License-Identifier: GPL-3.0-or-later
> > -[unix_http_server]
> > -file=/tmp/supervisor.sock ; (the path to the socket file)
> > -
> > -[supervisord]
> > -logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
> > -logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
> > -logfile_backups=10 ; (num of main logfile rotation backups;default 10)
> > -loglevel=info ; (log level;default info; others: debug,warn,trace)
> > -pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
> > -nodaemon=false ; (start in foreground if true;default false)
> > -minfds=1024 ; (min. avail startup file descriptors;default 1024)
> > -minprocs=200 ; (min. avail process descriptors;default 200)
> > -
> > -; the below section must remain in the config file for RPC
> > -; (supervisorctl/web interface) to work, additional interfaces may be
> > -; added by defining them in separate rpcinterface: sections
> > -[rpcinterface:supervisor]
> > -supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
> > -
> > -[supervisorctl]
> > -serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket
> > -
> > -[program:openssh]
> > -command=/usr/sbin/sshd