[elbe-devel] [PATCH 01/11] cov: Introduce elbe-ci coverage

[Olivier Dion]

On Wed, 19 Aug 2020, Torben Hohn <torben.hohn at linutronix.de> wrote:
> On Mon, Aug 17, 2020 at 12:20:10PM -0400, Olivier Dion wrote:
>> Coverage of source code is done by using python-coverage [1].  The
>> version 4.x is very important to use because this is the only version
>> available on Buster for the initvm (see Notes).
>> 
>> Coverage is initiated by a top process.  This process set an
>> environment variable to tell others processes (mainly recursive
>> execution of Elbe) that coverage should be activated.  The descendants
>> of the top process can only do coverage, but they will never combine
>> nor do report of the measurement they have made.  This is done by the
>> top process only.
>> 
>> The coverage is a per-thread attribute.  python-coverage [1] should be
>> thread aware, but I haven't tested it yet.  The current problem with
>> this implementation however is that it's using os.environ.  Which
>> means that if thread A starts coverage, then children forked from
>> thread B will also start coverage of them self.  This could be fix by
>> using a per-thread environment and merging this environment in
>> functions in 'elbepack/shellhelper.py' and ensuring that all recursive
>> call to Elbe use these functions.
>
> its ok if coverage is per process.
> Even if we are testing parallel builds, we want to know, how much our
> test covered.
>
> would that make things simpler ?
>
> i mean it would suffice, if there was a global initvm coverage flag.
> if an initvm is intended to be used in testing. We could just activate
> it / and restart the daemon.

I'm not sure however how do we want to restart the initvm with coverage
enable.  Since there's a context manager that wrap all of Elbe, we could
add 'ELBE_COVERAGE' to /etc/default/elbe-daemon then issue a 'systemd
restart elbe-daemon' using SSH.  This could be changed when using docker
by making the base image having ELBE_COVERAGE already in the daemon
environment.

What do you think?

-- 
Olivier Dion
Linutronix GmbH | Bahnhofstrasse 3 | D-88690 Uhldingen-Mühlhofen