[elbe-devel] Historical (2012-12-11): Draft ELBE Article

Tue Mar 12 16:04:39 CET 2024

Hi,

This is a HISTORICAL DOCUMENT posted to this mailing list in 2012. Since
the mailing list was not archiving at that time, it is no longer
available. I am re-posting it because it was a nice overview of the
features of ELBE at that time and implicitly communicates the
requirements at that time. This _might_ be useful when considering
making fundamental changes to ELBE.

John Ogness

Date: Tue, 11 Dec 2012 22:45:59 +0100
From: Manuel Traut <manut at linutronix.de>
To: elbe-devel at linutronix.de

ELBE - another aproach for building embedded Linux systems
==========================================================

ELBE means Embedded Linux Build Environment. It was designed to ease
building and maintaining Linux based embedded systems. This article describes
the ideas behind ELBE and how they differ from the often used cross-build
approach.

An ELBE project is described by a XML file. This file is interpreted by the
'elbe' tool.

ELBE uses an emulator (qemu or kvm) to provide a 'native' environment similar
to the embedded target.

Instead of cross-compiling all open-source components of the root-filesystem,
ELBE uses Debian binary packages. The XML file also defines which parts of the
full-blown Debian system should be used for the embedded target root-filesystem.

Remember that building and maintaining an embedded Linux based system
typically includes the following steps:

 1) Building/install all needed open-source libraries and applications

 2) Compiling self-developed libraries and applications

 3) Install self-developed components inside the root-filesystem

 4) A board specific bootloader and Linux kernel needs to be created and compiled
     for the target

 5) Creating NAND or Harddisk Images typically containing bootloader, kernel and
     the root-filesystem.

 6) Tracking (security) updates of the used open-source components and update
     them if necessary.

The next paragraphs describe how ELBE handles these steps:

1) ELBE uses Debian to get the needed open-source components

Debian is a well maintained Linux distribution and offers packages for all
relevant embedded architectures. The packages provided by Debian are tested by
a big userbase and 'http://qa.debian.org' gives good informations about the
quality of a package; including bugreports. This makes the addition of new
packages much easier, than building each package and it's dependencies cross.
Because Debian provides binary packages including informations of all necessary
dependencies.

The Debian mirrors and Debian packages that should be used are described in the
ELBE XML File.

A special Debian package including the debian-insaller kernel and initrd for
the target architecture needs to be on one of the provided Debian mirrors.
(This package is not available on official debian-mirrors).
For common architectures they are packaged in the repository:
'http://debian.linutronix.de/elbe' which can be added additional to an official
or own Debian mirror.

This special package is downloaded by 'elbe create' while the project build
directory is created.

After typing 'make' inside the build directory the Debian installer is started
within a qemu or kvm session. The Debian installer was preseeded by ELBE to
avoid asking any questions. Additional preseeding rules can be specified in
the ELBE XML file.

To create the target root-filesystem out of the full-blown Debian installation,
there are several copy modes, to copy the content of the buildenvironment into
the target root-filesystem. With so called 'finetuning rules' (that are also
specified in the XML file) the target root-filesystem can be further customized
and stripped down (e.g. delete docs, man-pages, locales, ...).

After an ELBE run, there is an ISO Image inside the build directory. This Image
contains all Debian packages used by ELBE. With the command 'elbe setcdrom' this
image can be set as mirror for further ELBE runs. Then ELBE can be used without
network connections or an local web-server providing a Debian mirror.

2) Build own applications native within an emulated environment

To build own libraries and applications the ELBE 'build environment' can be
used by typing 'make run-con' inside the build directory. The emulator starts
a Debian system that can be used to compile own libraries and applications
'native' for the target architecture. The system itself ensures, that the right
header files and library versions are used, because the target root-filesystem
was extracted from the same system.

If the applications doesn't need any special embedded hardware, it can be even
debugged with gdb inside the emulator. To make a quick test on the real target
hardware, the libraries and applications can be transfered via network to the
target hardware.

3) Debianize everything

It's recommended to debianize its own projects. The Debian Tool 'dh_make'
assists by this step. A debianized project can be build by running
'dpkg-buildpackage' in the projects base directory, inside the ELBE
buildenvironment. The resulting Debian packages can be added to an own
repository, or to the ISO image. If the packages are added to the 'pkg-list'
of the ELBE XML file, the own packages are inside the target root-filesystem
within the next ELBE run.

4) The ELBE buildenvironment can be used to build bootloader and kernel sources

It's normaly no problem to build the bootloader and kernel with a
cross-toolchain, because they have no dependencies to other libraries.
But it is also possible to build those components inside the ELBE
buildenvironment. It's recommended to create Debian packages of these
components and add them to the repository of the other packages. The bootloader
and kernel can be installed into the root-filesystem by adding the according
Debian packages to the 'pkg-list' of the ELBE XML file.

To get a Debian package of the Linux kernel 'make deb-pkg' inside the Linux
kernel source tree can be used.

5) Image generation is done in a post-install step of the debian-installer

UBI-NAND and HD images can be defined within the ELBE XML file.
Also the /etc/fstab file is defined inside the ELBE XML file.
The target image(s) are created according to the fstab. It's possible to copy
binaries like the linux kernel or a bootloader directly into a static UBI
volume.

It's also possible to just create a cpio Image or tarball of the hole target
root-filesystem.

The image generation process is done in a post-install hook of the
debian-installer.

6) Track changes on mirrors between two ELBE runs

Based on the ELBE project XML file a XML file called 'source.xml' is created
inside the 'build' directory. This file is modified by an ELBE run. After an
ELBE run it includes a 'full-pkg-list' including all packages used (including
packages installed as a dependencies) with their md5sums and package version
information.

If this 'source.xml' file is used for another ELBE run, ELBE reports all
packages that don't match with the given version number or md5sum. The file
that includes this information is called validation.txt.

If an official Debian (security) mirror is added to the XML file, the
validation.txt can be used to identify updates to the root-filesystem.

Obtaining ELBE
^^^^^^^^^^^^^^

The ELBE tooling itself is implemented in python and licensed under the terms of
the GPL.

Further informations can be found at 'http://linutronix.de/elbe'.

ELBE can be retrieved from github 'http://github.com/linutronix/elbe' or by the
Debian package management system.

The following repository needs to be added to the '/etc/sources.list' file of
a Debian/squeeze installation:

--
'deb http://debian.linutronix.de/elbe squeeze main'
--

After an

--
'apt-get update'
--

ELBE and its dependencies can be installed by

--
'apt-get install elbe'
--

There is a public Mailinglist 'elbe-devel at linutronix.de' that can be subscribed
by ???

Future Directions
^^^^^^^^^^^^^^^^^

An ELBE daemon should be developed to run inside the ELBE buildenvironment.
Modified XML files can be send to the daemon that than creates the specified
target root-filesystem. This can be used during development to speedup target
root-filesystem generation.

Tools to sync files and execute commands inside the buildenvironment will be
developed. They can be integrated into a graphical IDE to make the 'qemu native
build' as comfortable, as a real native build.

There is also a TODO file inside the ELBE source tree describing future
development directions.