[elbe-devel] [PATCH 1/2] Add optional mirror for host architecture

[Torben Hohn]

On Thu, Jun 04, 2020 at 10:54:47AM +0200, Köry Maincent wrote:
> Hello Torben,
> 
> Thanks for the review.

No Problem, sorry for being so slow, but this pylint patchqueue creates
conflicts all over the place and needs to go in quickly.

> 
> 
> On 04/06/2020 09:14, Torben Hohn wrote:
> > On Wed, May 20, 2020 at 11:53:03AM +0200, Kory Maincent wrote:
> > > If the primary mirror does not contain the packages for both target and host
> > > architecture the build_host_sysroot process fails.
> > > 
> > > This patch adds support for describing in the XML configuration an additional
> > > optional mirror containing packages for the host architecture, like:
> > > <mirror>
> > > <host>http://archive.ubuntu.com/ubuntu bionic main universe</host>
> > > </mirror>
> > 
> > Do we have to add "bionic main universe" here ?
> > The primary mirrors are only needed for debootstrap.
> > 
> > The other mirror entries support [arch=amd64] and we should be able to
> > have all other mirror urls in the urllist without having problems.
> > If a mirror only has amd64 packages, it needs to be marked that way.
> 
> If we add [arch=amd64] in the mirror urls of the urllist there will be a
> conflict with noauth.

Then we have to fix noauth.
BTW you could also not use noauth, and add trusted=yes directly into the
mirror url. That way it wouldnt stop checking all signatures, and only
check the relevant signatures.

Is noauth reuired to build ubuntu ?
We should make it possible to install the proper ubuntu key then.
Maybe we just need to install

https://packages.debian.org/buster/ubuntu-keyring



> apt can not support two separated options like this:
> deb [trusted=yes] [arch=armhf] http://...
> It need to be written like this:
> deb [trusted=yes arch=armhf] http://...
> 
> > 
> > 
> > > 
> > > It also adds the architecture type in the sources.list file to avoid issues
> > > with apt-get update.
> > > 
> > > Signed-off-by: Kory Maincent <kory.maincent at bootlin.com>
> > > ---
> > >   elbepack/elbexml.py | 40 ++++++++++++++++++++++++++++++----------
> > >   elbepack/rfs.py     | 23 +++++++++++++----------
> > >   schema/dbsfed.xsd   |  7 +++++++
> > >   3 files changed, 50 insertions(+), 20 deletions(-)
> > > 
> > > diff --git a/elbepack/elbexml.py b/elbepack/elbexml.py
> > > index fa8158392..ece069706 100644
> > > --- a/elbepack/elbexml.py
> > > +++ b/elbepack/elbexml.py
> > > @@ -132,6 +132,13 @@ class ElbeXML(object):
> > >           return mirror.replace("LOCALMACHINE", "10.0.2.2")
> > > +    def get_host_mirror(self):
> > > +        if self.prj.has("mirror/host"):
> > > +            m = self.prj.node("mirror")
> > > +            mirror = m.text("host").strip() + "\n"
> > > +
> > > +        return mirror
> > 
> > errm... this fails when <host> does not exist.
> > lets integrate this in get_primary_mirror()
> > and have a proper fallback.
> > 
> > 
> > 
> > 
> > > +
> > >       def get_primary_mirror(self, cdrompath, initvm=True):
> > >           if self.prj.has("mirror/primary_host"):
> > >               m = self.prj.node("mirror")
> > > @@ -146,7 +153,7 @@ class ElbeXML(object):
> > >           return replace_localmachine(mirror, initvm)
> > >       # XXX: maybe add cdrom path param ?
> > > -    def create_apt_sources_list(self, build_sources=False, initvm=True):
> > > +    def create_apt_sources_list(self, build_sources=False, initvm=True, arch="default"):
> > >           if self.prj is None:
> > >               return "# No Project"
> > > @@ -155,24 +162,37 @@ class ElbeXML(object):
> > >           noauth = ""
> > >           if self.prj.has("noauth"):
> > > -            noauth = "[trusted=yes] "
> > > +            noauth = "trusted=yes "
> > > +
> > > +        if arch == "default":
> > > +            arch = self.text("project/buildimage/arch", key="arch")
> > > +
> > > +        arch_prj = self.text("project/buildimage/arch", key="arch")
> > >           mirror = ""
> > > +        option = "[" + noauth + " arch=" + arch +"] "
> > >           if self.prj.has("mirror/primary_host"):
> > > -            mirror += "deb " + noauth + self.get_primary_mirror(None)
> > > -            mirror += " " + self.prj.text("suite") + " main\n"
> > > +            if (arch != arch_prj) and self.prj.has("mirror/host"):
> > > +                mirror += "deb " + option + self.get_host_mirror()
> > > +            else:
> > > +                mirror += "deb " + option + self.get_primary_mirror(None)
> > > +                mirror += " " + self.prj.text("suite") + " main\n"
> > > +
> > 
> > i find this logic too complex.
> > We really want the host and target suite to be identical, or we would
> > end up with different gcc versions.
> > 
> > I also think its too complex to decide, whether we are building a host
> > sysroot, based on architectures.
> > 
> > Please add a bool parameter "hostsdk" to get_primary_mirror() which will
> > make it look for <host> and fallback in the case its not there.
> > 
> > it just needs to be passed down from build_host_sysroot() via
> > BuildEnv.__init__() and BuildEnv.debootstrap()
> > 
> > this will simplify this patch a lot.
> 
> 
> Ok.
> 
> 
> 
> -- 
> Köry Maincent, Bootlin
> Embedded Linux and kernel engineering
> https://bootlin.com

-- 
Torben Hohn
Linutronix GmbH | Bahnhofstrasse 3 | D-88690 Uhldingen-Mühlhofen
Phone: +49 7556 25 999 18; Fax.: +49 7556 25 999 99

Hinweise zum Datenschutz finden Sie hier (Informations on data privacy 
can be found here): https://linutronix.de/kontakt/Datenschutz.php

Linutronix GmbH | Firmensitz (Registered Office): Uhldingen-Mühlhofen | 
Registergericht (Registration Court): Amtsgericht Freiburg i.Br., HRB700 
806 | Geschäftsführer (Managing Directors): Heinz Egger, Thomas Gleixner