[elbe-devel] [PATCH 1/2] Add optional mirror for host architecture
Torben Hohn
torben.hohn at linutronix.de
Tue Jul 21 14:02:40 CEST 2020
On Fri, Jun 05, 2020 at 01:10:50PM +0200, Köry Maincent wrote:
> Hello Torben,
>
> On 05/06/2020 09:41, Köry Maincent wrote:
> > Hello Torben,
> >
> > On 04/06/2020 18:13, Torben Hohn wrote:
> > > On Thu, Jun 04, 2020 at 10:54:47AM +0200, Köry Maincent wrote:
> > > > Hello Torben,
> > > >
> > > > Thanks for the review.
> > >
> > > No Problem, sorry for being so slow, but this pylint patchqueue creates
> > > conflicts all over the place and needs to go in quickly.
> >
> > No problem, pylint seems an important patch to add, good luck with it.
> >
> > >
> > > >
> > > >
> > > > On 04/06/2020 09:14, Torben Hohn wrote:
> > > > > On Wed, May 20, 2020 at 11:53:03AM +0200, Kory Maincent wrote:
> > > > > > If the primary mirror does not contain the packages for
> > > > > > both target and host
> > > > > > architecture the build_host_sysroot process fails.
> > > > > >
> > > > > > This patch adds support for describing in the XML
> > > > > > configuration an additional
> > > > > > optional mirror containing packages for the host architecture, like:
> > > > > > <mirror>
> > > > > > <host>http://archive.ubuntu.com/ubuntu bionic main universe</host>
> > > > > > </mirror>
> > > > >
> > > > > Do we have to add "bionic main universe" here ?
> > > > > The primary mirrors are only needed for debootstrap.
> > > > >
> > > > > The other mirror entries support [arch=amd64] and we should be able to
> > > > > have all other mirror urls in the urllist without having problems.
> > > > > If a mirror only has amd64 packages, it needs to be marked that way.
> > > >
> > > > If we add [arch=amd64] in the mirror urls of the urllist there will be a
> > > > conflict with noauth.
> > >
> > > Then we have to fix noauth.
> > > BTW you could also not use noauth, and add trusted=yes directly into the
> > > mirror url. That way it wouldnt stop checking all signatures, and only
> > > check the relevant signatures.
> >
> > Yes, we could indeed remove the noauth from Elbe code.
>
> Do you prefer to remove noauth or to add a specific architecture attribute
> to the url mirror?
>
> Take account that noauth is also used by:
> - pbuilder debootstrap config with no-check-gpg parameter
> - pbuilder src package, with allow-unauthenticate parameter pass to apt-get
> source.
Olivier is working on this now.
>
>
> >
> >
> > >
> > > Is noauth reuired to build ubuntu ?
> >
> > Elbe seemed to not have trouble to generate an Ubuntu image without
> > noauth, but when I want to work with pbuilder I got an issue with the
> > keyring:
> >
> > W: GPG error: http://127.0.0.1:8080/var/cache/elbe/c26b1694-ff47-4efc-9346-13b3b0811c47/repo
> > bionic InRelease: The following signatures couldn't be verified because
> > the public key is not available: NO_PUBKEY 4A4B63D0DEC56F34
> > E: The repository 'http://127.0.0.1:8080/var/cache/elbe/c26b1694-ff47-4efc-9346-13b3b0811c47/repo
> > bionic InRelease' is not signed.
> > W: Aborting with an error
> >
> > Strangely it is related to the local repository but with noauth the
> > issue disappear.
> >
> >
> > > We should make it possible to install the proper ubuntu key then.
> > > Maybe we just need to install
> > >
> > > https://packages.debian.org/buster/ubuntu-keyring
> >
> > Yes I did it in my initvm.
> >
> > >
> > >
> > >
> > > > apt can not support two separated options like this:
> > > > deb [trusted=yes] [arch=armhf] http://...
> > > > It need to be written like this:
> > > > deb [trusted=yes arch=armhf] http://...
> > > >
> > > > >
> > > > >
> > > > > >
> > > > > > It also adds the architecture type in the sources.list
> > > > > > file to avoid issues
> > > > > > with apt-get update.
> > > > > >
> > > > > > Signed-off-by: Kory Maincent <kory.maincent at bootlin.com>
> > > > > > ---
> > > > > > elbepack/elbexml.py | 40 ++++++++++++++++++++++++++++++----------
> > > > > > elbepack/rfs.py | 23 +++++++++++++----------
> > > > > > schema/dbsfed.xsd | 7 +++++++
> > > > > > 3 files changed, 50 insertions(+), 20 deletions(-)
> > > > > >
> > > > > > diff --git a/elbepack/elbexml.py b/elbepack/elbexml.py
> > > > > > index fa8158392..ece069706 100644
> > > > > > --- a/elbepack/elbexml.py
> > > > > > +++ b/elbepack/elbexml.py
> > > > > > @@ -132,6 +132,13 @@ class ElbeXML(object):
> > > > > > return mirror.replace("LOCALMACHINE", "10.0.2.2")
> > > > > > + def get_host_mirror(self):
> > > > > > + if self.prj.has("mirror/host"):
> > > > > > + m = self.prj.node("mirror")
> > > > > > + mirror = m.text("host").strip() + "\n"
> > > > > > +
> > > > > > + return mirror
> > > > >
> > > > > errm... this fails when <host> does not exist.
> > > > > lets integrate this in get_primary_mirror()
> > > > > and have a proper fallback.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > +
> > > > > > def get_primary_mirror(self, cdrompath, initvm=True):
> > > > > > if self.prj.has("mirror/primary_host"):
> > > > > > m = self.prj.node("mirror")
> > > > > > @@ -146,7 +153,7 @@ class ElbeXML(object):
> > > > > > return replace_localmachine(mirror, initvm)
> > > > > > # XXX: maybe add cdrom path param ?
> > > > > > - def create_apt_sources_list(self,
> > > > > > build_sources=False, initvm=True):
> > > > > > + def create_apt_sources_list(self,
> > > > > > build_sources=False, initvm=True, arch="default"):
> > > > > > if self.prj is None:
> > > > > > return "# No Project"
> > > > > > @@ -155,24 +162,37 @@ class ElbeXML(object):
> > > > > > noauth = ""
> > > > > > if self.prj.has("noauth"):
> > > > > > - noauth = "[trusted=yes] "
> > > > > > + noauth = "trusted=yes "
> > > > > > +
> > > > > > + if arch == "default":
> > > > > > + arch = self.text("project/buildimage/arch", key="arch")
> > > > > > +
> > > > > > + arch_prj = self.text("project/buildimage/arch", key="arch")
> > > > > > mirror = ""
> > > > > > + option = "[" + noauth + " arch=" + arch +"] "
> > > > > > if self.prj.has("mirror/primary_host"):
> > > > > > - mirror += "deb " + noauth + self.get_primary_mirror(None)
> > > > > > - mirror += " " + self.prj.text("suite") + " main\n"
> > > > > > + if (arch != arch_prj) and self.prj.has("mirror/host"):
> > > > > > + mirror += "deb " + option + self.get_host_mirror()
> > > > > > + else:
> > > > > > + mirror += "deb " + option +
> > > > > > self.get_primary_mirror(None)
> > > > > > + mirror += " " + self.prj.text("suite") + " main\n"
> > > > > > +
> > > > >
> > > > > i find this logic too complex.
> > > > > We really want the host and target suite to be identical, or we would
> > > > > end up with different gcc versions.
> > > > >
> > > > > I also think its too complex to decide, whether we are building a host
> > > > > sysroot, based on architectures.
> > > > >
> > > > > Please add a bool parameter "hostsdk" to
> > > > > get_primary_mirror() which will
> > > > > make it look for <host> and fallback in the case its not there.
> > > > >
> > > > > it just needs to be passed down from build_host_sysroot() via
> > > > > BuildEnv.__init__() and BuildEnv.debootstrap()
> > > > >
> > > > > this will simplify this patch a lot.
> > > >
> > > >
> > > > Ok.
> > > >
> > > >
> > > >
> > > > --
> > > > Köry Maincent, Bootlin
> > > > Embedded Linux and kernel engineering
> > > > https://bootlin.com
> > >
> >
>
> --
> Köry Maincent, Bootlin
> Embedded Linux and kernel engineering
> https://bootlin.com
>
> _______________________________________________
> elbe-devel mailing list
> elbe-devel at linutronix.de
> https://lists.linutronix.de/mailman/listinfo/elbe-devel
--
Torben Hohn
Linutronix GmbH | Bahnhofstrasse 3 | D-88690 Uhldingen-Mühlhofen
Phone: +49 7556 25 999 18; Fax.: +49 7556 25 999 99
Hinweise zum Datenschutz finden Sie hier (Informations on data privacy
can be found here): https://linutronix.de/kontakt/Datenschutz.php
Linutronix GmbH | Firmensitz (Registered Office): Uhldingen-Mühlhofen |
Registergericht (Registration Court): Amtsgericht Freiburg i.Br., HRB700
806 | Geschäftsführer (Managing Directors): Heinz Egger, Thomas Gleixner
More information about the elbe-devel
mailing list