[elbe-devel] [PATCH 1/2] Add optional mirror for host architecture
Köry Maincent
kory.maincent at bootlin.com
Fri Jun 5 09:41:03 CEST 2020
Hello Torben,
On 04/06/2020 18:13, Torben Hohn wrote:
> On Thu, Jun 04, 2020 at 10:54:47AM +0200, Köry Maincent wrote:
>> Hello Torben,
>>
>> Thanks for the review.
>
> No Problem, sorry for being so slow, but this pylint patchqueue creates
> conflicts all over the place and needs to go in quickly.
No problem, pylint seems an important patch to add, good luck with it.
>
>>
>>
>> On 04/06/2020 09:14, Torben Hohn wrote:
>>> On Wed, May 20, 2020 at 11:53:03AM +0200, Kory Maincent wrote:
>>>> If the primary mirror does not contain the packages for both target and host
>>>> architecture the build_host_sysroot process fails.
>>>>
>>>> This patch adds support for describing in the XML configuration an additional
>>>> optional mirror containing packages for the host architecture, like:
>>>> <mirror>
>>>> <host>http://archive.ubuntu.com/ubuntu bionic main universe</host>
>>>> </mirror>
>>>
>>> Do we have to add "bionic main universe" here ?
>>> The primary mirrors are only needed for debootstrap.
>>>
>>> The other mirror entries support [arch=amd64] and we should be able to
>>> have all other mirror urls in the urllist without having problems.
>>> If a mirror only has amd64 packages, it needs to be marked that way.
>>
>> If we add [arch=amd64] in the mirror urls of the urllist there will be a
>> conflict with noauth.
>
> Then we have to fix noauth.
> BTW you could also not use noauth, and add trusted=yes directly into the
> mirror url. That way it wouldnt stop checking all signatures, and only
> check the relevant signatures.
Yes, we could indeed remove the noauth from Elbe code.
>
> Is noauth reuired to build ubuntu ?
Elbe seemed to not have trouble to generate an Ubuntu image without
noauth, but when I want to work with pbuilder I got an issue with the
keyring:
W: GPG error:
http://127.0.0.1:8080/var/cache/elbe/c26b1694-ff47-4efc-9346-13b3b0811c47/repo
bionic InRelease: The following signatures couldn't be verified because
the public key is not available: NO_PUBKEY 4A4B63D0DEC56F34
E: The repository
'http://127.0.0.1:8080/var/cache/elbe/c26b1694-ff47-4efc-9346-13b3b0811c47/repo
bionic InRelease' is not signed.
W: Aborting with an error
Strangely it is related to the local repository but with noauth the
issue disappear.
> We should make it possible to install the proper ubuntu key then.
> Maybe we just need to install
>
> https://packages.debian.org/buster/ubuntu-keyring
Yes I did it in my initvm.
>
>
>
>> apt can not support two separated options like this:
>> deb [trusted=yes] [arch=armhf] http://...
>> It need to be written like this:
>> deb [trusted=yes arch=armhf] http://...
>>
>>>
>>>
>>>>
>>>> It also adds the architecture type in the sources.list file to avoid issues
>>>> with apt-get update.
>>>>
>>>> Signed-off-by: Kory Maincent <kory.maincent at bootlin.com>
>>>> ---
>>>> elbepack/elbexml.py | 40 ++++++++++++++++++++++++++++++----------
>>>> elbepack/rfs.py | 23 +++++++++++++----------
>>>> schema/dbsfed.xsd | 7 +++++++
>>>> 3 files changed, 50 insertions(+), 20 deletions(-)
>>>>
>>>> diff --git a/elbepack/elbexml.py b/elbepack/elbexml.py
>>>> index fa8158392..ece069706 100644
>>>> --- a/elbepack/elbexml.py
>>>> +++ b/elbepack/elbexml.py
>>>> @@ -132,6 +132,13 @@ class ElbeXML(object):
>>>> return mirror.replace("LOCALMACHINE", "10.0.2.2")
>>>> + def get_host_mirror(self):
>>>> + if self.prj.has("mirror/host"):
>>>> + m = self.prj.node("mirror")
>>>> + mirror = m.text("host").strip() + "\n"
>>>> +
>>>> + return mirror
>>>
>>> errm... this fails when <host> does not exist.
>>> lets integrate this in get_primary_mirror()
>>> and have a proper fallback.
>>>
>>>
>>>
>>>
>>>> +
>>>> def get_primary_mirror(self, cdrompath, initvm=True):
>>>> if self.prj.has("mirror/primary_host"):
>>>> m = self.prj.node("mirror")
>>>> @@ -146,7 +153,7 @@ class ElbeXML(object):
>>>> return replace_localmachine(mirror, initvm)
>>>> # XXX: maybe add cdrom path param ?
>>>> - def create_apt_sources_list(self, build_sources=False, initvm=True):
>>>> + def create_apt_sources_list(self, build_sources=False, initvm=True, arch="default"):
>>>> if self.prj is None:
>>>> return "# No Project"
>>>> @@ -155,24 +162,37 @@ class ElbeXML(object):
>>>> noauth = ""
>>>> if self.prj.has("noauth"):
>>>> - noauth = "[trusted=yes] "
>>>> + noauth = "trusted=yes "
>>>> +
>>>> + if arch == "default":
>>>> + arch = self.text("project/buildimage/arch", key="arch")
>>>> +
>>>> + arch_prj = self.text("project/buildimage/arch", key="arch")
>>>> mirror = ""
>>>> + option = "[" + noauth + " arch=" + arch +"] "
>>>> if self.prj.has("mirror/primary_host"):
>>>> - mirror += "deb " + noauth + self.get_primary_mirror(None)
>>>> - mirror += " " + self.prj.text("suite") + " main\n"
>>>> + if (arch != arch_prj) and self.prj.has("mirror/host"):
>>>> + mirror += "deb " + option + self.get_host_mirror()
>>>> + else:
>>>> + mirror += "deb " + option + self.get_primary_mirror(None)
>>>> + mirror += " " + self.prj.text("suite") + " main\n"
>>>> +
>>>
>>> i find this logic too complex.
>>> We really want the host and target suite to be identical, or we would
>>> end up with different gcc versions.
>>>
>>> I also think its too complex to decide, whether we are building a host
>>> sysroot, based on architectures.
>>>
>>> Please add a bool parameter "hostsdk" to get_primary_mirror() which will
>>> make it look for <host> and fallback in the case its not there.
>>>
>>> it just needs to be passed down from build_host_sysroot() via
>>> BuildEnv.__init__() and BuildEnv.debootstrap()
>>>
>>> this will simplify this patch a lot.
>>
>>
>> Ok.
>>
>>
>>
>> --
>> Köry Maincent, Bootlin
>> Embedded Linux and kernel engineering
>> https://bootlin.com
>
--
Köry Maincent, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
More information about the elbe-devel
mailing list