[elbe-devel] [PATCH 2/5] rpcaptcache: Make download_source better

[Torben Hohn]

On Tue, May 19, 2020 at 10:27:00PM -0400, Olivier Dion wrote:
> The method used to download the source package from a binary package
> description.  This is changed in order to be able to download source
> packages that are not reference by the binary packages cache.
> 
> A reverse lookup has been tried before without much success and
> python-apt lacks of any such binding.  Thus, will do what fetch_source
> used to do, but without passing from any binary packages.
> 
> Signed-off-by: Olivier Dion <dion at linutronix.de>

Reviewed-by: Torben Hohn <torben.hohn at linutronix.de>

> ---
>  elbepack/rpcaptcache.py | 67 ++++++++++++++++++++++++++++++++++-------
>  1 file changed, 56 insertions(+), 11 deletions(-)
> 
> diff --git a/elbepack/rpcaptcache.py b/elbepack/rpcaptcache.py
> index 14c3f654..847b41cd 100644
> --- a/elbepack/rpcaptcache.py
> +++ b/elbepack/rpcaptcache.py
> @@ -16,8 +16,9 @@ import time
>  from multiprocessing.util import Finalize
>  from multiprocessing.managers import BaseManager
>  
> -from apt_pkg import config, version_compare, TagFile
> +from apt_pkg import config, version_compare, TagFile, SourceRecords, Acquire, AcquireFile
>  from apt import Cache
> +from apt.package import FetchError
>  
>  from elbepack.aptprogress import (ElbeAcquireProgress, ElbeInstallProgress,
>                                    ElbeOpProgress)
> @@ -333,17 +334,61 @@ class RPCAPTCache(InChRootObject):
>                                      ElbeAcquireProgress())
>          return self.rfs.fname(rel_filename)
>  
> -    def download_source(self, pkgname, path, version=None):
> -        p = self.cache[pkgname]
> -        if version is None:
> -            pkgver = p.installed
> -        else:
> -            pkgver = p.versions[version]
> +    def download_source(self, src_name, src_version, dest_dir):
> +
> +        allow_untrusted = config.find_b("APT::Get::AllowUnauthenticated",
> +                                        False)
> +
> +        rec = SourceRecords()
> +        acq = Acquire(ElbeAcquireProgress())
> +
> +        # poorman's iterator
> +        while True:
> +            next_p = rec.lookup(src_name)
> +            # End of the list?
> +            if not next_p:
> +                raise ("No source found for %s_%s" % (src_name, src_version))
> +            if src_version == rec.version:
> +                break
> +
> +        # We don't allow untrusted package and the package is not
> +        # marks as trusted
> +        if not (allow_untrusted or rec.index.is_trusted):
> +            raise FetchError("Can't fetch source %s_%s; Source %r is not trusted" %
> +                             (src_name, src_version, rec.index.describe))
> +
> +        # Copy from src to dst all files of the source package
> +        dsc = None
> +        files = []
> +        for _file in rec.files:
> +            src = os.path.basename(_file.path)
> +            dst = os.path.join(dest_dir, src)
> +
> +            if 'dsc' ==  _file.type:
> +                dsc = dst
> +
> +            if not (allow_untrusted or _file.hashes.usable):
> +                raise FetchError("Can't fetch file %s. No trusted hash found." % dst)
> +
> +            # acq is accumlating the AcquireFile, the files list only
> +            # exists to prevent Python from GC the object .. I guess.
> +            # Anyway, if we don't keep the list, We will get an empty
> +            # directory
> +            files.append(AcquireFile(acq, rec.index.archive_uri(_file.path),
> +                                     _file.hashes, _file.size, src, destfile=dst))
> +        acq.run()
> +
> +        if dsc is None:
> +            raise ValueError("No source found for %s_%s" %
> +                             (src_name, src_version))
> +
> +        for item in acq.items:
> +            if item.STAT_DONE != item.status:
> +                raise FetchError("Can't fetch item %s: %s" %
> +                                  (item.destfile, item.error_text))
> +
> +        return self.rfs.fname(os.path.abspath(dsc))
>  
> -        rel_filename = pkgver.fetch_source(path,
> -                                           ElbeAcquireProgress(),
> -                                           unpack=False)
> -        return self.rfs.fname(rel_filename)
>  
>  def get_rpcaptcache(rfs, arch,
>                      notifier=None, norecommend=False, noauth=True):
> -- 
> 2.26.2
> 
> 
> _______________________________________________
> elbe-devel mailing list
> elbe-devel at linutronix.de
> https://lists.linutronix.de/mailman/listinfo/elbe-devel

-- 
Torben Hohn
Linutronix GmbH | Bahnhofstrasse 3 | D-88690 Uhldingen-Mühlhofen
Phone: +49 7556 25 999 18; Fax.: +49 7556 25 999 99

Hinweise zum Datenschutz finden Sie hier (Informations on data privacy 
can be found here): https://linutronix.de/kontakt/Datenschutz.php

Linutronix GmbH | Firmensitz (Registered Office): Uhldingen-Mühlhofen | 
Registergericht (Registration Court): Amtsgericht Freiburg i.Br., HRB700 
806 | Geschäftsführer (Managing Directors): Heinz Egger, Thomas Gleixner