[elbe-devel] [PATCH 1/4] install_elbe_version: implement "elbe control install_elbe_version"
Torben Hohn
torben.hohn at linutronix.de
Thu Nov 22 16:06:27 CET 2018
On Thu, Nov 22, 2018 at 03:53:16PM +0100, Manuel Traut wrote:
> On Thu, Nov 22, 2018 at 01:14:05PM +0100, Torben Hohn wrote:
> > On Fri, Nov 16, 2018 at 10:04:09AM +0100, Manuel Traut wrote:
> > > On Mon, Nov 12, 2018 at 09:23:11AM +0100, Torben Hohn wrote:
> > > > On Wed, Nov 07, 2018 at 10:03:01AM +0100, Manuel Traut wrote:
> > > > > On Mon, Oct 01, 2018 at 04:37:45PM +0200, Torben Hohn wrote:
> > > > > > when elbe on the hostmachine is upgraded, it usually can not talk
> > > > > > to the elbe version installed in the initvm.
> > > > > >
> > > > > > add a command that can install a specific elbe version in the initvm.
> > > > > >
> > > > > > since the elbe daemon will restart, we dont have a return code on the soap
> > > > > > interface. Maybe we want to sleep one second and check, whether
> > > > > > elbe daemon replies with the correct version, in
> > > > > > InstallElbeVersion(ClientAction).
> > > >
> > > > Can you please make a statement regarding this pragraph ?
> > >
> > > I didn't reply because i don't have a better solution. So go on with this.
> > >
> > > > How shall we handle the missing return code on the soap
> > > > intrerface ?
> > >
> > > Ignore it.
> > >
> > > > What shall we do with downgrades ?
> > >
> > > What's the problem with downgrades? Don't they work the same way?
> >
> > if the version we downgrade to, consists of less pkgs than
> > the current version, there will be a problem with the package that
> > we dont know about.
> >
> > Not sure... mayb it works.
>
> Should also work if we read the package list from the version.py of the
> soapclient.
>
> However this might become interesting if different clients with different
> versions are used. To we need some locking? Not changing elbe installation
> as long as a project is built? Or anotherone is changing the version?
the locking is basically happening in apt.
Also... the restart happens through cherrypy noticing the new files.
This restart waits for async jobs to finish.
So i believe this works currently, until concurrent async workers
are implemented.
>
> > > > Shall we pass the list of required packages into the function call ?
> > > > I mean we only want to sync the outside version into the initvm.
> > > > The outside version can know the package list.
> > >
> > > You could ckeck if aptitude is able to resolve the dependencies automatically
> > > (and correctly). If not, this seems to be an acceptable approach to me.
> >
> > aptitude can do it, but its the 3rd solution.
> > I dont know, how i can pick solutions via commandline. Dont think thats possible.
>
> no.. let's do it with a packagelist.
>
> > ----------------------------------------------------------------------------
> > root at elbe-daemon:~# aptitude install elbe-daemon=2.9.13+deb8+build371
> > The following packages will be upgraded:
> > elbe-daemon{b}
> > 1 packages upgraded, 0 newly installed, 0 to remove and 5 not upgraded.
> > Need to get 0 B/20.3 kB of archives. After unpacking 0 B will be used.
> > The following packages have unmet dependencies:
> > elbe-daemon : Depends: python-elbe-common (= 2.9.13+deb8+build371) but 2.9.13+deb8+build361 is installed and it is kept back.
> > Depends: python-elbe-buildenv (= 2.9.13+deb8+build371) but 2.9.13+deb8+build361 is installed and it is kept back.
> > elbe-soap : Depends: elbe-daemon (= 2.9.13+deb8+build361) but 2.9.13+deb8+build371 is to be installed.
> > open: 802; closed: 71; defer: 735; conflict: 719 .The following actions will resolve these dependencies:
> >
> > Remove the following packages:
> > 1) elbe-daemon
> > 2) elbe-soap
> >
> >
> >
> > Accept this solution? [Y/n/q/?] n
> > The following actions will resolve these dependencies:
> >
> > Keep the following packages at their current version:
> > 1) elbe-daemon [2.9.13+deb8+build361 (jessie, now)]
> >
> >
> >
> > Accept this solution? [Y/n/q/?] n
> > The following actions will resolve these dependencies:
> >
> > Upgrade the following packages:
> > 1) elbe-schema [2.9.13+deb8+build361 (jessie, now) -> 2.9.13+deb8+build371 (jessie)]
> > 2) elbe-soap [2.9.13+deb8+build361 (jessie, now) -> 2.9.13+deb8+build371 (jessie)]
> > 3) python-elbe-bin [2.9.13+deb8+build361 (jessie, now) -> 2.9.13+deb8+build371 (jessie)]
> > 4) python-elbe-buildenv [2.9.13+deb8+build361 (jessie, now) -> 2.9.13+deb8+build371 (jessie)]
> > 5) python-elbe-common [2.9.13+deb8+build361 (jessie, now) -> 2.9.13+deb8+build371 (jessie)]
> >
> >
> >
> > Accept this solution? [Y/n/q/?]
> > ----------------------------------------------------------------------------
> >
> > >
> > > > > updating docs/elbe-control.txt is missing
> > > >
> > > > I will add that in V2 when we decide, that we want to use this
> > > > approach.
> > > >
> > > > I believe, that the proxy settings of the initvm will be used.
> > > > IIRC the settings from the debian installer will end up in
> > > > /etc/apt/config.d/ somewhere.
> > >
> > > OK.
> > >
> > > > >
> > > > > > Another weakness of this approach, is that we have to specify all
> > > > > > elbe related packages, or apt will refuse to install:
> > > > > >
> > > > > > --------------------------------------------------------------------------
> > > > > > The following packages have unmet dependencies:
> > > > > > elbe-soap : Depends: elbe-daemon (= 2.9.11~bpo8+build251) but it is not going to be installed
> > > > > > python-elbe-common : Depends: elbe-schema (= 2.9.11~bpo8+build251) but 2.9.12+deb8+build311 is to be installed
> > > > > > Depends: python-elbe-bin (= 2.9.11~bpo8+build251) but it is not going to be installed
> > > > > > E: Unable to correct problems, you have held broken packages.
> > > > > > --------------------------------------------------------------------------
> > > > > >
> > > > > > this is probably more of a problem, when a downgrade should be done.
> > > > > >
> > > > > > Signed-off-by: Torben Hohn <torben.hohn at linutronix.de>
> > > > > > ---
> > > > > > elbepack/asyncworker.py | 31 ++++++++++++++++++++++++++++++-
> > > > > > elbepack/daemons/soap/esoap.py | 6 ++++++
> > > > > > elbepack/projectmanager.py | 7 ++++++-
> > > > > > elbepack/soapclient.py | 19 +++++++++++++++++++
> > > > > > 4 files changed, 61 insertions(+), 2 deletions(-)
> > > > > >
> > > > > > diff --git a/elbepack/asyncworker.py b/elbepack/asyncworker.py
> > > > > > index 7f6173ba..c8826ffa 100644
> > > > > > --- a/elbepack/asyncworker.py
> > > > > > +++ b/elbepack/asyncworker.py
> > > > > > @@ -7,7 +7,7 @@
> > > > > >
> > > > > > from threading import Thread
> > > > > > from Queue import Queue
> > > > > > -from os import path, getcwd, chdir
> > > > > > +from os import path, getcwd, chdir, system
> > > > > > from contextlib import contextmanager
> > > > > > from urllib import quote
> > > > > > import traceback
> > > > > > @@ -18,6 +18,7 @@ from elbepack.updatepkg import gen_update_pkg
> > > > > > from elbepack.pkgarchive import gen_binpkg_archive, checkout_binpkg_archive
> > > > > > from elbepack.rfs import DebootstrapException
> > > > > > from elbepack.elbeproject import AptCacheCommitError, AptCacheUpdateError
> > > > > > +from elbepack.filesystem import hostfs
> > > > > >
> > > > > >
> > > > > > class AsyncWorkerJob(object):
> > > > > > @@ -375,6 +376,34 @@ class APTCommitJob(AsyncWorkerJob):
> > > > > > db.reset_busy(self.project.builddir,
> > > > > > "build_failed")
> > > > > >
> > > > > > +class InstallElbeVersionJob(AsyncWorkerJob):
> > > > > > + def __init__(self, version):
> > > > > > + AsyncWorkerJob.__init__(self, None)
> > > > > > + self.version = version
> > > > > > +
> > > > > > + def enqueue(self, queue, db):
> > > > > > + AsyncWorkerJob.enqueue(self, queue, db)
> > > > > > +
> > > > > > + def execute(self, db):
> > > > > > + pkgs = ['python-elbe-buildenv',
> > > > > > + 'elbe-soap',
> > > > > > + 'python-elbe-common',
> > > > > > + 'elbe-daemon',
> > > > > > + 'elbe-schema',
> > > > > > + 'python-elbe-bin']
> > > > > > +
> > > > > > + pkgs = ['"%s=%s*"' % (p, self.version) for p in pkgs]
> > > > > > +
> > > > > > + # Prevent, that elbe daemon is restarted by the
> > > > > > + # prerm/postinst scripts.
> > > > > > + # elbe daemon does it itself, because cherrypy
> > > > > > + # notices that.
> > > > > > + hostfs.write_file("usr/sbin/policy-rc.d",
> > > > > > + 0o755, "#!/bin/sh\nexit 101\n")
> > > > > > +
> > > > > > + system('apt-get install -y --force-yes %s' % ' '.join(pkgs))
> > > > >
> > > > > what about users needing a proxy?
> > > > >
> > > > > > + hostfs.remove('usr/sbin/policy-rc.d')
> > > > > >
> > > > > > class GenUpdateJob(AsyncWorkerJob):
> > > > > > def __init__(self, project, base_version):
> > > > > > diff --git a/elbepack/daemons/soap/esoap.py b/elbepack/daemons/soap/esoap.py
> > > > > > index b973b592..1c62fa87 100644
> > > > > > --- a/elbepack/daemons/soap/esoap.py
> > > > > > +++ b/elbepack/daemons/soap/esoap.py
> > > > > > @@ -64,6 +64,12 @@ class ESoap (ServiceBase):
> > > > > > def list_users(self):
> > > > > > return [u.name for u in self.app.pm.db.list_users()]
> > > > > >
> > > > > > + @rpc(String)
> > > > > > + @soap_faults
> > > > > > + @authenticated_admin
> > > > > > + def install_elbe_version(self, version):
> > > > > > + self.app.pm.install_elbe_version(version)
> > > > > > +
> > > > > > @rpc(String,String,String,String,Boolean)
> > > > > > @soap_faults
> > > > > > @authenticated_admin
> > > > > > diff --git a/elbepack/projectmanager.py b/elbepack/projectmanager.py
> > > > > > index e8613bca..3d2c018e 100644
> > > > > > --- a/elbepack/projectmanager.py
> > > > > > +++ b/elbepack/projectmanager.py
> > > > > > @@ -22,7 +22,8 @@ from elbepack.asyncworker import (AsyncWorker, BuildJob, APTUpdateJob,
> > > > > > APTUpdUpgrJob, BuildSysrootJob,
> > > > > > PdebuildJob, CreatePbuilderJob,
> > > > > > UpdatePbuilderJob, BuildChrootTarJob,
> > > > > > - BuildSDKJob, BuildCDROMsJob)
> > > > > > + BuildSDKJob, BuildCDROMsJob,
> > > > > > + InstallElbeVersionJob)
> > > > > >
> > > > > > from elbepack.elbexml import ValidationMode
> > > > > >
> > > > > > @@ -537,6 +538,10 @@ class ProjectManager(object):
> > > > > > logline = str(part) + '###' + str(logline)
> > > > > > return self.db.is_busy(ep.builddir), logline
> > > > > >
> > > > > > + def install_elbe_version(self, version):
> > > > > > + with self.lock:
> > > > > > + self.worker.enqueue(InstallElbeVersionJob(version))
> > > > > > +
> > > > > > def _get_current_project(self, userid, allow_busy=True):
> > > > > > # Must be called with self.lock held
> > > > > > if userid not in self.userid2project:
> > > > > > diff --git a/elbepack/soapclient.py b/elbepack/soapclient.py
> > > > > > index c7d822d3..30efece0 100644
> > > > > > --- a/elbepack/soapclient.py
> > > > > > +++ b/elbepack/soapclient.py
> > > > > > @@ -791,6 +791,25 @@ class UpdatePbuilderAction(ClientAction):
> > > > > >
> > > > > > ClientAction.register(UpdatePbuilderAction)
> > > > > >
> > > > > > +class InstallElbeVersion(ClientAction):
> > > > > > +
> > > > > > + tag = 'install_elbe_version'
> > > > > > +
> > > > > > + def __init__(self, node):
> > > > > > + ClientAction.__init__(self, node)
> > > > > > +
> > > > > > + def execute(self, client, _opt, args):
> > > > > > + if len(args) != 1:
> > > > > > + print(
> > > > > > + "usage: elbe control install_elbe_version <version>",
> > > > > > + file=sys.stderr)
> > > > > > + sys.exit(20)
> > > > > > +
> > > > > > + version = args[0]
> > > > > > + client.service.install_elbe_version(version)
> > > > > > +
> > > > > > +
> > > > > > +ClientAction.register(InstallElbeVersion)
> > > > > >
> > > > > > class RepoAction(ClientAction):
> > > > > > repoactiondict = {}
> > > > > > --
> > > > > > 2.11.0
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > elbe-devel mailing list
> > > > > > elbe-devel at linutronix.de
> > > > > > https://lists.linutronix.de/mailman/listinfo/elbe-devel
> > > >
> > > > --
> > > > Torben Hohn
> > > > Linutronix GmbH | Bahnhofstrasse 3 | D-88690 Uhldingen-Mühlhofen
> > > > Phone: +49 7556 25 999 18; Fax.: +49 7556 25 999 99
> > > >
> > > > Hinweise zum Datenschutz finden Sie hier (Informations on data privacy
> > > > can be found here): https://linutronix.de/kontakt/Datenschutz.php
> > > >
> > > > Linutronix GmbH | Firmensitz (Registered Office): Uhldingen-Mühlhofen |
> > > > Registergericht (Registration Court): Amtsgericht Freiburg i.Br., HRB700
> > > > 806 | Geschäftsführer (Managing Directors): Heinz Egger, Thomas Gleixner
> > >
> > >
> >
> > --
> > Torben Hohn
> > Linutronix GmbH | Bahnhofstrasse 3 | D-88690 Uhldingen-Mühlhofen
> > Phone: +49 7556 25 999 18; Fax.: +49 7556 25 999 99
> >
> > Hinweise zum Datenschutz finden Sie hier (Informations on data privacy
> > can be found here): https://linutronix.de/kontakt/Datenschutz.php
> >
> > Linutronix GmbH | Firmensitz (Registered Office): Uhldingen-Mühlhofen |
> > Registergericht (Registration Court): Amtsgericht Freiburg i.Br., HRB700
> > 806 | Geschäftsführer (Managing Directors): Heinz Egger, Thomas Gleixner
>
>
--
Torben Hohn
Linutronix GmbH | Bahnhofstrasse 3 | D-88690 Uhldingen-Mühlhofen
Phone: +49 7556 25 999 18; Fax.: +49 7556 25 999 99
Hinweise zum Datenschutz finden Sie hier (Informations on data privacy
can be found here): https://linutronix.de/kontakt/Datenschutz.php
Linutronix GmbH | Firmensitz (Registered Office): Uhldingen-Mühlhofen |
Registergericht (Registration Court): Amtsgericht Freiburg i.Br., HRB700
806 | Geschäftsführer (Managing Directors): Heinz Egger, Thomas Gleixner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.linutronix.de/pipermail/elbe-devel/attachments/20181122/b77da57a/attachment-0001.sig>
More information about the elbe-devel
mailing list