[elbe-devel] [PATCH v2 1/3] init: add <max-cpus> field and make Makefile.mako obey the <mem> field

[Torben Hohn]

On Fri, Oct 19, 2018 at 04:26:35PM +0200, Manuel Traut wrote:
> On Fri, Sep 28, 2018 at 10:34:18AM +0200, Torben Hohn wrote:
> > On machines with many cores, 1GiB of memory is not very much. Especially,
> > when g++ is used in a pbuilder with -jauto.
> > 
> > Add <max-cpus> field, that clamps the number of cpus for the initvm.
> > Make it default to 8. Also let the Makefile.mako obey the memory settings.
> > 
> > Signed-off-by: Torben Hohn <torben.hohn at linutronix.de>
> > ---
> >  elbepack/init/Makefile.mako    | 13 +++++++++++--
> >  elbepack/init/libvirt.xml.mako |  3 ++-
> >  elbepack/xmldefaults.py        |  3 ++-
> >  schema/dbsfed.xsd              | 11 +++++++++++
> >  4 files changed, 26 insertions(+), 4 deletions(-)
> > 
> > diff --git a/elbepack/init/Makefile.mako b/elbepack/init/Makefile.mako
> > index 4c2f323b..6e55056b 100644
> > --- a/elbepack/init/Makefile.mako
> > +++ b/elbepack/init/Makefile.mako
> > @@ -5,8 +5,17 @@
> >  ##
> >  ## SPDX-License-Identifier: GPL-3.0-or-later
> >  ##
> > -MEMSIZE?=1024
> > -SMP?=`nproc`
> > +<%
> > +import multiprocessing
> > +from elbepack.filesystem import size_to_int
> > +
> > +cpus = int(prj.text('max-cpus', default=defs, key='max-cpus'))
> > +cpus = min(multiprocessing.cpu_count(), cpus)
> > +memory = size_to_int(prj.text('mem', default=defs, key='mem')) / 1024 / 1024
> > +%>
> > +
> > +MEMSIZE?=${memory}
> > +SMP?=${cpus}
> >  INTERPRETER?=${prj.text('interpreter', default=defs, key='interpreter')}
> 
> A big adavantage of the Makefile to control the initvm is that it can be
> moved easily around and that we can detect the runtime environment during
> call-time. This is lost now :(

During normal operation, the only usage of this Makefile is to install
the initvm.

This does not have to move around.

Normal starting of the initvm is done via libvirt.
The number of CPUs might be increased using virt-manager.

> 
> I'd like to have it implemented directly in the Makefile
> 
> The rest of the patch looks OK.
> 
> >  # this is a workaround for
> > diff --git a/elbepack/init/libvirt.xml.mako b/elbepack/init/libvirt.xml.mako
> > index c2b3d077..1260c55f 100644
> > --- a/elbepack/init/libvirt.xml.mako
> > +++ b/elbepack/init/libvirt.xml.mako
> > @@ -19,7 +19,8 @@ from elbepack.filesystem import size_to_int
> >  uid = uuid.uuid4()
> >  
> >  name = cfg['initvm_domain']
> > -cpus = multiprocessing.cpu_count()
> > +cpus = int(prj.text('max-cpus', default=defs, key='max-cpus'))
> > +cpus = min(multiprocessing.cpu_count(), cpus)
> >  memory = size_to_int(prj.text('mem', default=defs, key='mem')) / 1024
> >  
> >  imagetype = prj.text('img', default=defs, key='img')
> > diff --git a/elbepack/xmldefaults.py b/elbepack/xmldefaults.py
> > index 3a9e8684..dbc93db1 100644
> > --- a/elbepack/xmldefaults.py
> > +++ b/elbepack/xmldefaults.py
> > @@ -142,7 +142,8 @@ archindep_defaults = {
> >      "size": "20G",
> >      "img": "qcow2",
> >      "mem": "1GiB",
> > -    "swap-size": "0"
> > +    "swap-size": "0",
> > +    "max-cpus": "8"
> >  }
> >  
> >  defaults = {"armel": armel_defaults,
> > diff --git a/schema/dbsfed.xsd b/schema/dbsfed.xsd
> > index eaf707f1..731439db 100644
> > --- a/schema/dbsfed.xsd
> > +++ b/schema/dbsfed.xsd
> > @@ -358,6 +358,17 @@
> >            </documentation>
> >          </annotation>
> >        </element>
> > +      <element name="max-cpus" type="integer" minOccurs="0" maxOccurs="1">
> > +        <annotation>
> > +          <documentation>
> > +	    The number of cpus used by the initvm is clamped to this value.
> > +	    This shall protect the initvm from running out of memory, when
> > +	    running on machines with large numbers of cores.
> > +	    When increasing this value, be sure to also increase mem and
> > +	    maybe add some swap.
> > +          </documentation>
> > +        </annotation>
> > +      </element>
> >        <element name="img" type="rfs:string" minOccurs="0" maxOccurs="1">
> >          <annotation>
> >            <documentation>
> > -- 
> > 2.11.0
> > 
> > 
> > _______________________________________________
> > elbe-devel mailing list
> > elbe-devel at linutronix.de
> > https://lists.linutronix.de/mailman/listinfo/elbe-devel

-- 
Torben Hohn
Linutronix GmbH | Bahnhofstrasse 3 | D-88690 Uhldingen-Mühlhofen
Phone: +49 7556 25 999 18; Fax.: +49 7556 25 999 99

Hinweise zum Datenschutz finden Sie hier (Informations on data privacy 
can be found here): https://linutronix.de/kontakt/Datenschutz.php

Linutronix GmbH | Firmensitz (Registered Office): Uhldingen-Mühlhofen | 
Registergericht (Registration Court): Amtsgericht Freiburg i.Br., HRB700 
806 | Geschäftsführer (Managing Directors): Heinz Egger, Thomas Gleixner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.linutronix.de/pipermail/elbe-devel/attachments/20181112/7b3ef173/attachment.sig>