[elbe-devel] [PATCH v2 1/3] init: add <max-cpus> field and make Makefile.mako obey the <mem> field

[Manuel Traut]

On Thu, Nov 22, 2018 at 10:53:05AM +0100, Torben Hohn wrote:
> On machines with many cores, 1GiB of memory is not very much. Especially,
> when g++ is used in a pbuilder with -jauto.
> 
> Add <max-cpus> field, that clamps the number of cpus for the initvm.
> Make it default to 8. Also let Makefile.mako clamp the SMP default
> to the max-cpus value.
> 
> Signed-off-by: Torben Hohn <torben.hohn at linutronix.de>

Reviewed-by: Manuel Traut <manut at linutronix.de>


> ---
>  elbepack/init/Makefile.mako    | 12 ++++++++++--
>  elbepack/init/libvirt.xml.mako |  3 ++-
>  elbepack/xmldefaults.py        |  3 ++-
>  schema/dbsfed.xsd              | 11 +++++++++++
>  4 files changed, 25 insertions(+), 4 deletions(-)
> 
> diff --git a/elbepack/init/Makefile.mako b/elbepack/init/Makefile.mako
> index 4c2f323b..8e98176e 100644
> --- a/elbepack/init/Makefile.mako
> +++ b/elbepack/init/Makefile.mako
> @@ -5,8 +5,16 @@
>  ##
>  ## SPDX-License-Identifier: GPL-3.0-or-later
>  ##
> -MEMSIZE?=1024
> -SMP?=`nproc`
> +<%
> +import multiprocessing
> +from elbepack.filesystem import size_to_int
> +
> +max_cpus = int(prj.text('max-cpus', default=defs, key='max-cpus'))
> +memory = size_to_int(prj.text('mem', default=defs, key='mem')) / 1024 / 1024
> +%>
> +
> +MEMSIZE?=${memory}
> +SMP?=$$((`nproc` > ${max_cpus} ? ${max_cpus} : `nproc`))
>  INTERPRETER?=${prj.text('interpreter', default=defs, key='interpreter')}
>  
>  # this is a workaround for
> diff --git a/elbepack/init/libvirt.xml.mako b/elbepack/init/libvirt.xml.mako
> index c2b3d077..1260c55f 100644
> --- a/elbepack/init/libvirt.xml.mako
> +++ b/elbepack/init/libvirt.xml.mako
> @@ -19,7 +19,8 @@ from elbepack.filesystem import size_to_int
>  uid = uuid.uuid4()
>  
>  name = cfg['initvm_domain']
> -cpus = multiprocessing.cpu_count()
> +cpus = int(prj.text('max-cpus', default=defs, key='max-cpus'))
> +cpus = min(multiprocessing.cpu_count(), cpus)
>  memory = size_to_int(prj.text('mem', default=defs, key='mem')) / 1024
>  
>  imagetype = prj.text('img', default=defs, key='img')
> diff --git a/elbepack/xmldefaults.py b/elbepack/xmldefaults.py
> index 3a9e8684..dbc93db1 100644
> --- a/elbepack/xmldefaults.py
> +++ b/elbepack/xmldefaults.py
> @@ -142,7 +142,8 @@ archindep_defaults = {
>      "size": "20G",
>      "img": "qcow2",
>      "mem": "1GiB",
> -    "swap-size": "0"
> +    "swap-size": "0",
> +    "max-cpus": "8"
>  }
>  
>  defaults = {"armel": armel_defaults,
> diff --git a/schema/dbsfed.xsd b/schema/dbsfed.xsd
> index 17aa6283..d52b80f6 100644
> --- a/schema/dbsfed.xsd
> +++ b/schema/dbsfed.xsd
> @@ -358,6 +358,17 @@
>            </documentation>
>          </annotation>
>        </element>
> +      <element name="max-cpus" type="integer" minOccurs="0" maxOccurs="1">
> +        <annotation>
> +          <documentation>
> +	    The number of cpus used by the initvm is clamped to this value.
> +	    This shall protect the initvm from running out of memory, when
> +	    running on machines with large numbers of cores.
> +	    When increasing this value, be sure to also increase mem and
> +	    maybe add some swap.
> +          </documentation>
> +        </annotation>
> +      </element>
>        <element name="img" type="rfs:string" minOccurs="0" maxOccurs="1">
>          <annotation>
>            <documentation>
> -- 
> 2.11.0
> 
> 
> _______________________________________________
> elbe-devel mailing list
> elbe-devel at linutronix.de
> https://lists.linutronix.de/mailman/listinfo/elbe-devel