[elbe-devel] [PATCH 4/4] soapclient: workaround socket exception

[Manuel Traut]

On Fri, Sep 21, 2018 at 08:59:45AM +0200, Torben Hohn wrote:
> On Thu, Sep 20, 2018 at 11:09:00AM +0200, Manuel Traut wrote:
> > On Thu, Sep 20, 2018 at 10:52:17AM +0200, Torben Hohn wrote:
> > > On Fri, Sep 14, 2018 at 01:53:05PM +0200, Manuel Traut wrote:
> > > > 17:32:00 2018-08-30 15:29:09 - INFO     - |     line = self.fp.readline(_MAXLINE + 1)
> > > > 17:32:00 2018-08-30 15:29:09 - INFO     - |   File "/usr/lib/python2.7/socket.py", line 480, in readline
> > > > 17:32:00 2018-08-30 15:29:09 - INFO     - |     data = self._sock.recv(self._rbufsize)
> > > > 17:32:00 2018-08-30 15:29:09 - INFO     - | socket.error: [Errno 104] Connection reset by peer
> > > > 17:32:00 2018-08-30 15:29:09 - INFO     - | WARNING: exit code 1 from a shell command.
> > > > 17:32:00 2018-08-30 15:29:09 - INFO     - | ERROR: Function failed: do_rootfs (log file is located at /home/build/tmp-glibc/work/cortexa9hf-neon-oe-linux-gnueabi/image1/1.0-r0/temp/log.do_rootfs.10761)
> > > > 
> > > > Signed-off-by: Manuel Traut <manut at linutronix.de>
> > > > ---
> > > >  elbepack/soapclient.py | 9 ++++++++-
> > > >  1 file changed, 8 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/elbepack/soapclient.py b/elbepack/soapclient.py
> > > > index 25ba4090..953bc1f4 100644
> > > > --- a/elbepack/soapclient.py
> > > > +++ b/elbepack/soapclient.py
> > > > @@ -585,7 +585,14 @@ class WaitProjectBusyAction(ClientAction):
> > > >          part = 1
> > > >  
> > > >          while True:
> > > > -            busy = client.service.get_project_busy(builddir, part)
> > > > +            try:
> > > > +                busy = client.service.get_project_busy(builddir, part)
> > > > +            except socket.error as e:
> > > > +                print("socket error during wait busy occured, retry..",
> > > > +                      file=sys.stderr)
> > > > +                print(e.message)
> > > > +                continue
> > > 
> > > can we limit the retry to socket.errno == 104 ?
> > > because this would yield an endless loop, if the initvm goes away, or
> > > even more bonkers.
> > >
> > > this would pose even more problems during testing.
> > >
> > > also not happy with no retry count at this point.
> > 
> > You could easily quit by pressing CTRL+C
> 
> the test system does not do that.

But it uses timeouts that cancels the build-process if it consumes to much
time.

> > I hope there will be a better solution before becoming elbe-stable.
> > Would it be OK, to add a TODO marker in the code, that this is more or less
> > for debugging and should be improved, as soon as we know more detailed what's
> > going on?
> 
> if you limit it to errno == 104, its ok.
> then it would exit, when the initvm is down.
> 
> a little comment there, would also be nice.
> but i dont want that thing to loop endlessly, when no initvm
> is there.

OK, i'll  send a v2.

> > 
> > > 
> > > > +
> > > >              if busy == 'FINISH':
> > > >                  break
> > > >              else:
> > > > -- 
> > > > 2.19.0.rc2
> > > > 
> > > 
> > > -- 
> > > Torben Hohn
> > > Linutronix GmbH | Bahnhofstrasse 3 | D-88690 Uhldingen-Mühlhofen
> > > Phone: +49 7556 25 999 18; Fax.: +49 7556 25 999 99
> > > 
> > > Hinweise zum Datenschutz finden Sie hier (Informations on data privacy 
> > > can be found here): https://linutronix.de/kontakt/Datenschutz.php
> > > 
> > > Linutronix GmbH | Firmensitz (Registered Office): Uhldingen-Mühlhofen | 
> > > Registergericht (Registration Court): Amtsgericht Freiburg i.Br., HRB700 
> > > 806 | Geschäftsführer (Managing Directors): Heinz Egger, Thomas Gleixner
> > 
> > 
> 
> -- 
> Torben Hohn
> Linutronix GmbH | Bahnhofstrasse 3 | D-88690 Uhldingen-Mühlhofen
> Phone: +49 7556 25 999 18; Fax.: +49 7556 25 999 99
> 
> Hinweise zum Datenschutz finden Sie hier (Informations on data privacy 
> can be found here): https://linutronix.de/kontakt/Datenschutz.php
> 
> Linutronix GmbH | Firmensitz (Registered Office): Uhldingen-Mühlhofen | 
> Registergericht (Registration Court): Amtsgericht Freiburg i.Br., HRB700 
> 806 | Geschäftsführer (Managing Directors): Heinz Egger, Thomas Gleixner