[elbe-devel] [PATCH] Prevent UnicodeEncodeError on initvm creation

[Torben Hohn]

On Fri, Feb 07, 2020 at 12:16:12PM +0100, Bastian Germann wrote:
> The stacktrace is...
> 
> [CMD] gpg --no-options                     --no-default-keyring
>             --keyring /home/bage/initvm/.elbe-in/elbe-keyring
>           --export                     --output
> /home/bage/initvm/.elbe-in/elbe-keyring.gpg
> Exception in thread Thread-1:
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
>     self.run()
>   File "/usr/lib/python2.7/threading.py", line 754, in run
>     self.__target(*self.__args, **self.__kwargs)
>   File "/home/bage/linutronix/elbe/elbe/elbepack/log.py", line 239, in
> __call__
>     self.run()
>   File "/home/bage/linutronix/elbe/elbe/elbepack/log.py", line 263, in run
>     self.lines[-1] += rest
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 32:
> ordinal not in range(128)

ok. so the error is there because self.lines contains bytes, which is
correct, imo.

and then a unicode sting is appended.
please add the BT into the commit log, and add:

	Reviewed-by: Torben Hohn <torben.hohn at linutronix.de>



> 
> Am 07.02.20 um 12:05 schrieb Torben Hohn:
> > On Sat, Feb 01, 2020 at 10:22:22AM +0100, bage at linutronix.de wrote:
> >> From: Bastian Germann <bage at linutronix.de>
> >>
> >> 4643a4e8bf1("Python3: harmonize binary/unicode strings") introduced a bytes to
> >> string conversion in elbepack.log that makes initvm creation throw a
> >> UnicodeEncodeError (that is ignored) with Python 2.  Remove that conversion.
> >>
> >> I successfully built some examples with Python 3 and it seems the former change
> >> is unnecessary at this point.
> > 
> > Sorry, i dont understand, what the problem is here.
> > 
> > rest is bytes() (that is str in python2)
> > but its the returnvalue of os.read which is binary data.
> > this needs to be decoded, to become unicode.
> > 
> > can you show the exact exception. because i dont see what shall fail.
> > why are we using TextIOWrapper() and not plain .decode() ?
> > 
> > TextIOWrapper is only necessary, if we can not be sure, that the bytes
> > we have read are chopped at utf-8 char boundaries.
> > 
> > but i only see that read() is called. that reads until eof...
> > 
> >>
> >> Signed-off-by: Bastian Germann <bage at linutronix.de>
> >> ---
> >>  elbepack/log.py | 4 ----
> >>  1 file changed, 4 deletions(-)
> >>
> >> diff --git a/elbepack/log.py b/elbepack/log.py
> >> index bffc6c1d4..f17df9e71 100644
> >> --- a/elbepack/log.py
> >> +++ b/elbepack/log.py
> >> @@ -11,8 +11,6 @@ import select
> >>  import threading
> >>  from contextlib import contextmanager
> >>  
> >> -from io import TextIOWrapper, BytesIO
> >> -
> >>  root = logging.getLogger()
> >>  root.setLevel(logging.DEBUG)
> >>  local = threading.local()
> >> @@ -257,8 +255,6 @@ class AsyncLogging(object):
> >>              if not rest:
> >>                  break
> >>  
> >> -        rest = TextIOWrapper(BytesIO(rest), encoding='utf-8', errors='replace').read()
> >> -
> >>          if self.lines:
> >>              self.lines[-1] += rest
> >>              self.block.info("\n".join(self.lines))
> >> -- 
> >> 2.20.1
> >>
> > 
> 
> _______________________________________________
> elbe-devel mailing list
> elbe-devel at linutronix.de
> https://lists.linutronix.de/mailman/listinfo/elbe-devel

-- 
Torben Hohn
Linutronix GmbH | Bahnhofstrasse 3 | D-88690 Uhldingen-Mühlhofen
Phone: +49 7556 25 999 18; Fax.: +49 7556 25 999 99

Hinweise zum Datenschutz finden Sie hier (Informations on data privacy 
can be found here): https://linutronix.de/kontakt/Datenschutz.php

Linutronix GmbH | Firmensitz (Registered Office): Uhldingen-Mühlhofen | 
Registergericht (Registration Court): Amtsgericht Freiburg i.Br., HRB700 
806 | Geschäftsführer (Managing Directors): Heinz Egger, Thomas Gleixner