[elbe-devel] [PATCH] Prevent UnicodeEncodeError on initvm creation

[Bastian Germann]

The stacktrace is...

[CMD] gpg --no-options                     --no-default-keyring
            --keyring /home/bage/initvm/.elbe-in/elbe-keyring
          --export                     --output
/home/bage/initvm/.elbe-in/elbe-keyring.gpg
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/bage/linutronix/elbe/elbe/elbepack/log.py", line 239, in
__call__
    self.run()
  File "/home/bage/linutronix/elbe/elbe/elbepack/log.py", line 263, in run
    self.lines[-1] += rest
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 32:
ordinal not in range(128)

Am 07.02.20 um 12:05 schrieb Torben Hohn:
> On Sat, Feb 01, 2020 at 10:22:22AM +0100, bage at linutronix.de wrote:
>> From: Bastian Germann <bage at linutronix.de>
>>
>> 4643a4e8bf1("Python3: harmonize binary/unicode strings") introduced a bytes to
>> string conversion in elbepack.log that makes initvm creation throw a
>> UnicodeEncodeError (that is ignored) with Python 2.  Remove that conversion.
>>
>> I successfully built some examples with Python 3 and it seems the former change
>> is unnecessary at this point.
> 
> Sorry, i dont understand, what the problem is here.
> 
> rest is bytes() (that is str in python2)
> but its the returnvalue of os.read which is binary data.
> this needs to be decoded, to become unicode.
> 
> can you show the exact exception. because i dont see what shall fail.
> why are we using TextIOWrapper() and not plain .decode() ?
> 
> TextIOWrapper is only necessary, if we can not be sure, that the bytes
> we have read are chopped at utf-8 char boundaries.
> 
> but i only see that read() is called. that reads until eof...
> 
>>
>> Signed-off-by: Bastian Germann <bage at linutronix.de>
>> ---
>>  elbepack/log.py | 4 ----
>>  1 file changed, 4 deletions(-)
>>
>> diff --git a/elbepack/log.py b/elbepack/log.py
>> index bffc6c1d4..f17df9e71 100644
>> --- a/elbepack/log.py
>> +++ b/elbepack/log.py
>> @@ -11,8 +11,6 @@ import select
>>  import threading
>>  from contextlib import contextmanager
>>  
>> -from io import TextIOWrapper, BytesIO
>> -
>>  root = logging.getLogger()
>>  root.setLevel(logging.DEBUG)
>>  local = threading.local()
>> @@ -257,8 +255,6 @@ class AsyncLogging(object):
>>              if not rest:
>>                  break
>>  
>> -        rest = TextIOWrapper(BytesIO(rest), encoding='utf-8', errors='replace').read()
>> -
>>          if self.lines:
>>              self.lines[-1] += rest
>>              self.block.info("\n".join(self.lines))
>> -- 
>> 2.20.1
>>
>