[elbe-devel] [PATCH 07/25] py3: care on encoding of license files

[Torben Hohn]

On Fri, Dec 01, 2017 at 04:51:04PM +0100, Manuel Traut wrote:
> This tries to find the correct encoding of a copyright file and write the
> decoded content of the copyright file to licence.xml.
> 
> Licence texts that can't be decoded are dropped and an error message is
> logged.

Crap

> 
> Signed-off-by: Manuel Traut <manut at linutronix.de>
> ---
>  elbepack/efilesystem.py | 30 ++++++++++++++++++++----------
>  elbepack/licencexml.py  | 18 ++++++++++--------
>  2 files changed, 30 insertions(+), 18 deletions(-)
> 
> diff --git a/elbepack/efilesystem.py b/elbepack/efilesystem.py
> index be7d7e25..bd9f7453 100644
> --- a/elbepack/efilesystem.py
> +++ b/elbepack/efilesystem.py
> @@ -142,18 +142,28 @@ class ElbeFilesystem(Filesystem):
>                          (os.path.join(dir, "copyright"), e.strerror))
>                  lic_text = "Error while processing license file %s: '%s'" % (os.path.join(dir, "copyright"), e.strerror)
>  
> -            try:
> -                lic_text = unicode (lic_text, encoding='utf-8')
> -            except:
> -                lic_text = unicode (lic_text, encoding='iso-8859-1')
> -
> -
>              if not f is None:
>                  f.write(unicode(os.path.basename(dir)))
> -                f.write(u":\n================================================================================")
> -                f.write(u"\n")
> -                f.write(lic_text)
> -                f.write(u"\n\n")
> +                f.write(unicode(":\n================================================================================"))
> +                f.write(unicode("\n"))

what is this ?
2to3 converting u"" to unicode"" ?
to my understanding, py3 strings are always unicode, if they are not b''
(bytes).


> +                try:
> +                    f.write(lic_text.encode('utf-8'))

This is what bpython 3 says:
>>> f = open("/tmp/xxxtest","w")
>>> f
<_io.TextIOWrapper name='/tmp/xxxtest' mode='w' encoding='UTF-8'>
>>> f.write ("abc".encode ("utf-8"))
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: must be str, not bytes


> +                except TypeError as e:
> +                    log.printo(e)
> +                    log.printo("error by writing licence of %s" % (os.path.join(dir, "copyright")))
> +                    log.printo(str(lic_text))
> +                    f.write(unicode(str(lic_text)))
> +                except UnicodeDecodeError as e:
> +                    log.printo(e)
> +                    log.printo("error by writing licence of %s" % (os.path.join(dir, "copyright")))
> +                    log.printo(str(lic_text))
> +                    n = f.name
> +                    f.close()
> +                    with open (n, 'a') as f:
> +                        f.write(lic_text)
> +                    f = open(n, 'ab')

If we have some technical problem with ONE license, we just emit an Error ?
This is not the right thing todo.
Sorry. Either ALL licenses, or no licenses. Talk with tglx about this.




> +
> +                f.write(unicode("\n\n"))
>  
>              if not xml_fname is None:
>                  licence_xml.add_copyright_file (os.path.basename(dir), lic_text)
> diff --git a/elbepack/licencexml.py b/elbepack/licencexml.py
> index a40dffcd..d2c4979a 100644
> --- a/elbepack/licencexml.py
> +++ b/elbepack/licencexml.py
> @@ -66,9 +66,17 @@ class copyright_xml (object):
>          xmlpkg = self.pkglist.append('pkglicense')
>          xmlpkg.et.attrib['name'] = pkg_name
>          txtnode = xmlpkg.append ('text')
> -        txtnode.et.text = copyright
>  
> -        bytesio = io.StringIO (unicode(txtnode.et.text))
> +        # just return if we cant decode the copyright file; we also return
> +        # if we can't interpret it, so this should be ok
> +        try:
> +            txtnode.et.text = unicode(copyright)
> +            bytesio = io.StringIO (txtnode.et.text)
> +        except TypeError as e:
> +            return
> +        except UnicodeDecodeError as e:
> +            return
> +

This also looks broken.

>          try:
>              c = Copyright (bytesio)
>              files = []
> @@ -125,9 +133,3 @@ class copyright_xml (object):
>  
>      def write(self, fname):
>          self.outxml.write (fname, encoding="iso-8859-1")
> -
> -        
> -
> -
> -
> -
> -- 
> 2.15.1
> 
> 
> _______________________________________________
> elbe-devel mailing list
> elbe-devel at linutronix.de
> https://lists.linutronix.de/mailman/listinfo/elbe-devel

-- 
Mit freundlichen Grüßen
Torben Hohn

Linutronix GmbH

Standort: Bremen

Phone: +49 7556 25 999 18; Fax.: +49 7556 25 999 99

Firmensitz / Registered Office: D-88690 Uhldingen, Bahnhofstr. 3
Registergericht / Local District Court: Amtsgericht Freiburg i. Br.; HRB
Nr. / Trade register no.: 700 806

Geschäftsführer / Managing Directors: Heinz Egger, Thomas Gleixner

Eine Bitte von uns: Sollten Sie diese E-Mail irrtümlich erhalten haben,
benachrichtigen Sie uns in diesem Falle bitte sobald wie es Ihnen
möglich ist, durch Antwort-Mail. Vielen Dank!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: Digital signature
URL: <http://lists.linutronix.de/pipermail/elbe-devel/attachments/20171206/0654c264/attachment.sig>