[elbe-devel] [PATCH v2 17/27] log: filter out ansi sequences for the soap logger

[Torben Hohn]

spyne wsgi chokes on ansi color sequences in strings, because lxml
does not like them.

in the log below, the W: lines are color coded yellow.

-------------------------------------------------------------------------------------------------------------------------------------------------
[CMD] pbuilder --create --configfile "/var/cache/elbe/6e569880-0c95-4463-9c11-ff358ab73d3c/pbuilderrc" --aptconfdir "/var/cache/elbe/6e569880-0c95-4463-9c11-ff358ab73d3c/aptconfdir" --debootstrapopts --include="git gnupg"
W: /root/.pbuilderrc does not exist
W: cgroups are not available on the host, not using them.
ERROR:spyne.util.coopmt:Exception in coroutine
ERROR:spyne.util.coopmt:All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/spyne/util/coopmt.py", line 51, in start
    next(ret)
  File "/usr/lib/python3/dist-packages/spyne/protocol/xml.py", line 856, in _get_members_etree
    sub_name)
  File "/usr/lib/python3/dist-packages/spyne/protocol/xml.py", line 525, in to_parent
    return handler(ctx, cls, inst, parent, ns, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/spyne/protocol/xml.py", line 679, in modelbase_to_parent
    elt.text = self.to_unicode(cls, inst)
  File "src/lxml/etree.pyx", line 1018, in lxml.etree._Element.text.__set__
  File "src/lxml/apihelpers.pxi", line 710, in lxml.etree._setNodeText
  File "src/lxml/apihelpers.pxi", line 698, in lxml.etree._createTextNode
  File "src/lxml/apihelpers.pxi", line 1493, in lxml.etree._utf8
ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters
ERROR:spyne.server.wsgi:All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/spyne/server/wsgi.py", line 452, in handle_rpc
    self.get_out_string(p_ctx)
  File "/usr/lib/python3/dist-packages/spyne/server/_base.py", line 142, in get_out_string_pull
    ret = ctx.out_protocol.serialize(ctx, message=ProtocolBase.RESPONSE)
  File "/usr/lib/python3/dist-packages/spyne/protocol/soap/soap11.py", line 334, in serialize
    out_body_doc, body_message_class.get_namespace())
  File "/usr/lib/python3/dist-packages/spyne/protocol/xml.py", line 525, in to_parent
    return handler(ctx, cls, inst, parent, ns, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/spyne/protocol/xml.py", line 890, in complex_to_parent
    add_type)
  File "/usr/lib/python3/dist-packages/spyne/util/coopmt.py", line 51, in start
    next(ret)
  File "/usr/lib/python3/dist-packages/spyne/protocol/xml.py", line 745, in gen_members_parent
    ret = self._get_members_etree(ctx, cls, inst, elt)
  File "/usr/lib/python3/dist-packages/spyne/util/coopmt.py", line 51, in start
    next(ret)
  File "/usr/lib/python3/dist-packages/spyne/protocol/xml.py", line 856, in _get_members_etree
    sub_name)
  File "/usr/lib/python3/dist-packages/spyne/protocol/xml.py", line 525, in to_parent
    return handler(ctx, cls, inst, parent, ns, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/spyne/protocol/xml.py", line 679, in modelbase_to_parent
    elt.text = self.to_unicode(cls, inst)
  File "src/lxml/etree.pyx", line 1018, in lxml.etree._Element.text.__set__
  File "src/lxml/apihelpers.pxi", line 710, in lxml.etree._setNodeText
  File "src/lxml/apihelpers.pxi", line 698, in lxml.etree._createTextNode
  File "src/lxml/apihelpers.pxi", line 1493, in lxml.etree._utf8
ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters
-------------------------------------------------------------------------------------------------------------------------------------------------

Filter out ANSI sequences in the async_logging code, before passing them through
the soap interface.

Maybe we should do it a bit later, to catch all occurences.

Signed-off-by: Torben Hohn <torben.hohn at linutronix.de>
---
 elbepack/log.py | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/elbepack/log.py b/elbepack/log.py
index 22f747dab..db1fb770b 100644
--- a/elbepack/log.py
+++ b/elbepack/log.py
@@ -8,6 +8,8 @@ import collections
 import logging
 import os
 import threading
+import re
+
 from contextlib import contextmanager
 
 root = logging.getLogger()
@@ -263,7 +265,12 @@ class AsyncLogging(object):
 
             # Log the line now for echo back
             if cnt:
-                self.stream.info("\n".join(self.lines[-cnt:]))
+                logbuf = "\n".join(self.lines[-cnt:])
+
+                # filter out ansi sequences.
+                logbuf = re.sub('\u001b\[.*?[@-~]', '', logbuf)
+
+                self.stream.info(logbuf)
 
             # Keep rest for next line buffering
             rest = buf[j:]
-- 
2.20.1