Fix for to_text and to_bytes error handlers

* surrogate_then_strict doesn't exist.  Switch to surrogate_or_strict
  instead.
* Found some bugs in the _text.py implementation
  * The composed error handlers (error handlers which are made up of two
    or more python encoding error handlers) had a wrong string in it,
    'surrogate_or_escape' doesn't exist.  Replaced that with
    'surrogate_or_replace' which is the correct handler name.
  * Left comment about the implicit conditions that are part of the
    surrogate_then_replace code path

Fixes #23865
Fixes #23861
This commit is contained in:
Toshio Kuratomi
2017-07-21 10:23:16 -07:00
parent 9986639681
commit fc5d71de0d
3 changed files with 16 additions and 13 deletions

View File

@@ -262,7 +262,7 @@ def get_hash(data, hashtype='sha1'):
except:
return None
h.update(to_bytes(data, errors='surrogate_then_strict'))
h.update(to_bytes(data, errors='surrogate_or_strict'))
return h.hexdigest()
@@ -453,11 +453,11 @@ def do_groupby(environment, value, attribute):
def b64encode(string):
return to_text(base64.b64encode(to_bytes(string, errors='surrogate_then_strict')))
return to_text(base64.b64encode(to_bytes(string, errors='surrogate_or_strict')))
def b64decode(string):
return to_text(base64.b64decode(to_bytes(string, errors='surrogate_then_strict')))
return to_text(base64.b64decode(to_bytes(string, errors='surrogate_or_strict')))
class FilterModule(object):