Debian Bug report logs - #806826
pyyaml does not support literals in unicode over codepoint 0xffff

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: "John R. Lenton" <john.lenton@canonical.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: pyyaml does not support literals in unicode over codepoint 0xffff

Date: Tue, 01 Dec 2015 22:44:34 +0000

[Message part 1 (text/plain, inline)]

Source: pyyaml
Severity: normal
Tags: upstream patch

Dear Maintainer,

the yaml spec says that

   “The allowed character range explicitly excludes the surrogate
    block #xD800-#xDFFF, DEL #x7F, the C0 control block #x0-#x1F
    (except for #x9, #xA, and #xD), the C1 control block #x80-#x9F,
    #xFFFE, and #xFFFF.”

however pyyaml has chosen to negate that check and apply it to only
plane 0. This means that any yaml document that contains unicode
literals in higher planes will fail to parse (and, on output, use the
rather unfriendly \Uxxxxxxxx format).

The attached patch fixes this in a minimally intrusive way, by
extending the checks to cover the additional codepoints where
appropriate. A better fix would be to use the check as the spec
specifies it, but that would be a bigger change.

I tried to file this upstream first, but was unable to find a way to
report bugs there; sorry.

-- System Information:
Debian Release: stretch/sid
  APT prefers xenial-updates
  APT policy: (500, 'xenial-updates'), (500, 'xenial-security'), (500, 'xenial'), (100, 'xenial-backports')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.2.0-19-generic (SMP w/4 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

[wide-unicode.diff (text/x-diff, attachment)]

Message #10 received at 806826-close@bugs.debian.org (full text, mbox, reply):

From: Barry Warsaw <barry@debian.org>

To: 806826-close@bugs.debian.org

Subject: Bug#806826: fixed in pyyaml 3.11-3

Date: Wed, 02 Dec 2015 21:58:21 +0000

Source: pyyaml
Source-Version: 3.11-3

We believe that the bug you reported is fixed in the latest version of
pyyaml, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 806826@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Barry Warsaw <barry@debian.org> (supplier of updated pyyaml package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Wed, 02 Dec 2015 16:29:22 -0500
Source: pyyaml
Binary: python-yaml python-yaml-dbg python3-yaml python3-yaml-dbg
Architecture: source amd64
Version: 3.11-3
Distribution: unstable
Urgency: medium
Maintainer: Barry Warsaw <barry@debian.org>
Changed-By: Barry Warsaw <barry@debian.org>
Description:
 python-yaml - YAML parser and emitter for Python
 python-yaml-dbg - YAML parser and emitter for Python (debug build)
 python3-yaml - YAML parser and emitter for Python3
 python3-yaml-dbg - YAML parser and emitter for Python3 (debug build)
Closes: 806826
Changes:
 pyyaml (3.11-3) unstable; urgency=medium
 .
   * Team upload.
   * d/control: Bump Standards-Version with no other changes necessary.
   * d/patches/support-high-codepoints.patch: Added to fix support for
     codepoints above 0xffff.  Given by John R. Lenton.  (Closes: #806826)
   * d/tests/{control,python2.sh,python3.sh}: Added to test the above patch
     since a unittest is problematic.
Checksums-Sha1:
 0c8527b560332c8e323aff2608af0cd435984af9 2248 pyyaml_3.11-3.dsc
 b5b0a0b54b7b7e2cc787bcf40ec08787a2239fb5 8232 pyyaml_3.11-3.debian.tar.xz
 8efe84fe0260b6d9fc926deaf3eadd491d801f52 80208 python-yaml-dbg_3.11-3_amd64.deb
 1c551fb7487f81a020067851cc2c22d96284798e 107936 python-yaml_3.11-3_amd64.deb
 f62e8fdd2097bf148becd4577f58ce82b0ac9d79 94532 python3-yaml-dbg_3.11-3_amd64.deb
 2c13ac583e3239c519dfbbea6bb435a869577ab6 129014 python3-yaml_3.11-3_amd64.deb
Checksums-Sha256:
 068592696e2690ee8a24e83b4779c167b14ef67a8595bc545904c6eafe2b80b9 2248 pyyaml_3.11-3.dsc
 d04f11dc2a5018f79c4540e523b4f65788bc3f7b85e74fca8c51ccfc8ef9f82c 8232 pyyaml_3.11-3.debian.tar.xz
 9bf638617bafad8bd85bb06ba387d3eea87ecad603a63ff3432bd89bfb6967b4 80208 python-yaml-dbg_3.11-3_amd64.deb
 ad7c5ba7152bd2a891fa0732c6c3e3f4c9fc406737890b37398f0be73b16a78b 107936 python-yaml_3.11-3_amd64.deb
 7a076d1ec29da78686664d0d78b8a5e565f6b32a2536745d9e82181369444cd8 94532 python3-yaml-dbg_3.11-3_amd64.deb
 964c6aaca99aa59171b685721c6d6d4c9d9c3cfcead95cb2ba6acbba06f99820 129014 python3-yaml_3.11-3_amd64.deb
Files:
 e8eee475362fa45e1f3c6665fbacef4b 2248 python optional pyyaml_3.11-3.dsc
 5ff5902d242faaa8708f9b6e7baadb90 8232 python optional pyyaml_3.11-3.debian.tar.xz
 48b90a1e648e7d1ab9a4089d57309bc7 80208 debug extra python-yaml-dbg_3.11-3_amd64.deb
 5a62c810395261996104ce15100ea4d3 107936 python optional python-yaml_3.11-3_amd64.deb
 49e46249c1ae453417522e9079ab89e4 94532 debug extra python3-yaml-dbg_3.11-3_amd64.deb
 0e769d86349834707e9ccdf2f3cc098a 129014 python optional python3-yaml_3.11-3_amd64.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCAAGBQJWX2ZSAAoJEBJutWOnSwa/Z2EP/1hUPM/3OkVxI+pj/raE0XD5
1ZGpuj4iNZWwPc+dVmXLgQdER8OMRBFMkS/Ebn8ryq2w+ZHAXV2cMhmEC7G/1D6Q
xiy0KvRacRfZZCzV2Ptf+Q5aKmSpnc95ix1cHqzT/hpPbRj/8Dw7PN9YdI0HbUrw
LUnf2bBhe+74clswV9H0k55aHGivZ6lo7wiNjIjmKWL0HB3h9V1tlb9SQ9m+xk+Q
4Tzzwq+AsHLGA/89svgACd54LuzXXPgaDFqzhJ+vIXEDIdL1chYivMkN4yutrT+f
DhQBugGRdY/ycHyfQ3qQlkq6x8biQBpHla1dlrckdjKJF5XRYPmq/zOpfO8ZtaP3
B/FoDyLAG+NfXuSZ3Q1Eo3mjYDpEIgq/iJwQupbHYkjah7tFLrzHFcQqo6H38f6Q
3uTbMWHV1E1ozl9EUyQPx+EObvCKdQvW2xJdQiaLTdpQhoXhOtOiiY9SQncxj6b9
D3rWrI8KQB0Inm6YUHKH3vPgsXy2N26BLO8DrddHeP+9QTj57/BFq0jsfLzGnyCA
XURqK1vJGDwMyMKBOaXORWQjDXBVLU8tcTySvUkHQUNbopwIGFnqn+eqgURwZAZC
vGKlzbw/GZXo922tNqKi1NAaC1F9ViE2urFXfdIoGOlzoSA+mDcZhm44ChMm+OUi
I17f32l/LWk6qC85zyG1
=jaSE
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.