Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unicode, e.g. crescent moon 🌙 & beers 🍻 breaks under Python 3 #159

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Unicode, e.g. crescent moon 🌙 & beers 🍻 breaks under Python 3 #159

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
657fae8
Add coding: utf-8 header needed for Py2.6/Py2.7
pzrq Jan 18, 2016
d7afa20
Add test for the unicode crescent moon 🌙 issue
pzrq Jan 18, 2016
48c43b8
Do one thing per line for easier debugging
pzrq Jan 19, 2016
031f35e
Add os: osx to .travis.yml
pzrq Jan 28, 2016
66bd536
Try switching back to 2.7
pzrq Jan 28, 2016
f32952d
Last long shot try for OSX Travis
pzrq Jan 28, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 12 additions & 45 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,54 +1,21 @@
# https://travis-ci.org/peterbe/premailer

# This indicates to Travis that we will not use or need sudo
# so that we can benefit from and use the cache->directories
# directive.
sudo: no

env: PIP_DOWNLOAD_CACHE="pip_cache"
cache:
directories:
- pip_cache

language: python

before_install:
# twine is necessary so that travis can deploy
- travis_retry pip install twine
- travis_retry pip install tox coveralls

install:
- travis_retry python setup.py install

# See https://github.com/travis-ci/travis-ci/issues/4794
# and https://github.com/audreyr/cookiecutter/pull/540/files
python: 3.5

python: 2.7
sudo: false
os:
- linux
- osx
env:
- TOX_ENV=flake8
- TOX_ENV=lint
- TOX_ENV=py26
- TOX_ENV=py27
- TOX_ENV=pypy
- TOX_ENV=py32
- TOX_ENV=py33
- TOX_ENV=py34
- TOX_ENV=py35
- TOX_ENV=pypy
- TOX_ENV=docs

install:
- pip install tox

script:
- tox -e $TOX_ENV

after_success:
# Report coverage results to coveralls.io
- coveralls

deploy:
provider: pypi
user: peterbe
password:
secure: mdzD1SHDQ84ZK8tVXdF2LuMsws7it/7foompnuNUTQSVaKdjaHpwTYQtmdLUX0Jv54Plx4861GWNf6CD5IVfU7G8R2Ls/5/tQECFG01AspGz23lL1PxUg9tpNm+LPkQtkJzvS3f4i6/mODnKcoaSnhL63JuGaUNa72B1yBaN4hY=
on:
repo: peterbe/premailer
distributions: "sdist bdist_wheel"

notifications:
email:
on_success: never
3 changes: 2 additions & 1 deletion premailer/premailer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -462,7 +462,8 @@ def transform(self, pretty_print=True, **kwargs):
kwargs.setdefault('method', self.method)
kwargs.setdefault('pretty_print', pretty_print)
kwargs.setdefault('encoding', 'utf-8') # As Ken Thompson intended
out = etree.tostring(root, **kwargs).decode(kwargs['encoding'])
out_bytes = etree.tostring(root, **kwargs)
out = out_bytes.decode(kwargs['encoding'])
if self.method == 'xml':
out = _cdata_regex.sub(
lambda m: '/*<![CDATA[*/%s/*]]>*/' % m.group(1),
Expand Down
18 changes: 18 additions & 0 deletions premailer/tests/test_premailer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# -*- coding: utf-8 -*-
from __future__ import absolute_import, unicode_literals
import sys
import re
Expand Down Expand Up @@ -2225,6 +2226,23 @@ def test_capture_cssutils_logging(self):
p.transform() # it should work
eq_(mylog.getvalue(), '')

def test_unicode_crescent_moon_0x1f319(self):
html = '''<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta charset="UTF-8">
</head>
<body>
Dear Skyelar 🌙,
</body>
</html>
'''
p = Premailer(html)

# Calling transform should not mangle
# the unicode character 🌙
mojibake = p.transform()
compare_html(mojibake, html)

def test_type_test(self):
"""test the correct type is returned"""

Expand Down