Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AssertionError: 'iso-8859-2' == 'windows-1252' during encoding test #433

Open
awilfox opened this issue Jan 13, 2020 · 1 comment
Open

AssertionError: 'iso-8859-2' == 'windows-1252' during encoding test #433

awilfox opened this issue Jan 13, 2020 · 1 comment

Comments

@awilfox
Copy link

awilfox commented Jan 13, 2020

Python 3.6.9 on Linux kernel 5.4.5 with musl libc 1.2.0.

    def test_encoding():
        for filename in get_data_files("encoding"):
            tests = _TestData(filename, b"data", encoding=None)
            for test in tests:
>               runParserEncodingTest(test[b'data'], test[b'encoding'])

html5lib/tests/test_encoding.py:102: 
_ _ _ _
data = b'<!DOCTYPE HTML>\n<script>document.write(\'<meta charset="ISO-8859-\' + \'2">\')</script>', encoding = 'iso-8859-2'

    def runParserEncodingTest(data, encoding):
        p = HTMLParser()
        assert p.documentEncoding is None
        p.parse(data, useChardet=False)
        encoding = encoding.lower().decode("ascii")
    
>       assert encoding == p.documentEncoding, errorMessage(data, encoding, p.documentEncoding)
E       AssertionError: Input:
E         b'<!DOCTYPE HTML>\n<script>document.write(\'<meta charset="ISO-8859-\' + \'2">\')</script>'
E         Expected:
E         'iso-8859-2'
E         Recieved
E         'windows-1252'
E         
E       assert 'iso-8859-2' == 'windows-1252'
E         - iso-8859-2
E         + windows-1252

html5lib/tests/test_encoding.py:84: AssertionError
==== short test summary info ====
FAILED html5lib/tests/test_encoding.py::test_encoding - AssertionError: Input:
==== 1 failed, 15715 passed, 14980 skipped, 666 xfailed in 137.44s (0:02:17) ====
@openandclose
Copy link

openandclose commented Jan 23, 2020 •
edited
Loading

Not directly related,
but is the test data itseif right, spec-wise?

A light investigation tells me it is.
(Numerous reservations and exceptions in 'document.write' part of the html spec,
notably WICG/interventions#17,
doesn't apply here, I think).

But Firefox and Chrome think differently

<!DOCTYPE HTML>
<script>document.write('<meta charset="ISO-8859-' + '2">')</script>

Firefox: UTF-8
Chrome: windows-1252

For comparison:

<!DOCTYPE HTML>
<script>document.write('<meta charset="ISO-8859-2">')</script>

Firefox: ISO-8859-2
Chrome: windows-1252

(Firefox 72.0, Chromium 79.0.3945.117, document.characterSet in web consle)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@awilfox @openandclose