Port script to Python 3 #163
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Closed
Make sure to encode string objects to bytes and decode byte objects to
strings wherever necessary, as a preparatory step for transitioning to
Python 3. More specifically:
* Encode URLs before hashing them, as hashlib.sha256() expects a bytes
object in Python 3
* Convert chunk headers to bytes before writing them to a binary file
* Decode data right after receiving it from urlopen
* Decode the content of HTTP responses before printing it
* Decode data right after reading it from a binary file
* Encode the data used to mock urlopen in tests
Also, fix a minor bug by using the 'b' prefix at the start of each line
of a multi-line bytes literal instead of only using it for the first
line. Finally, omit the "utf-8" argument when encoding, as UTF-8 is the
default encoding.
Apply the following changes to make the list creation script compatible
with Python 3. After these changes it will no longer support Python 2.
* Import configparser instead of ConfigParser, as this module was
renamed in Python 3
* Import quote and unquote from urllib.parse and urlopen from
urllib.request instead of importing urllib2
* Remove mock from the dependencies as it is part of the standard
library in Python 3
* Pin dependencies to the latest version
* Use %d instead of %s when formatting a binary chunk header to avoid
getting a TypeError. Also, use %d instead of %u, which is deprecated
* Set the "version" key equal to the empty string instead of None when
reverting configuration to avoid getting a "TypeError: option values
must be strings".
* Update CircleCI configuration to use a docker image with Python 3.8
* Update Python version requirement in README
* Update README instructions to use virtualenv with Python 3
Closes mozilla-services#108.
Use config.read_file() to read configuration from a file instead of config.readfp(), which is deprecated. Also, use ConfigParser instead of SafeConfigParser, which is an alias that will be removed in future Python versions.
|
I just rebased this on stage. |
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR makes the necessary changes to enable the script to run under Python 3 and drops Python 2 support.
Note: Since Python 3.3 hash randomization is enabled by default. This means that the order of domains in the lists will be different every time the script is run, causing unnecessary list updates. Enforcing alphanumeric ordering in domains when writing the lists will resolve this. Consequently, this PR should not be merged before #132 is closed.
This is also why
test_get_plugin_lists()andtest_get_tracker_lists()are failing most of the time.What is more, running the script currently fails with
UnicodeDecodeErrorwhenchunk_metadata()tries to decode bytes after the end of the chunk header. This will be fixed when #162 lands, as that PR changes the implementation ofchunk_metadata().Closes #108