Simplify our unpickling logic and add tests for Python 2 pickles #2160
+270
−20
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is fixing the issue that Alex pointed out here #2156 (comment) .
We do not need the
str
case - the stuff we are decoding is always bytes - otherwise we would not be in the pickling fallback. Nor do we need to specify abytes
encoding when decoding Python 2 objects. The "latin-1" fallback case is handled bypd.read_pickle
so it's redundant for us to have it too.The tests I added also pass without my changes to
_normalization.py
and I think suffice to show there is no behaviour change here.TODO add tests with a non-standard encoding?
Old Logic
Old logic for pickles inside the msgpack was:
Where raw means: