Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Read bytes to avoid ijson deprecation #458

Closed

Conversation

jpmckinney
Copy link
Contributor

@jpmckinney jpmckinney commented Aug 23, 2024

Otherwise:

DeprecationWarning: 
ijson works by reading bytes, but a string reader has been given instead. This
probably, but not necessarily, means a file-like object has been opened in text
mode ('t') rather than binary mode ('b').

An automatic conversion is being performed on the fly to continue, but on the
other hand this creates unnecessary encoding/decoding operations that decrease
the efficiency of the system. In the future this automatic conversion will be
removed, and users will receive errors instead of this warning. To avoid this
problem make sure file-like objects are opened in binary mode instead of text
mode.

encoding="utf-8" needs to be removed, as codes.open will use mode "rt" instead of "rb" if it's present, and ijson will read the bytes as UTF-8 anyway.

Note that ijson has no special exception class for UTF-8 errors, so one test had to change.

@jpmckinney jpmckinney force-pushed the ijson-deprecation branch 3 times, most recently from 97376ec to 76b12c5 Compare August 23, 2024 22:15
@odscjames
Copy link
Contributor

Does this mean we can then remove the except UnicodeDecodeError as err: ... raise BadlyFormedJSONErrorUTF8(*err.args) below? Tests run ok without it.

If so we can then technically remove the BadlyFormedJSONErrorUTF8 class entirely as that's the only place it's used. However other people may be using it and thus get breakages, so maybe it's best to leave the class but put a docstring on it saying it is deprecated?

(Also a note in CHANGELOG.md would be good, thanks!)

@jpmckinney
Copy link
Contributor Author

Done

odscjames pushed a commit that referenced this pull request Sep 10, 2024
@odscjames
Copy link
Contributor

Thank you!

@odscjames
Copy link
Contributor

Merged in, thanks!

@odscjames odscjames closed this Sep 12, 2024
@jpmckinney jpmckinney deleted the ijson-deprecation branch September 12, 2024 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants