You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I set up a simple benchmark with a 67MB pickle and measured deserialization speed in 7 scenarios.
library
time
Python pickle.load
341 ms
Python json.load
397 ms
serde_json from_str
327 ms
bincode from_slice
314 ms
py-marshal marshal_load
691 ms
serde-pickle from_reader
1250 ms
serde-pickle from_slice
1310 ms
Is this a known behavior? Is there any hope of this getting improved in the foreseeable future? I share my setup below, so you can point out any issues or things I missed.
Data
>>> import random, string
>>> data = [''.join(random.sample(string.ascii_letters, 32)) for _ in range(2_000_000)]
Thanks for the report, I can more or less reproduce the results. (Please include all of the code next time though, it makes it much easier.)
This crate hasn't been optimized for speed (yet), so it's not surprising that it won't outperform Python's pickle module. As for a comparison between different formats, that is always a little more difficult to reason about.
In any case I can't spend much time on this at present - PRs are welcome and I expect there might be some easy wins achievable with basic profiling.
I set up a simple benchmark with a 67MB pickle and measured deserialization speed in 7 scenarios.
pickle.load
json.load
from_str
from_slice
marshal_load
from_reader
from_slice
Is this a known behavior? Is there any hope of this getting improved in the foreseeable future? I share my setup below, so you can point out any issues or things I missed.
Data
Python load
Rust load
Dependencies
The text was updated successfully, but these errors were encountered: