persistent_dict: add complex hashing, numpy scalar hashing

[matthiasdiener]

Please squash

[matthiasdiener]

This is ready for review.

[inducer]

I think a better approach would be to have all numpy scalars go to the same method, in which you convert to shape-() array and then use to_bytes on that.

[matthiasdiener]

Should we add similar functionality for ndarrays as well? maybe something like (tobytes, shape, dtype)?

[inducer]

No, definitely not!

• They're mutable.
• With that implementation, it would violate the fundamental property a hash satisfies, i.e. a == b imples hash(a) == hash(b). That's because the array's byte pattern depends on storage order (e.g. F/C ordering), for arrays that Numpy would otherwise consider equal.

[matthiasdiener]

It seems that tobytes() for complex256 (only that dtype) isn't stable:

import numpy as np

for i in range(10):
    k = np.complex256(1.1+2.2j)
    print(k.tobytes())

$ python t.py
b'\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\xff?\x00\x00\x00\x00\x00\x00\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\x00@\x00\x00\x00\x00\x00\x00'
b'\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\xff?\x00\x00\x00\x00\x00\x00\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\x00@\x00\x00\x00\x00\x00\x00'
b'\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\xff?\x00\x00\x00\x00\x00\x00\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\x00@\x00\x00\x00\x00\x00\x00'
b'\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\xff?\x00\x00\x00\x00\x00\x00\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\x00@\x00\x00\x00\x00\x00\x00'
b'\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\xff?\x00\x00\x00\x00\x00\x00\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\x00@\x00\x00\x00\x00\x00\x00'
b'\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\xff?\x00\x00\x00\x00\x00\x00\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\x00@\x00\x00\x00\x00\x00\x00'
b'\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\xff?\x00\x00\x00\x00\x00\x00\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\x00@\x00\x00\x00\x00\x00\x00'
b'\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\xff?\x00\x00\x00\x00\x00\x00\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\x00@\x00\x00\x00\x00\x00\x00'
b'\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\xff?\x86\xc5\xeel\x16\xfe\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\x00@`w\x00\x00\x00\x00'
b'\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\xff?\x86\xc5\xeel\x16\xfe\x00\xd0\xcc\xcc\xcc\xcc\xcc\x8c\x00@`w\x00\x00\x00\x00'

(same issue with an array of complex256)

[inducer]

That makes a little bit of sense to me, as (at least on x86), float128 (where complex256 is just two of those) is just the x87 (not a typo) 80-bit FP type. So there are six bytes that aren't accounted for and possibly arbitrary.

Maybe specially handle those? (Cast to float (yes, Python float, both parts for complex), call repr and hash that)?

[matthiasdiener]

That makes a little bit of sense to me, as (at least on x86), float128 (where complex256 is just two of those) is just the x87 (not a typo) 80-bit FP type. So there are six bytes that aren't accounted for and possibly arbitrary.

Oh 😞 In other words, this might also affect float128 itself?

Maybe specially handle those? (Cast to float (yes, Python float, both parts for complex), call repr and hash that)?

If you prefer that idea to f1b766c, I can implement that.

[matthiasdiener]

I implemented your suggestion in aeb824a (using Python's complex() for converting np.complex256 because that seemed a bit simper).

[inducer]

Thanks!