`test_interpreters` misses tests with all `pickle` protocols

[sobolevn]

There are multiple places in test_interpreters where pickle only tests the default protocol.

Like:

cpython/Lib/test/test_interpreters/test_api.py

Lines 413 to 417 in 8ac7613

	def test_pickle(self):
	interp = interpreters.create()
	data = pickle.dumps(interp)
	unpickled = pickle.loads(data)
	self.assertEqual(unpickled, interp)

I guess that we can improve the tests with more protocols.

But, when I do that protocols 0 and 1 fail for all objects.
I decided to just limit the tests for now, because I am not sure if this is actually an intended behavior:

======================================================================
ERROR: test_pickle (test.test_interpreters.test_channels.TestRecvChannelAttrs.test_pickle) (protocol=0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_interpreters/test_channels.py", line 128, in test_pickle
    self.assertEqual(unpickled, ch)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/unittest/case.py", line 925, in assertEqual
    assertion_func(first, second, msg=msg)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/unittest/case.py", line 915, in _baseAssertEqual
    if not first == second:
           ^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/test/support/channels.py", line 105, in __eq__
    return other._id == self._id
                        ^^^^^^^^
AttributeError: 'RecvChannel' object has no attribute '_id'. Did you mean: 'id'?

======================================================================
ERROR: test_pickle (test.test_interpreters.test_channels.TestRecvChannelAttrs.test_pickle) (protocol=1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_interpreters/test_channels.py", line 128, in test_pickle
    self.assertEqual(unpickled, ch)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/unittest/case.py", line 925, in assertEqual
    assertion_func(first, second, msg=msg)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/unittest/case.py", line 915, in _baseAssertEqual
    if not first == second:
           ^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/test/support/channels.py", line 105, in __eq__
    return other._id == self._id
                        ^^^^^^^^
AttributeError: 'RecvChannel' object has no attribute '_id'. Did you mean: 'id'?

======================================================================
ERROR: test_pickle (test.test_interpreters.test_channels.TestSendChannelAttrs.test_pickle) (protocol=0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_interpreters/test_channels.py", line 161, in test_pickle
    self.assertEqual(unpickled, ch)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/unittest/case.py", line 925, in assertEqual
    assertion_func(first, second, msg=msg)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/unittest/case.py", line 915, in _baseAssertEqual
    if not first == second:
           ^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/test/support/channels.py", line 105, in __eq__
    return other._id == self._id
                        ^^^^^^^^
AttributeError: 'SendChannel' object has no attribute '_id'. Did you mean: 'id'?

======================================================================
ERROR: test_pickle (test.test_interpreters.test_channels.TestSendChannelAttrs.test_pickle) (protocol=1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_interpreters/test_channels.py", line 161, in test_pickle
    self.assertEqual(unpickled, ch)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/unittest/case.py", line 925, in assertEqual
    assertion_func(first, second, msg=msg)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/unittest/case.py", line 915, in _baseAssertEqual
    if not first == second:
           ^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/test/support/channels.py", line 105, in __eq__
    return other._id == self._id
                        ^^^^^^^^
AttributeError: 'SendChannel' object has no attribute '_id'. Did you mean: 'id'?

======================================================================
FAIL: test_pickle (test.test_interpreters.test_api.InterpreterObjectTests.test_pickle) (protocol=0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_interpreters/test_api.py", line 419, in test_pickle
    self.assertEqual(unpickled, interp)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
AssertionError: <concurrent.interpreters.Interpreter object at 0x102928d60> != Interpreter(22)

======================================================================
FAIL: test_pickle (test.test_interpreters.test_api.InterpreterObjectTests.test_pickle) (protocol=1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_interpreters/test_api.py", line 419, in test_pickle
    self.assertEqual(unpickled, interp)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
AssertionError: <concurrent.interpreters.Interpreter object at 0x10293e9e0> != Interpreter(22)

======================================================================
FAIL: test_pickle (test.test_interpreters.test_queues.QueueTests.test_pickle) (protocol=0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_interpreters/test_queues.py", line 195, in test_pickle
    self.assertEqual(unpickled, queue)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: <concurrent.interpreters._queues.Queue object at 0x1026fc8d0> != Queue(17)

======================================================================
FAIL: test_pickle (test.test_interpreters.test_queues.QueueTests.test_pickle) (protocol=1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_interpreters/test_queues.py", line 195, in test_pickle
    self.assertEqual(unpickled, queue)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: <concurrent.interpreters._queues.Queue object at 0x102709800> != Queue(17)

----------------------------------------------------------------------
Ran 165 tests in 6.277s

Linked PRs

• gh-136285: Improve pickle protocol testing in test_interpreters #136286

[serhiy-storchaka]

Thank you for catching this. Indeed, pickle tests should use all protocols. In particular to catch such kind of errors.

It seems that these classes are implemented in Python. So the bug is in their implementation. What is the expected semantic for pickling these objects?

[sobolevn]

I will take a look tomorrow. And fix it probably :)

CC @ericsnowcurrently

[serhiy-storchaka]

What is the meaning of unpickling an interpreter/queue/channel in the different process? It is successful is id is just an integer, but in different process that id can be not used or used for different interpreter/queue/channel than we expect. It looks like transferring a file descriptor between processes.