Test failing with coffea 2024.5.0

[GaetanLepage]

Describe the bug

Context: Updating coffea in the nixpkgs repo (NixOS/nixpkgs#307316).
We are updating uproot at the same time (to 5.3.6).

The following two tests fail in our sandbox:

FAILED tests/test_lumi_tools.py::test_lumimask[https://raw.githubusercontent.com/CoffeaTeam/coffea/master/tests/samples/Cert_294927-306462_13TeV_EOY2017ReReco_Collisions17_JSON.txt] - FileNotFoundError: https://r>
FAILED tests/test_lumi_tools.py::test_lumilist_dask - TypeError: ('Could not serialize object of type HighLevelGraph', '<ToPickle...

The first failure makes sense, as our sandbox does not have internet access. (From my understanding, this is the only test in the coffea test suite that requires a network access).
However, I don't know if the error in the second one is expected.

To Reproduce
Steps to reproduce the behavior:

1. Run pytest
2. See error

Expected behavior
The tests don't fail.

Output

>               result = pickle.dumps(x, **dump_kwargs)
E               AttributeError: Can't pickle local object 'unpack_collections.<locals>.repack'
....
                raise TypeError(msg) from exc
>           raise TypeError(msg, str_x) from exc
E           TypeError: ('Could not serialize object of type HighLevelGraph', '<ToPickle: HighLevelGraph with 20 layers.\n<dask.highlevelgraph.HighLevelGraph object at 0x7ffc468a18d0>\n 0. Dict-ee30ab14-73e1-45

Desktop (please complete the following information):

• OS: NixOS
• Version 24.05

[GaetanLepage]

Full log: https://pastebin.com/YPdvMUJ6

[ikrommyd]

The second is caused by the first one failing. It happens on my machine as well If I change https://raw.githubusercontent.com/CoffeaTeam/coffea/master/tests/samples/Cert_294927-306462_13TeV_EOY2017ReReco_Collisions17_JSON.txt" to something that does not exist.
If you run the second test individually it should succeed.
I'm seeing aiohttp errors in the log like aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host raw.githubusercontent.com:443 ssl:default [Temporary failure in name resolution] as it's trying to read the remote file over http. What I don't see at the moment is why.

[GaetanLepage]

Ok, so the second test also requires to access the internet ? If so, I will disable both and go on.
This is the usual procedure for us as our build environment is isolated from the network.

[ikrommyd]

Ah okay, makes sense then. Yes, it was an update to read these jsons over the internet using fsspec instead of local paths only. The first test is failing because of no network. I don't really understand why the second is failing if the first one fails as it should be unrelated to the first one. There appears to be some pollution there. I think you can disable just the first one (tests/test_lumi_tools.py::test_lumimask) if your testing suite has no internet access and it should be good.

@lgray could you check why there might be pollution? If https reading fails in test_lumimask, then test_lumilist_dask fails as well with
TypeError: ('Could not serialize object of type HighLevelGraph', '<ToPickle: HighLevelGraph with 20 layers.\n<dask.highlevelgraph.HighLevelGraph object at 0x151272ad0>\n 0. lumilist-unique-chunked-5065684c7097269d3cf23855fcb99e15\n 1. lumilist-unique-finalize-0dc7eb9b6cdfd1bdd74fda4865fb...

[GaetanLepage]

Indeed, ignoring the first one was enough.
Thanks for the quick help @ikrommyd !