Exception raised : portalocker.exceptions.LockException: [Errno 9] Bad file descriptor

[alcoat]

https://github.com/python-cachier/cachier/blob/5776ee8d63c8a5719ccd3737f0f504bc1a7eb679/cachier/pickle_core.py#L145C67-L145C67

On linux system, the cache file seems to be closed twice. When replacing this part of code with:
#with portalocker.Lock(fpath, mode='rb') as cache_file:
cache_file = portalocker.Lock(fpath, mode='rb')
if cache_file is not None:
try:
self.cache = pickle.load(open(cache_file.filename,"rb"))
except EOFError:
self.cache = {}
the cachier seems to work fine.

[shaypal5]

I'm sorry, that's not enough information to address the bug.

The package is tested and used on Linux, and this is the first report of such an issue. Also, you didn't explain in anyway how this hampers in any way use of the package.

Also, I would not move from a context manager - which is the correct and more safe way to lock a file - to a simple use of Lock without a much better explanation.

This is issue is marked with wontfix until the issue is shown to be reproducible on any Linux machine, and we get a better explanation of what you believe might be causing the redundant locking, the negative effects of this phenomena and possibly even a better suggestion for a fix (I can't see why portalocker's context manager will cause double locking but their Lock method will not).

[alcoat]

Ok here is my environnement : mamba create -n cachier cachier
And the test script:
"""
import cachier
import time
@cachier.cachier()
def _takes_3_seconds(label, value):
time.sleep(3)
return f'{label} {value}'
print(_takes_3_seconds('two', 2))
"""
python cachier_test.py
Traceback (most recent call last):
File "/export/home/coat/cmems/cachier_test.py", line 12, in
print(_takes_3_seconds('two', 2))
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/coat/.conda/envs/cachier/lib/python3.12/site-packages/cachier/core.py", line 263, in func_wrapper
key, entry = core.get_entry(args, kwds)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/coat/.conda/envs/cachier/lib/python3.12/site-packages/cachier/base_core.py", line 42, in get_entry
return self.get_entry_by_key(key)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/coat/.conda/envs/cachier/lib/python3.12/site-packages/cachier/pickle_core.py", line 189, in get_entry_by_key
self._reload_cache()
File "/home/coat/.conda/envs/cachier/lib/python3.12/site-packages/cachier/pickle_core.py", line 124, in _reload_cache
with portalocker.Lock(fpath, mode='rb') as cache_file:
File "/home/coat/.conda/envs/cachier/lib/python3.12/site-packages/portalocker/utils.py", line 163, in enter
return self.acquire()
^^^^^^^^^^^^^^
File "/home/coat/.conda/envs/cachier/lib/python3.12/site-packages/portalocker/utils.py", line 290, in acquire
raise exceptions.LockException(exception)
portalocker.exceptions.LockException: [Errno 9] Bad file descriptor

The system used is Centos 7 or RockyLinux 8 on ext4 filesystem

[alcoat]

I am not suggesting a solution with the modification on my first comment : this is really uggly and should not be used. However cachier is not usable for me at this time.

[alcoat]

In fact it seems to be the same as #86 with NFS mount. When forcing the cache_dir on the local directory (like /tmp) everything is working

[lordjabez]

@alcoat you said it works if you move cache_dir to /tmp ? What's the type of filesystem you're using when it doesn't work? It wouldn't surprise me if non-local drives such NFS mounts, Samba shares, etc. won't work.

Why is it a non-starter for you to simply use a local disk?

[alcoat]

@lordjabez Yes, it works when you move the cache_dir to a local disk. Most of my apps are using NFS shares and do depend on a single computer. After looking for solutions, I found that it is not a problem with cachier but portalocker.
In fact I'm trying to use an app in which cachier is used on a network with share disk spaces

[lordjabez]

Perhaps you can create an issue with portalocker and link here? We can update the dependency when it's fixed?

[shaypal5]

Great, I've responded in the portalocker issue you've created with a link to the first NFS-related issue. Let's see when does this gets solved.

[alcoat]

[like] Coat, Alain reacted to your message:

…

________________________________ From: Shay Palachy-Affek ***@***.***> Sent: Tuesday, December 5, 2023 7:08:26 PM To: python-cachier/cachier ***@***.***> Cc: Coat, Alain ***@***.***>; State change ***@***.***> Subject: Re: [python-cachier/cachier] Exception raised : portalocker.exceptions.LockException: [Errno 9] Bad file descriptor (Issue #128) Great, I've responded in the portalocker issue you've created with a link to the first NFS-related issue. Let's see when does this gets solved. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https://github.com/python-cachier/cachier/issues/128*issuecomment-1841456651__;Iw!!ElGdukoduuk!WBpC30Z7nsrQTXukPgXZY70clmdtC80EKE8g072MC4z_hRs79yygzQVxex0pjdMIZ6-I199ZZ8bnifHUpDQWIE3n0R8$>, or unsubscribe<https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AXHLAOMMYH52V4STKUDGCNDYH5WKVAVCNFSM6AAAAABAFCJCEWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBRGQ2TMNRVGE__;!!ElGdukoduuk!WBpC30Z7nsrQTXukPgXZY70clmdtC80EKE8g072MC4z_hRs79yygzQVxex0pjdMIZ6-I199ZZ8bnifHUpDQW6hyf_Oo$>. You are receiving this because you modified the open/close state.Message ID: ***@***.***>