Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pyfilesystem data object descriptors unreliable at flushing/closing #18

Open
d-w-moore opened this issue Aug 22, 2024 · 0 comments
Open
Assignees

Comments

@d-w-moore
Copy link
Collaborator

d-w-moore commented Aug 22, 2024

I am running on Ubuntu 20 with Python iRODS Client 2.1.0. This took minor changes to fs-irods to allow Python 3.8 compatibility.
One would expect the following script to leave behind a data object of nonzero length.

$ more ./test_unicode_output_sans_ctx_manager.py
import fs_irods
import irods.helpers as h
s=h.make_session()
fs = fs_irods.iRODSFS(s)
f = fs.open('/home/rods/test_unicode_output.txt','w')
f.write('\u1002')

We run it this way : (but, - spoiler! - for this issue it won't matter whether auto-closing data objects are enabled or not).

$ PYTHON_IRODSCLIENT_CONFIG__DATA_OBJECTS__AUTO_CLOSE=True python test_unicode_output_sans_ctx_manager.py

If running using the -m"pdb" option we'll notice f is of the type:

<_io.TextIOWrapper name='/home/rods/test_unicode_output.txt' encoding='utf-8'>

On the output end we'll see

$ ils -l
  rods              0 demoResc            0 2024-08-22.12:22 X test_unicode_output.txt

Size should be 3 bytes, the utf8 encode of the unicode character written, but instead it is zero. This means flush and/or close isn't done at the right time relative to GC-sequenced object destruct.
On the way, one might see this message

ERROR:irods.connection:Unable to send message. Connection to remote host may have closed. Releasing connection from pool.

as iRODS scrambles to write changes to the file too late.

Binary mode eg "wb" doesn't help.
What does help ?

  • forced use of context manager or
  • atexit.register( lambda f.close( ) )
@d-w-moore d-w-moore changed the title reliable t pyfilesystem data object descriptors unreliable at flushing/closing Aug 22, 2024
@d-w-moore d-w-moore self-assigned this Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant