Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Read: Close Oldest Iterations #1422

Open
ax3l opened this issue Apr 8, 2023 · 2 comments
Open

Read: Close Oldest Iterations #1422

ax3l opened this issue Apr 8, 2023 · 2 comments
Assignees
@franzpoeschel
Labels
api: new additions to the API
Milestone
0.17.0

Comments

@ax3l
Copy link
Member

ax3l commented Apr 8, 2023

In reads of very large data series, it is common to run into ulimits.

One convinient way to support scripting that we could offer is to implement a user-facing option on how many iterations to not call .close() on when opening a new iteration in non-streaming read mode.

That way, we could clean up the oldest iterations we opened when opening a new one while looping through series for analysis and the user does not need to keep track. Such an option would also make it simpler for openPMD-viewer et al. to implement iterations through series.
@ax3l ax3l added the api: new additions to the API label Apr 8, 2023
@ax3l ax3l added this to the 0.16.0 milestone Apr 8, 2023
@ax3l
Copy link
Member Author

ax3l commented Apr 11, 2023 •
edited
Loading

Prerequisite: for reading, we currently cannot re-open closed() iterations yet #1606

@ax3l ax3l modified the milestones: 0.16.0, 0.17.0 Jun 11, 2024
@ax3l
Copy link
Member Author

ax3l commented Jun 11, 2024

Related to #1592 #1606

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@franzpoeschel franzpoeschel

Labels
api: new additions to the API
Projects
None yet
Milestone
0.17.0
Development

No branches or pull requests
2 participants
@ax3l @franzpoeschel