Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preprocessing filesets in coffea 0.7.x #1052

Open
ikrommyd opened this issue Mar 1, 2024 · 1 comment
Open

Preprocessing filesets in coffea 0.7.x #1052

ikrommyd opened this issue Mar 1, 2024 · 1 comment
Labels
question Further information is requested

Comments

@ikrommyd
Copy link
Contributor

ikrommyd commented Mar 1, 2024

Is it possible to have some type of fileset preprocessing for coffea 0.7.x that sort of checks if a file is available and readable?
Should come in handy as a #1036 enhancement since the preprocessing part was removed due to its dependance on dasky things.

@ikrommyd ikrommyd added the question Further information is requested label Mar 1, 2024
@valsdav
Copy link
Contributor

valsdav commented Mar 1, 2024

I think that if the preprocessing in 0.7 breaks down because a file is not available, one can just blacklist a site and re-run the dataset tools to refresh the list of replicas. This approach has been a bit manual but successful in 0.7.x until now :)

Being able to get a report of the failed files from the preprocessing would be indeed good, as usually it's difficult to understand which file is failing..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants