-
Notifications
You must be signed in to change notification settings - Fork 126
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Preprocessing a file at FNAL leads an unclear exception #1138
Comments
@bockjoo I thought you described this as an uproot problem in the slack where you saw something was deserializing incorrectly when using https. |
I'll note that I've been seeing this with the occasional file opened via xrootd. One specific example: the /DoubleMuon/Run2016F*NanoAODv9-v1/NANOAOD file (there's just one, about 2.1GB, which I'm still investigating because it seems to open fine in uproot from wisconsin, but whichever is being picked up by the datadiscoverycli with round-robin replica choice is triggering this error... and also the first option is the T1_US_FNAL disks which are under maintenance today) |
Here's a single-file CMS dataset for which many replicas fail:
This code should permit seeing the failure in action:
Output for me right now:
|
Hi, I also encountered the same issue for two datasets with many files. I have tried adding |
It means that no sites returned a valid list of files when trying to establish their existence. |
I think I was reading a file from root:// protocol, not https, with |
When uproot raises an exception, it does not provide the file name and the reason for error, which should be added |
How could that happen when the error does not appear during I thought the error was raised as long as there was one bad file. |
At the moment, this fails:
as is reported here, which I reported to a FNAL admin.
instead of
Normally, it's supposed to be accessed using
which will open the file from one of these sites:
|
Redirectors are known to be flakey for accessing files consistently, prior success unfortunately means you were only lucky. This particular error happens when you try to slice an array that consists entirely of I'll make a PR that should at least report this outcome more clearly. I'll @ you and you can try it. |
Could one of you please try #1168? |
This fix produces the updated error msg.
If my |
@JoyYTZhou I have added something to the PR that should give this functionality. Have a look in the PR and give it a try. |
@lgray Now I do get the preprocessed result dumped to my terminal (I would've preferred it to be a json.gz), but it still only dumps the result for the dataset that has failed (which shows none in every field). Based on the printed table index, that failed dataset was not the first to be processed, yet none of the previous results was dumped. If somehow the failed dataset was always picked to be run first then the preprocessed result wouldn't be useful. I could also delete the failed dataset from my query, that always works. |
All of the passed results are returned as two dictionaries:
What gets dumped to the screen are only the failed runs, as a standard python user warning. You may save or further process the returned dictionaries however you wish. |
@JoyYTZhou have you been able to 1) pass If you don't want to see the printout when the dataset fails you can use the control mechanisms available to you via https://docs.python.org/3/library/warnings.html. |
@lgray If the successfully parsed datasets are returned by |
https://github.com/CoffeaTeam/coffea/pull/1137/files this needs to be updated to add the extra arg |
@lgray Actually never mind, yes the results get dumped when |
Composability does not defeat the purpose of a shortcut. I'll add allow_empty_datasets in do_preprocess. My bad for missing you were using that as opposed preprocess directly. |
OK added to the rucio utils. Please give it a try. |
Yes, I get the successful outputs now. Thank you. I think this fix closes the issue. |
I am trying to preprocess a file at FNAL with Coffea2024.6.1, but got this exception:
It was unclear what went wrong.
The text was updated successfully, but these errors were encountered: