-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Column optimization breaks in case of computing function of records in addition to exploded records? #426
Comments
I've been messing around with this issue again. @agoose77 is there a quick way to determine where the The exact code block @lgray posted above does indeed raise the error, but if I change it up a bit and do something like this: ...
mval_dask, (a_dask, b_dask) = dask.compute(mval_dask, (a_dask, b_dask))
print(a_dask) # <-- error
print(b_dask) # <-- error
print((a_dask, b_dask)) #<-- NO error! Something strange going on with @lgray the compute appears to be running successfully, but are you running into a case where some fields you expect to exist in the resulting array are instead populated with a |
@douglasdavis are we on the same page that the cause of this is the issue discussed in #430? I'm just away from my PC at the moment, but I'll touch base on this when I'm back. |
I'll give it a try! |
Yeah it's a specific array (usually the first one in the record?) that is a PlaceholderArray and it is only that field, the other are properly backed by data. The failure is triggered when I attempt to access the data of the bugged field. |
@agoose77 yes I think we're on the same page with #430! I was just taking a step back to remember what exactly was going on and that led me to observing that confusing repr/str stuff - just wanted to confirm with Lindsey what the more concrete issue was for him, which he just confirmed :) And it also got me thinking about how it would be useful to walk down an array and easily find placeholders |
@douglasdavis fab! RE your actual question, no — not right now. You can use For debugging, you could just set |
Sounds like a useful thing for debugging. It isn't great if you can't repr an object. |
Yes, that's the argument in favour. Right now the error is somewhat useful, because normal pathways should never need to repr these arrays — this would be a touching operation. Perhaps we want a context variable that can be set to opt-in. |
I think a dumb repr like |
results in:
And we shouldn't be running into
PlaceholderArray
s after column optimization!If you flip column optimization to False there are no more placeholder arrays, but an assertion is throw because of an error I need to track down in coffea (but there's no exception from within awkward array).
You'll need to use
coffea@master
to have this produce the appropriate error.The text was updated successfully, but these errors were encountered: