-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH: xarray support for COSMIC GPS #18
Conversation
Somehow some commits from my other pull ended up in here. sigh |
Marking this as ready for review as it is nominally complete. More accurately, the basic loading, cleaning, etc. is in there. Getting the tests to pass will be most effective after getting the previous pulls in. |
Ensure time mangling, if still present, is noted in the docstring. |
…ysat/pysatCDAAC into xarray_support � Conflicts: � pysatCDAAC/instruments/cosmic_gps.py
I was behind on my packages in comparison. I cloned my environment, upgraded xarray and pandas to the same version you have, and in another round upgraded to the absolute latest. I still get the correct file dates and times in either case.... What about numpy version? |
numpy 1.19.0 |
Installed a close numpy, as well as the latest, no dice. Files keep working for me.
|
I did try to specifically get version 1.19 for numpy but specifying numpy=1.19.0 doesn't work and numpy=1.19 gets me numpy v 1.19.2. This is via conda. |
I'm on python 3.7.6. What about netCDF4 version? |
python 3.8.2, netCDF4 1.5.6 |
OK! I got through a fresh install and files are still looking good on my end. I don't have any other immediate ideas for things to try on my end. Open to suggestions of course and I'll give it some thought while poking at other issues.
|
I created issue #24 to track the download and file issue. That bug is out of scope for this particular pull since no changes have been made to those lines here. |
OK, the date corruption is still messing with this, but I've tried to fix it as best as I can. Deleted all data, downloaded only Jan 1 2014 data for ionprf.
Here's the data loaded:
I'm also getting a user warning from pysat about metadata, which is presumably because we are converting attributes to variables when we sum across files. Not sure why so much data is being dropped. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some style bits on comments. Could you verify what loaded data looks like on your end? Not sure if my "dropped data" issue above is related to the file issues documented elsewhere.
Co-authored-by: Jeff Klenzing <[email protected]>
I'm also getting nans in occ_id and some others. When I check
I'm going to have to come back to this. I've got some checkout work to do before 8/31. I'd expect some nans in the coordinate variables near the end. I had to extend all profiles to the longest profile length for the day. |
Initial checks using the pysat branch at pysat/pysat#913: File issues are solved. Loaded
|
Verified this loads as expected for |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Trying to load and work with data, it looks like everything is working as expected. Tests runs for 'ionprf' and 'scnlvl1'. There are some issues (default metadata), but I think those can be written up as a separate issue.
Going to start working on a branch downstream of this. Recommend we merge this is in to develop and treat other major changes as new branches. |
Thanks for the reminder that this was already ready already 🐒 |
Addresses #1 by including support for loading COSMIC GPS data into xarray. Full testing of this pull requires incorporating existing pulls, namely #14 and #17, as well as updates to GitHub actions and such :)