IMOS Netcdf compatible with GliderTools?

[soerenthomsen]

I copy this over for housekeeping:

@nmalan asked:

"@dennisstanley64, have you experimented at all with getting the IMOS glider netcdfs to play nice with GliderTools? I've been messing around with this (with limited success, some things work, others don't), and was wondering if anyone else had trodden this path before?"

[soerenthomsen]

@dennisstanley64 answered

i haven't tried glider tools (insert embarrassed emoji ) we have our own publicly available tool called glidescope which can be use to browse data and plot things up. generally i use this to roughly explore data then load it into python for more detailed analysis.
If the problem is opening the netcdfs in python. then there is a work around for this and that is to load them in via xarray.
here is some code i use to import three net cdfs into pandas data frames.the key bit is the last lines where i pass the netcdf data into xarray and then doing a to_dataframe from xarray instead of from netCDF directly. i don't know why this is more error tolerant but the direct import from netCDF doesn't work but via xarray does. 



import pandas as pdimport xarray as xrfrom netCDF4 import Dataset

# now get glider data gliderdatafilenameDec="IMOS_ANFOG_BCEOPSTUV_20191113T012730Z_SL287_FV01_timeseries_END-20191205T005735Z.nc"gliderdatafilenameJan="IMOS_ANFOG_BCEOPSTUV_20200123T021637Z_SL287_FV01_timeseries_END-20200224T233110Z.nc"gliderdatafilenameFeb="IMOS_ANFOG_BCEOPSTUV_20200227T032645Z_SL287_FV01_timeseries_END-20200327T232039Z.nc"# open the dataset
columns_to_drop =['PLATFORM', 'DEPLOYMENT', 'SENSOR1', 'SENSOR2', 'SENSOR3', 'SENSOR4',       'TIME_quality_control', 'LATITUDE', 'LATITUDE_quality_control',       'LONGITUDE', 'LONGITUDE_quality_control', 'HEAD',       'HEAD_quality_control', 'UCUR', 'UCUR_quality_control', 'VCUR',       'VCUR_quality_control', 'UCUR_GPS', 'UCUR_GPS_quality_control',       'VCUR_GPS', 'VCUR_GPS_quality_control', 'PHASE',       'PHASE_quality_control', 'PROFILE', 'PROFILE_quality_control', 'PRES',       'PRES_quality_control', 'DEPTH_quality_control',        'TEMP_quality_control', 'CNDC', 'CNDC_quality_control',        'PSAL_quality_control', 'DOX2', 'DOX2_quality_control', 'DOX1',       'DOX1_quality_control', 'CPHL', 'CPHL_quality_control', 'CDOM',       'CDOM_quality_control', 'VBSC', 'VBSC_quality_control', 'BBP',       'BBP_quality_control', 'IRRAD443', 'IRRAD443_quality_control',       'IRRAD490', 'IRRAD490_quality_control', 'IRRAD555',       'IRRAD555_quality_control', 'IRRAD670', 'IRRAD670_quality_control']  # columns to leave out of the dataframe to stop gobbling memory
gliderdataDec= xr.open_dataset(gliderdatafilenameDec, drop_variables = columns_to_drop).to_dataframe() #load netcdf into pandas dataframe via xarray#gliderdata.append(xr.open_dataset(gliderdatafilenameJan, drop_variables = columns_to_drop).to_dataframe())gliderdataJan= xr.open_dataset(gliderdatafilenameJan, drop_variables = columns_to_drop).to_dataframe() #load netcdf into pandas dataframe via xarraygliderdataFeb= xr.open_dataset(gliderdatafilenameFeb, drop_variables = columns_to_drop).to_dataframe() #load netcdf into pandas dataframe via xarray

 Hope this helpsDennis

    On Wednesday, 10 November 2021, 01:15:58 pm AEST, Neil Malan ***@***.***> wrote: