Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ArgumentError when loading variable #8

Open
clupprich opened this issue Jul 4, 2023 · 3 comments
Open

ArgumentError when loading variable #8

clupprich opened this issue Jul 4, 2023 · 3 comments

Comments

@clupprich
Copy link

clupprich commented Jul 4, 2023 •
edited
Loading

I'm getting an ArgumentError when loading some variables from a NetCDF file:

filename = "nowcast_202307030230.nc"
{:ok, file} = NetCDF.File.open(filename)

# works fine
{:ok, variable} = NetCDF.Variable.load(file, "TT")

# raises
{:ok, variable} = NetCDF.Variable.load(file, "lon")
** (ArgumentError) argument error
    (netcdf 0.2.2) NetCDF.Native.variable_load(%NetCDF.File{resource: #Reference<0.1547288168.4283564033.94091>, filename: "/Users/christoph/Projects/livebook/nowcast_202307030230.nc", variables: ["FX", "FF", "DD", "RR", "TT", "PT", "TD", "RH", "time", "y", "x", "lambert_conformal_conic", "lat", "lon"]}, "lat")
    (netcdf 0.2.2) lib/netcdf/variable.ex:23: NetCDF.Variable.load/2
    /Users/christoph/Projects/livebook/geosphere.livemd#cell:ghrbl4a23w2yw3n3wsxb7xqufg5jfwq7:3: (file)

Here's the ncdump -h of that file:

ncdump -h nowcast_202307030230.nc
netcdf nowcast_202307030230 {
dimensions:
	time = 13 ;
	y = 431 ;
	x = 701 ;
variables:
	int FX(time, y, x) ;
		FX:_FillValue = -999 ;
		FX:units = "m s**-1" ;
		FX:long_name = "10m gust speed" ;
		FX:standard_name = "wind_speed_of_gust" ;
		FX:description = "gust speed 10m above ground" ;
		FX:cell_method = "" ;
		FX:grid_mapping = "lambert_conformal_conic" ;
		FX:esri_pe_string = "PROJCS[\"MGI / Austria Lambert\",GEOGCS[\"MGI\",DATUM[\"Militar_Geographische_Institute\",SPHEROID[\"Bessel 1841\",6377397.155,299.1528128,AUTHORITY[\"EPSG\",\"7004\"]],TOWGS84[577.326,90.129,463.919,5.137,1.474,5.297,2.4232],AUTHORITY[\"EPSG\",\"6312\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4312\"]],PROJECTION[\"Lambert_Conformal_Conic_2SP\"],PARAMETER[\"standard_parallel_1\",49],PARAMETER[\"standard_parallel_2\",46],PARAMETER[\"latitude_of_origin\",47.5],PARAMETER[\"central_meridian\",13.33333333333333],PARAMETER[\"false_easting\",400000],PARAMETER[\"false_northing\",400000],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],AUTHORITY[\"EPSG\",\"31287\"]]" ;
		FX:coordinates = "lon lambert_conformal_conic lat" ;
		FX:scale_factor = 0.01 ;
	int FF(time, y, x) ;
		FF:_FillValue = -999 ;
		FF:units = "m s**-1" ;
		FF:long_name = "10m wind speed" ;
		FF:standard_name = "wind_speed" ;
		FF:description = "wind speed 10m above ground" ;
		FF:cell_method = "" ;
		FF:grid_mapping = "lambert_conformal_conic" ;
		FF:esri_pe_string = "PROJCS[\"MGI / Austria Lambert\",GEOGCS[\"MGI\",DATUM[\"Militar_Geographische_Institute\",SPHEROID[\"Bessel 1841\",6377397.155,299.1528128,AUTHORITY[\"EPSG\",\"7004\"]],TOWGS84[577.326,90.129,463.919,5.137,1.474,5.297,2.4232],AUTHORITY[\"EPSG\",\"6312\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4312\"]],PROJECTION[\"Lambert_Conformal_Conic_2SP\"],PARAMETER[\"standard_parallel_1\",49],PARAMETER[\"standard_parallel_2\",46],PARAMETER[\"latitude_of_origin\",47.5],PARAMETER[\"central_meridian\",13.33333333333333],PARAMETER[\"false_easting\",400000],PARAMETER[\"false_northing\",400000],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],AUTHORITY[\"EPSG\",\"31287\"]]" ;
		FF:coordinates = "lon lambert_conformal_conic lat" ;
		FF:scale_factor = 0.01 ;
	int DD(time, y, x) ;
		DD:_FillValue = -999 ;
		DD:units = "degree" ;
		DD:long_name = "10m wind direction" ;
		DD:standard_name = "wind_from_direction" ;
		DD:description = "wind direction 10m above ground" ;
		DD:cell_method = "" ;
		DD:grid_mapping = "lambert_conformal_conic" ;
		DD:esri_pe_string = "PROJCS[\"MGI / Austria Lambert\",GEOGCS[\"MGI\",DATUM[\"Militar_Geographische_Institute\",SPHEROID[\"Bessel 1841\",6377397.155,299.1528128,AUTHORITY[\"EPSG\",\"7004\"]],TOWGS84[577.326,90.129,463.919,5.137,1.474,5.297,2.4232],AUTHORITY[\"EPSG\",\"6312\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4312\"]],PROJECTION[\"Lambert_Conformal_Conic_2SP\"],PARAMETER[\"standard_parallel_1\",49],PARAMETER[\"standard_parallel_2\",46],PARAMETER[\"latitude_of_origin\",47.5],PARAMETER[\"central_meridian\",13.33333333333333],PARAMETER[\"false_easting\",400000],PARAMETER[\"false_northing\",400000],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],AUTHORITY[\"EPSG\",\"31287\"]]" ;
		DD:coordinates = "lon lambert_conformal_conic lat" ;
		DD:scale_factor = 0.01 ;
	int RR(time, y, x) ;
		RR:_FillValue = -999 ;
		RR:units = "kg m**-2" ;
		RR:long_name = "precipitation sum" ;
		RR:standard_name = "precipitation_amount" ;
		RR:description = "precipitation sum" ;
		RR:cell_method = "time: accum" ;
		RR:grid_mapping = "lambert_conformal_conic" ;
		RR:esri_pe_string = "PROJCS[\"MGI / Austria Lambert\",GEOGCS[\"MGI\",DATUM[\"Militar_Geographische_Institute\",SPHEROID[\"Bessel 1841\",6377397.155,299.1528128,AUTHORITY[\"EPSG\",\"7004\"]],TOWGS84[577.326,90.129,463.919,5.137,1.474,5.297,2.4232],AUTHORITY[\"EPSG\",\"6312\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4312\"]],PROJECTION[\"Lambert_Conformal_Conic_2SP\"],PARAMETER[\"standard_parallel_1\",49],PARAMETER[\"standard_parallel_2\",46],PARAMETER[\"latitude_of_origin\",47.5],PARAMETER[\"central_meridian\",13.33333333333333],PARAMETER[\"false_easting\",400000],PARAMETER[\"false_northing\",400000],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],AUTHORITY[\"EPSG\",\"31287\"]]" ;
		RR:coordinates = "lon lambert_conformal_conic lat" ;
		RR:scale_factor = 0.01 ;
	int TT(time, y, x) ;
		TT:_FillValue = -999 ;
		TT:units = "degree_Celsius" ;
		TT:long_name = "air temperature 2m above ground" ;
		TT:standard_name = "air_temperature" ;
		TT:description = "air temperature 2m above ground" ;
		TT:cell_method = "" ;
		TT:grid_mapping = "lambert_conformal_conic" ;
		TT:esri_pe_string = "PROJCS[\"MGI / Austria Lambert\",GEOGCS[\"MGI\",DATUM[\"Militar_Geographische_Institute\",SPHEROID[\"Bessel 1841\",6377397.155,299.1528128,AUTHORITY[\"EPSG\",\"7004\"]],TOWGS84[577.326,90.129,463.919,5.137,1.474,5.297,2.4232],AUTHORITY[\"EPSG\",\"6312\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4312\"]],PROJECTION[\"Lambert_Conformal_Conic_2SP\"],PARAMETER[\"standard_parallel_1\",49],PARAMETER[\"standard_parallel_2\",46],PARAMETER[\"latitude_of_origin\",47.5],PARAMETER[\"central_meridian\",13.33333333333333],PARAMETER[\"false_easting\",400000],PARAMETER[\"false_northing\",400000],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],AUTHORITY[\"EPSG\",\"31287\"]]" ;
		TT:coordinates = "lon lambert_conformal_conic lat" ;
		TT:scale_factor = 0.01 ;
	int PT(time, y, x) ;
		PT:_FillValue = -999 ;
		PT:units = "" ;
		PT:long_name = "precipitation type" ;
		PT:standard_name = "predominant_precipitation_type_at_surface" ;
		PT:description = "precipitation type" ;
		PT:cell_method = "" ;
		PT:grid_mapping = "lambert_conformal_conic" ;
		PT:esri_pe_string = "PROJCS[\"MGI / Austria Lambert\",GEOGCS[\"MGI\",DATUM[\"Militar_Geographische_Institute\",SPHEROID[\"Bessel 1841\",6377397.155,299.1528128,AUTHORITY[\"EPSG\",\"7004\"]],TOWGS84[577.326,90.129,463.919,5.137,1.474,5.297,2.4232],AUTHORITY[\"EPSG\",\"6312\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4312\"]],PROJECTION[\"Lambert_Conformal_Conic_2SP\"],PARAMETER[\"standard_parallel_1\",49],PARAMETER[\"standard_parallel_2\",46],PARAMETER[\"latitude_of_origin\",47.5],PARAMETER[\"central_meridian\",13.33333333333333],PARAMETER[\"false_easting\",400000],PARAMETER[\"false_northing\",400000],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],AUTHORITY[\"EPSG\",\"31287\"]]" ;
		PT:coordinates = "lon lambert_conformal_conic lat" ;
		PT:scale_factor = 1. ;
	int TD(time, y, x) ;
		TD:_FillValue = -999 ;
		TD:units = "degree_Celsius" ;
		TD:long_name = "dew point temperature 2m above ground" ;
		TD:standard_name = "dew_point_temperature" ;
		TD:description = "dew point temperature 2m above ground" ;
		TD:cell_method = "" ;
		TD:grid_mapping = "lambert_conformal_conic" ;
		TD:esri_pe_string = "PROJCS[\"MGI / Austria Lambert\",GEOGCS[\"MGI\",DATUM[\"Militar_Geographische_Institute\",SPHEROID[\"Bessel 1841\",6377397.155,299.1528128,AUTHORITY[\"EPSG\",\"7004\"]],TOWGS84[577.326,90.129,463.919,5.137,1.474,5.297,2.4232],AUTHORITY[\"EPSG\",\"6312\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4312\"]],PROJECTION[\"Lambert_Conformal_Conic_2SP\"],PARAMETER[\"standard_parallel_1\",49],PARAMETER[\"standard_parallel_2\",46],PARAMETER[\"latitude_of_origin\",47.5],PARAMETER[\"central_meridian\",13.33333333333333],PARAMETER[\"false_easting\",400000],PARAMETER[\"false_northing\",400000],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],AUTHORITY[\"EPSG\",\"31287\"]]" ;
		TD:coordinates = "lon lambert_conformal_conic lat" ;
		TD:scale_factor = 0.01 ;
	int RH(time, y, x) ;
		RH:_FillValue = -999 ;
		RH:units = "%" ;
		RH:long_name = "relative humidity 2m above ground" ;
		RH:standard_name = "relative_humidity" ;
		RH:description = "relative humidity 2m above ground" ;
		RH:cell_method = "" ;
		RH:grid_mapping = "lambert_conformal_conic" ;
		RH:esri_pe_string = "PROJCS[\"MGI / Austria Lambert\",GEOGCS[\"MGI\",DATUM[\"Militar_Geographische_Institute\",SPHEROID[\"Bessel 1841\",6377397.155,299.1528128,AUTHORITY[\"EPSG\",\"7004\"]],TOWGS84[577.326,90.129,463.919,5.137,1.474,5.297,2.4232],AUTHORITY[\"EPSG\",\"6312\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4312\"]],PROJECTION[\"Lambert_Conformal_Conic_2SP\"],PARAMETER[\"standard_parallel_1\",49],PARAMETER[\"standard_parallel_2\",46],PARAMETER[\"latitude_of_origin\",47.5],PARAMETER[\"central_meridian\",13.33333333333333],PARAMETER[\"false_easting\",400000],PARAMETER[\"false_northing\",400000],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],AUTHORITY[\"EPSG\",\"31287\"]]" ;
		RH:coordinates = "lon lambert_conformal_conic lat" ;
		RH:scale_factor = 0.01 ;
	double time(time) ;
		time:_FillValue = NaN ;
		time:standard_name = "time" ;
		time:long_name = "time of measurement" ;
		time:units = "seconds since 1961-01-01" ;
		time:calendar = "gregorian" ;
	int y(y) ;
		y:standard_name = "projection_y_coordinate" ;
		y:long_name = "y coordinate of projection" ;
		y:axis = "Y" ;
		y:units = "m" ;
	int x(x) ;
		x:standard_name = "projection_x_coordinate" ;
		x:long_name = "x coordinate of projection" ;
		x:axis = "X" ;
		x:units = "m" ;
	double lambert_conformal_conic ;
		lambert_conformal_conic:Northernmost_Northing = 620500LL ;
		lambert_conformal_conic:Southernmost_Northing = 189500LL ;
		lambert_conformal_conic:Easternmost_Easting = 720500LL ;
		lambert_conformal_conic:Westernmost_Easting = 19500LL ;
		lambert_conformal_conic:crs_wkt = "PROJCS[\"MGI / Austria Lambert\",GEOGCS[\"MGI\",DATUM[\"Militar_Geographische_Institute\",SPHEROID[\"Bessel 1841\",6377397.155,299.1528128,AUTHORITY[\"EPSG\",\"7004\"]],TOWGS84[577.326,90.129,463.919,5.137,1.474,5.297,2.4232],AUTHORITY[\"EPSG\",\"6312\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4312\"]],PROJECTION[\"Lambert_Conformal_Conic_2SP\"],PARAMETER[\"standard_parallel_1\",49],PARAMETER[\"standard_parallel_2\",46],PARAMETER[\"latitude_of_origin\",47.5],PARAMETER[\"central_meridian\",13.33333333333333],PARAMETER[\"false_easting\",400000],PARAMETER[\"false_northing\",400000],UNIT[\"metre\",1,AUTHORITY[\"EPSG\",\"9001\"]],AUTHORITY[\"EPSG\",\"31287\"]]" ;
		lambert_conformal_conic:GeoTransform = "19500 1000.000000 189500 1000.000000" ;
		lambert_conformal_conic:grid_mapping_name = "lambert_conformal_conic" ;
		lambert_conformal_conic:standard_parallel = 49., 46. ;
		lambert_conformal_conic:latitude_of_projection_origin = 47.5 ;
		lambert_conformal_conic:longitude_of_central_meridian = 13.3333333333333 ;
		lambert_conformal_conic:false_easting = 400000. ;
		lambert_conformal_conic:false_northing = 400000. ;
		lambert_conformal_conic:units = "" ;
	float lat(y, x) ;
		lat:_FillValue = NaNf ;
		lat:units = "degrees_north" ;
		lat:long_name = "latitude" ;
		lat:standard_name = "latitude" ;
	float lon(y, x) ;
		lon:_FillValue = NaNf ;
		lon:units = "degrees_east" ;
		lon:long_name = "longitude" ;
		lon:standard_name = "longitude" ;

// global attributes:
		:name = "INCAPlus" ;
		:grid_mapping = "lambert_conformal_conic" ;
		:Conventions = "CF-1.7" ;
		:title = "INCAPlus Analysis and Forecast" ;
		:institution = "GeoSphere Austria" ;
		:source = "modeled data" ;
		:references = "https://www.geosphere.at" ;
		:author = "" ;
		:history = "Original data produced by Geosphere Austria" ;
		:comment = "No additional comments" ;
		:freq = "15T" ;
		:forecast_freq = "15T" ;
		:spatial_domain = "INCAPlus_15m_PSI" ;
		:spatial_resolution = 1000. ;
		:crs = "EPSG:31287" ;
}

I'm fairly new to Elixir (even newer to NIFs), but I'll try digging in to see what's happening here.
@clupprich
Copy link
Author

Small update: loading the values of that variable works fine:

{:ok, variable} = NetCDF.Native.variable_values(file, "lon")
{:ok,
 {[8.468606948852539, 8.481375694274902, 8.494144439697266, 8.506914138793945, 8.519682884216309,
   8.532452583312988, 8.545222282409668, 8.557992935180664, 8.57076358795166, 8.583534240722656,
   8.596304893493652, 8.609076499938965, 8.621847152709961, 8.63461971282959, 8.647391319274902,
   8.660163879394531, 8.67293643951416, 8.685708999633789, 8.698481559753418, 8.711255073547363,
   8.724028587341309, 8.736802101135254, 8.749576568603516, 8.762350082397461, 8.775124549865723,
   8.7878999710083, 8.800674438476562, 8.81344985961914, 8.826225280761719, 8.839000701904297,
   8.851777076721191, 8.864553451538086, 8.87732982635498, 8.890106201171875, 8.902883529663086,
   8.91565990447998, 8.928437232971191, 8.941215515136719, 8.95399284362793, 8.966771125793457,
   8.979549407958984, 8.992328643798828, 9.005106925964355, 9.0178861618042, 9.030665397644043,
   9.043444633483887, 9.056224822998047, ...], :f32}}

but loading the attributes raises the argument error:

{:ok, variable} = NetCDF.Native.variable_attributes(file, "lon")
** (ArgumentError) argument error
    (netcdf 0.2.2) NetCDF.Native.variable_attributes(%NetCDF.File{resource: #Reference<0.1547288168.4283564033.96725>, filename: "/Users/christoph/Projects/livebook/nowcast_202307030230.nc", variables: ["FX", "FF", "DD", "RR", "TT", "PT", "TD", "RH", "time", "y", "x", "lambert_conformal_conic", "lat", "lon"]}, "lon")
    /Users/christoph/Projects/livebook/geosphere.livemd#cell:ghrbl4a23w2yw3n3wsxb7xqufg5jfwq7:3: (file)

@clupprich
Copy link
Author

I could trace the error down to the attributes argument in this block here:

netcdf/native/ex_netcdf/src/lib.rs

Lines 169 to 174 in 44bb77b

Ok(NetCDFVariable::new(
variable_name.to_string(),
values,
value_type,
attributes,
))

As it seems the call to NetCDFVariable::new fails for whatever reason with the attributes of that variable. When I replace attributes with any empty vector (vec![]), things are working fine.

@clupprich
Copy link
Author

Oh, I found it: because there's no representation of NaN in BEAM, we can't map a _FillValue of NaNf. Any advice on how to solve this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@clupprich