Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError around hvplot.xarray (grimpfunc) within jupyter lab #1

Open
tbartholomaus opened this issue Nov 29, 2022 · 5 comments
Open

Comments

@tbartholomaus
Copy link

Hi Ian,

I'm somewhat loath to submit another issue here after my first, but now that I've got everything up and running on a machine at home (macbook pro 2016 intel chip) and my linux machine at work (CentOS 7), I really think this might be an actual bug.

I've installed the same greenlandMapping environment on my 2021 M1 Pro macbook pro, and proceeded without issue. At a python console, I have no problem importing grimpfunc:

(greenlandMapping) tbartholomaus@GEO-D618311 ~ % python
Python 3.9.15 | packaged by conda-forge | (main, Nov 22 2022, 08:55:37)
[Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import grimpfunc
>>>

However, when I do the same within jupyter lab, I get a fairly long set of errors, despite having run python -m ipykernel install --user --name=greenlandMapping several times, both with and without the greenlandMapping environment activated.

Within jupyter lab, with the greenlandMapping environment active and sitting idle (I can, for example, successfully import dask, geopandas, rioxarray, etc.), when I try and import grimpfunc I get an error with the boxPicker submodule that stems from the internal call to import hvplot.xarray

I'm copying this error below, and also attaching two screenshots, if they offer any more context.

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In [1], line 1
----> 1 import grimpfunc

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/grimpfunc/__init__.py:4
      1 __all__ = ['boxPicker', 'cmrUrls', 'Flowlines', 'get_urls', 'GrIMPSubsetter',
      2            'NASALogin', 'pointInspector']
----> 4 from grimpfunc.boxPicker import boxPicker
      5 from grimpfunc.cmrUrls import cmrUrls
      6 from grimpfunc.cmr import get_urls

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/grimpfunc/boxPicker.py:11
      9 import panel as pn
     10 import holoviews as hv
---> 11 import hvplot.xarray
     12 import grimpfunc as grimp
     13 import rioxarray

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/hvplot/xarray.py:61
     57     xr.register_dataarray_accessor(interactive)(XArrayInteractive)
     59     post_patch(extension, logo)
---> 61 patch()

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/hvplot/xarray.py:59, in patch(name, interactive, extension, logo)
     56 xr.register_dataset_accessor(interactive)(XArrayInteractive)
     57 xr.register_dataarray_accessor(interactive)(XArrayInteractive)
---> 59 post_patch(extension, logo)

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/hvplot/__init__.py:166, in post_patch(extension, logo)
    164 def post_patch(extension='bokeh', logo=False):
    165     if extension and not getattr(_hv.extension, '_loaded', False):
--> 166         hvplot_extension(extension, logo=logo)

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/pyviz_comms/__init__.py:64, in extension.__new__(cls, *args, **kwargs)
     62 except Exception:
     63     pass
---> 64 return param.ParameterizedFunction.__new__(cls, *args, **kwargs)

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/param/parameterized.py:3654, in ParameterizedFunction.__new__(class_, *args, **params)
   3652 inst = class_.instance()
   3653 inst.param._set_name(class_.__name__)
-> 3654 return inst.__call__(*args,**params)

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/hvplot/utilities.py:45, in hvplot_extension.__call__(self, *args, **params)
     40 def __call__(self, *args, **params):
     41     # importing e.g. hvplot.pandas always loads the bokeh extension.
     42     # so hvplot.extension('matplotlib', compatibility='bokeh') doesn't
     43     # require the user or the code to explicitely load bokeh.
     44     compatibility = params.pop('compatibility', None)
---> 45     super().__call__(*args, **params)
     46     backend = _hv.Store.current_backend
     47     if compatibility in ['matplotlib', 'plotly'] and backend != compatibility:

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/holoviews/ipython/__init__.py:149, in notebook_extension.__call__(self, *args, **params)
    147     completer.completions_sorting_key = self.completions_sorting_key
    148 if not p.allow_jedi_completion and hasattr(IPCompleter, 'use_jedi'):
--> 149     ip.run_line_magic('config', 'IPCompleter.use_jedi = False')
    151 resources = self._get_resources(args, params)
    153 Store.display_formats = p.display_formats

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/IPython/core/interactiveshell.py:2364, in InteractiveShell.run_line_magic(self, magic_name, line, _stack_depth)
   2362     kwargs['local_ns'] = self.get_local_scope(stack_depth)
   2363 with self.builtin_trap:
-> 2364     result = fn(*args, **kwargs)
   2365 return result

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/IPython/core/magics/config.py:163, in ConfigMagics.config(self, s)
    159 from traitlets.config.loader import Config
    160 # some IPython objects are Configurable, but do not yet have
    161 # any configurable traits.  Exclude them from the effects of
    162 # this magic, as their presence is just noise:
--> 163 configurables = sorted(set([ c for c in self.shell.configurables
    164                              if c.__class__.class_traits(config=True)
    165                              ]), key=lambda x: x.__class__.__name__)
    166 classnames = [ c.__class__.__name__ for c in configurables ]
    168 line = s.strip()

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/IPython/core/magics/config.py:164, in <listcomp>(.0)
    159 from traitlets.config.loader import Config
    160 # some IPython objects are Configurable, but do not yet have
    161 # any configurable traits.  Exclude them from the effects of
    162 # this magic, as their presence is just noise:
    163 configurables = sorted(set([ c for c in self.shell.configurables
--> 164                              if c.__class__.class_traits(config=True)
    165                              ]), key=lambda x: x.__class__.__name__)
    166 classnames = [ c.__class__.__name__ for c in configurables ]
    168 line = s.strip()

AttributeError: type object 'CommManager' has no attribute 'class_traits'

As I mentioned at the top, I'm up and running on other systems, so this isn't urgent, but I thought I'd mention as a potential issue in case this comes up for anyone else, or if this is actually easily addressed or meaningful to you. Without debugging the call to import hvplot.xarray, specifically within a jupyter lab kernel, I'm not sure this is something I can get to the bottom of without a lot of effort.

Thanks again.

Screen Shot 2022-11-28 at 5 12 15 PM
Screen Shot 2022-11-28 at 5 18 21 PM

@fastice
Copy link
Owner

fastice commented Nov 29, 2022 via email

@fastice
Copy link
Owner

fastice commented Nov 29, 2022 via email

@tbartholomaus
Copy link
Author

Thanks for jumping right back into this, Ian. Importing grimpfunc in both M1 and linux machines, via jupyter lab all works great. But it looks like unpinning some of those packages might have led to other problems for the mac- looks like a gdal problem.

In a cell from your workbook that had previously worked fine

myTerminusUrls = grimp.cmrUrls(mode='terminus')  # mode image restricts search to the image products
myTerminusUrls.initialSearch();

myTermini = {}
for url in myTerminusUrls.getURLS():
    year = os.path.basename(url).split('_')[1]  # Extract year from name
    myTermini[year] = gpd.read_file(f'/vsicurl/&url={url}')  # Add terminus to data frame
    print(f'/vsicurl/&url={url}')

I now get the following error on the gpd.read_file from url. But! only on mac. On my linux this runs just fine.
Here's the complete error message:

---------------------------------------------------------------------------
CPLE_OpenFailedError                      Traceback (most recent call last)
File fiona/_shim.pyx:83, in fiona._shim.gdal_open_vector()

File fiona/_err.pyx:291, in fiona._err.exc_wrap_pointer()

CPLE_OpenFailedError: '/vsicurl/&url=https://n5eil01u.ecs.nsidc.org/DP4/MEASURES/NSIDC-0642.002/1999.10.13/termini_1999_2000_v02.0.shp' not recognized as a supported file format.

During handling of the above exception, another exception occurred:

DriverError                               Traceback (most recent call last)
Cell In [18], line 4
      2 for url in myTerminusUrls.getURLS():
      3     year = os.path.basename(url).split('_')[1]  # Extract year from name
----> 4     myTermini[year] = gpd.read_file(f'/vsicurl/&url={url}')  # Add terminus to data frame
      5     print(f'/vsicurl/&url={url}')

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/geopandas/io/file.py:259, in _read_file(filename, bbox, mask, rows, engine, **kwargs)
    256     path_or_bytes = filename
    258 if engine == "fiona":
--> 259     return _read_file_fiona(
    260         path_or_bytes, from_bytes, bbox=bbox, mask=mask, rows=rows, **kwargs
    261     )
    262 elif engine == "pyogrio":
    263     return _read_file_pyogrio(
    264         path_or_bytes, bbox=bbox, mask=mask, rows=rows, **kwargs
    265     )

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/geopandas/io/file.py:303, in _read_file_fiona(path_or_bytes, from_bytes, bbox, mask, rows, where, **kwargs)
    300     reader = fiona.open
    302 with fiona_env():
--> 303     with reader(path_or_bytes, **kwargs) as features:
    304         crs = features.crs_wkt
    305         # attempt to get EPSG code

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/fiona/env.py:408, in ensure_env_with_credentials.<locals>.wrapper(*args, **kwargs)
    405 @wraps(f)
    406 def wrapper(*args, **kwargs):
    407     if local._env:
--> 408         return f(*args, **kwargs)
    409     else:
    410         if isinstance(args[0], str):

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/fiona/__init__.py:264, in open(fp, mode, driver, schema, crs, encoding, layer, vfs, enabled_drivers, crs_wkt, **kwargs)
    261     path = parse_path(fp)
    263 if mode in ('a', 'r'):
--> 264     c = Collection(path, mode, driver=driver, encoding=encoding,
    265                    layer=layer, enabled_drivers=enabled_drivers, **kwargs)
    266 elif mode == 'w':
    267     if schema:
    268         # Make an ordered dict of schema properties.

File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/fiona/collection.py:162, in Collection.__init__(self, path, mode, driver, schema, crs, encoding, layer, vsi, archive, enabled_drivers, crs_wkt, ignore_fields, ignore_geometry, **kwargs)
    160 if self.mode == 'r':
    161     self.session = Session()
--> 162     self.session.start(self, **kwargs)
    163 elif self.mode in ('a', 'w'):
    164     self.session = WritingSession()

File fiona/ogrext.pyx:540, in fiona.ogrext.Session.start()

File fiona/_shim.pyx:90, in fiona._shim.gdal_open_vector()

DriverError: '/vsicurl/&url=https://n5eil01u.ecs.nsidc.org/DP4/MEASURES/NSIDC-0642.002/1999.10.13/termini_1999_2000_v02.0.shp' not recognized as a supported file format.

Again, this is only a problem on mac (2021 M1 MBP), not on linux, and it looks like, from a comment in the former version of environment.yml that this might be a known issue?

I'm copying below a section of conda list from "identical" environments on both machines.

Mac (2021 M1) Linux
fiona                     1.8.22          py310h5a14c0a_0    conda-forge
flit-core                 3.8.0              pyhd8ed1ab_0    conda-forge
folium                    0.13.0             pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.1               h5bb23bf_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.38.0          py310h90acd4f_1    conda-forge
freetype                  2.12.1               h3f81eb7_1    conda-forge
freexl                    1.0.6                hb7f2c08_1    conda-forge
frozenlist                1.3.3           py310h90acd4f_0    conda-forge
fsspec                    2022.11.0          pyhd8ed1ab_0    conda-forge
future                    0.18.2             pyhd8ed1ab_6    conda-forge
gdal                      3.5.2           py310he22d2f2_3    conda-forge
geopandas                 0.12.1             pyhd8ed1ab_1    conda-forge
geopandas-base            0.12.1             pyha770c72_1    conda-forge
geos                      3.11.0               hb486fe8_0    conda-forge
geotiff                   1.7.1                ha1a2aeb_3    conda-forge
geoviews                  1.9.5              pyhd8ed1ab_0    conda-forge
geoviews-core             1.9.5              pyha770c72_0    conda-forge
gettext                   0.21.1               h8a4c099_0    conda-forge
giflib                    5.2.1                hbcb3906_2    conda-forge
glib                      2.74.1               hbc0c0cd_1    conda-forge
glib-tools                2.74.1               hbc0c0cd_1    conda-forge
grimpfunc                 0.0.4                    pypi_0    pypi
grimpqgis                 0.0.1                    pypi_0    pypi
fiona                     1.8.22                   pypi_0    pypi
flit-core                 3.8.0              pyhd8ed1ab_0    conda-forge
folium                    0.13.0             pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.1               hc2a2eb6_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.38.0                   pypi_0    pypi
freetype                  2.12.1               hca18f0e_0    conda-forge
freexl                    1.0.6                h166bdaf_1    conda-forge
frozenlist                1.3.3                    pypi_0    pypi
fsspec                    2022.11.0          pyhd8ed1ab_0    conda-forge
future                    0.18.2             pyhd8ed1ab_6    conda-forge
gdal                      3.5.2                    pypi_0    pypi
geopandas                 0.12.1             pyhd8ed1ab_1    conda-forge
geopandas-base            0.12.1             pyha770c72_1    conda-forge
geos                      3.11.0               h27087fc_0    conda-forge
geotiff                   1.7.1                h4fc65e6_3    conda-forge
geoviews                  1.9.5              pyhd8ed1ab_0    conda-forge
geoviews-core             1.9.5              pyha770c72_0    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
giflib                    5.2.1                h36c2ea0_2    conda-forge
glib                      2.74.1               h6239696_1    conda-forge
glib-tools                2.74.1               h6239696_1    conda-forge
grimpfunc                 0.0.4                    pypi_0    pypi
grimpqgis                 0.0.1                    pypi_0    pypi

@fastice
Copy link
Owner

fastice commented Nov 29, 2022 via email

@tbartholomaus
Copy link
Author

I think your read on this must have been right. I tried yesterday afternoon, and again this morning, and it's working no problem. Kind of unfortunate that this is the way NSIDC declines access attempts, but it's good to know now. I think I'm in business!

Thanks again, Ian. Really, your vision of and support for community/open science is just great. Very much appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants