bad poncho import error message in Task Vine

[benclifford]

Describe the bug
Trying to use Task Vine serverless mode, I get an error: "The poncho module is not available. Cannot create library {name}."

On deeper investigation, this is actually masking a different error, that conda_pack is not importable - see the below stack trace from lightly hacked code.

This is misleading to users who do have poncho available and are trying to debug this.

Process TaskVine-Submit-Process:
Traceback (most recent call last):
  File "/home/benc/parsl/src/cctools/I/lib/python3.12/site-packages/ndcctools/taskvine/manager.py", line 891, in create_library_from_functions
    from ndcctools.poncho import package_serverize
  File "/home/benc/parsl/src/cctools/I/lib/python3.12/site-packages/ndcctools/poncho/package_serverize.py", line 9, in <module>
    from ndcctools.poncho import package_create as create
  File "/home/benc/parsl/src/cctools/I/lib/python3.12/site-packages/ndcctools/poncho/package_create.py", line 12, in <module>
    import conda_pack
ModuleNotFoundError: No module named 'conda_pack'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.12/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/benc/parsl/src/parsl/parsl/process_loggers.py", line 26, in wrapped
    r = func(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/home/benc/parsl/src/parsl/parsl/executors/taskvine/manager.py", line 248, in _taskvine_submit_wait
    serverless_lib = m.create_library_from_functions('common-parsl-taskvine-lib',
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/benc/parsl/src/cctools/I/lib/python3.12/site-packages/ndcctools/taskvine/manager.py", line 893, in create_library_from_functions
    raise ModuleNotFoundError(f"The poncho module is not available. Cannot create library {name}.") from ie
ModuleNotFoundError: The poncho module is not available. Cannot create library common-parsl-taskvine-lib.

cc @tphung3 @dthain

To Reproduce
install task vine without conda pack. try using serverless mode.

Environment
my laptop

[dthain]

Hmm, I wonder what the precise installation method is here. conda install ndcctools includes conda-pack as a dependency. Was this installed in some other way?

@BarrySlyDelgado is Mr Poncho. Do you have any other ideas?

[benclifford]

yeah it's my dev source build!

if i pip install conda_pack then I get the right behaviour and can run stuff - so it's really the phrasing rather than any fundamental failure here...

[dthain]

So maybe we should not try so hard to catch the error, and just let the import fail in the normal way. There was some prior reason we did it that way, but the reason eludes me...

[benclifford]

yeah i think that would be good. there are other places in parsl where we have done an import and made the assumption that the import failed must mean that its because that particular dependency is not installed -- rather than accept that there could be all sorts of reasons.

[benclifford]

alternatively, what I did to diagnose above was to write this:

catch ImportError as ie
raise ModuleNotFoundError(f"The poncho module is not available. Cannot create library {name}.") from ie

which makes the stack trace contain the original exception rather than masking it.

I'm not super fussed either way, and if you don't get round to doing anything here, i'll probably commit that (with a message rephase) when I'm feeling like a simple PR...

[dthain]

cooperative-computing-lab/cctools#3893

[BarrySlyDelgado]

We now forgo catching the import exception when creating a library with the change introduced in cooperative-computing-lab/cctools#3907