Lazy processor import

[gipert]

Since now the pre-compilation takes really long, maybe we could make it lazy (e.g. pre-compile a processor only when actually requested)?

[iguinn]

Numba is capable of this with JIT compiling, but since we are using guvectorize and being very specific about function signatures, I can't find a way to do this with numba specifically.

There is this package: https://pypi.org/project/lazy-imports/. It seems actively maintained, but it would be adding a dependency.

One big risk to lazy importing is that the compilation could happen concurrently (this was causing a bug before with snakemake which we fixed by making sure to compile things before the dsp workflow got started). Another option could be to have compilation happen during setup; this would mean installation would take much longer, but importing wouldn't (unless you are actively developing). It would also make it so that the cached functions would be installed in the global site-packages in our containers, which might be a good thing.