Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement the conda environment builder #5

Open
ctrueden opened this issue Jun 22, 2023 · 3 comments
Open

Implement the conda environment builder #5

ctrueden opened this issue Jun 22, 2023 · 3 comments
Labels
enhancement New feature or request
Milestone

Comments

@ctrueden
Copy link
Member

To build a conda environment, we need to:

  1. download and cache micromamba
  2. use it to build the environment based on a list of conda packages, which should always include the appose project

As a starting point for (1), there is some code here written by @carlosuc3m for bootstrapping a conda setup for use with jep. I would cherry-pick any useful code from there, adapting it to use micromamba instead.

This builder is necessary to fulfill Appose's primary initial use case: invoking DL models from Java, by executing Python in a separate process.

@ctrueden ctrueden added this to the 0.3.0 milestone Jun 22, 2023
@ctrueden ctrueden added the enhancement New feature or request label Jun 22, 2023
@ctrueden
Copy link
Member Author

The album project also has conda/mamba/minimamba-based environment building support. See the source code here.

@ctrueden
Copy link
Member Author

ctrueden commented Nov 7, 2023

On 2023-11-06, @carlosuc3m wrote on Zulip:

I have been working on a fork
https://github.com/carlosuc3m/appose-java/tree/env-builder
I got the env builder to work with micromamba
Micromamba downloads a .tar.bz2 zip so iI had to use apache commons compress to decompress the executable:
https://github.com/carlosuc3m/appose-java/blob/env-builder/src/main/java/org/apposed/appose/MambaInstallerUtils.java
Also it is important to note that the micromamba link does quite a lotof redirecting, so I tried to deal with it with this method:
https://github.com/carlosuc3m/appose-java/blob/93a23355fb7bdfcef6b1945e6315399683a19ff4/src/main/java/org/apposed/appose/MambaInstallerUtils.java#L160

Also in apple and linux the file needs to be set as an executable and sometimes it might encounter security problems with apple, there seems to be a way to circumvent it using a command:

xattr -d com.apple.quarantine /Users/Cgarcia/git/deep-icy/appose/micromamba/bin/micromamba

In theory it bypasses the gatekeeper of Apple.

With these changes I have been able to install micromamba, install envs and run python from micromamba pretty smoothly.

However, I have not been able to use certain options that conda allows such as setting environment variables for a certain env as conda does.

@ctrueden
Copy link
Member Author

ctrueden commented Nov 7, 2023

@carlosuc3m For macOS (and maybe all platforms), I wonder if shipping micromamba with Fiji would alleviate the pain, assuming we finally start shipping a DMG installer for Fiji for macOS (fiji/fiji-builds#1). I believe that copying a .app out of a DMG to a new location causes macOS to clear the quarantine flag on all involved files under the hood. We will need to test this, of course. It may work in our favor with micromamba here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant