Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Monorepo confusion about editablePackageSources and flake.nix location #1719

Open
drawnwren opened this issue Jun 29, 2024 · 1 comment
Open

Comments

@drawnwren
Copy link

drawnwren commented Jun 29, 2024

I have a monorepo structured like so:

repo/
    services/
        a/
            pyproject.toml
         b/
            pyproject.toml

Where a imports b as editable with b = {path="../b", develop=true} in the pyproject.toml. I'm having trouble getting my nix development derivation to import b correctly into the shell.

I've tried:

  1. a/flake.nix with a derivation like:
        devShells.default = let 
            envShell = mkPoetryEnv {
              projectDir = self;
              overrides = p2n-overrides;
              preferWheels = true;
              editablePackageSources = {
                b = ../b;
              };
            }; 
          in
          pkgs.mkShell {
            name = "A";
            buildInputs = [ envShell ];
            #inputsFrom = [ self.packages.${system}.a ];
            packages = with pkgs; [ poetry docker-compose unzip ruff-lsp stdenv.cc.cc.lib zip awscli2 postgresql];
            LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib";
        };
      });

My understanding was that nix (like docker) copies the entire sub-directory structure, so it might be missing "sibling" folders (i.e. w/ same parent directory)
2. I've also tried putting flake.nix in services/ with something like

        devShells.default = let 
            envShell = mkPoetryEnv {
              projectDir = ./a;
              overrides = p2n-overrides;
              preferWheels = true;
              editablePackageSources = {
                b = ./b;
                a = ./a;
              };
            }; 
          in
          pkgs.mkShell {
            name = "A";
            buildInputs = [ envShell ];
            packages = with pkgs; [ poetry docker-compose unzip ruff-lsp stdenv.cc.cc.lib zip awscli2 postgresql];
            LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib";
        };
      });

But I still get the ModuleNotFoundError for b.

What's the expected way for me to make this structure work?

@drawnwren drawnwren changed the title Monorepo confusion Monorepo confusion about editablePackageSources and flake.nix location Jun 29, 2024
@fmnxl
Copy link
Contributor

fmnxl commented Sep 10, 2024

What worked for me was using the submodule as a flake input:

inputs = {
  my-library = {
    url = "git+file:my-library-dir";
    flake = false;
  };
}

Then setting editablePackageSources to point to it:

outputs = { self, ..., my-library }: {
  # ...
  devShell = mkPoetryEnv {
    editablePackageSources = {
      my-library = my-library;
    };
};

And keep pyproject.toml as if importing the library from pip. No need to set path and develop = true

The downside is I have to run nix flake lock --update-input my-library every time I make a change.

But with how flakes are coupled to the git state, I don't think it'd be possible to import the library without flattening it into the parent's/monorepo git tree. As in, I don't think it's possible for flakes to link to submodules dynamically, as everything is locked by flake.lock.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants