Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WORKSPACE] How can we install dependencies needed only #8837

Open
MatthD opened this issue Apr 26, 2024 · 8 comments
Open

[WORKSPACE] How can we install dependencies needed only #8837

MatthD opened this issue Apr 26, 2024 · 8 comments

Comments

@MatthD
Copy link

MatthD commented Apr 26, 2024

Description

We have a project with multi app/components

In the end we want to only install deps needed by one package in a Dockerfile .
To have the simplest dev Experience we have pu all the deps of all app/package inside workspace.jsonc root dependencies (not specified for a component)

So in our Dockerfile we does not mount the bitmap, but we do bit init; bit add $component

But doing bit install install all the dependencies inside workspace.jsonc

Thanks to bit show $component we know which package must be installed for a package But is there a way to only install the one really needed for our production build ? (to resume install the needed package of available apps an no more (like from what return bit show app) with version specified inside workspace.jsonc)

Thanks

Specifications

  • Bit version: 1.6.126
  • Node version: 20.11.0
  • npm / yarn version: 10.2.4
  • Platform: Mac0s 14.4.1 arm M2
  • Bit compiler (include version): pnpm
@davidfirst
Copy link
Member

Can you elaborate more on the use-case why all these dependencies are in workspace.jsonc although they're not in use?
In general, I'm not aware of such an option to ignore part of workspace.jsonc and install only the ones that are used by the components.
What you might want to do is the opposite - remove all dependencies from workspace.jsonc so then bit install searches for the dependencies from the components only. (you'll need --add-missing-deps if those deps are newly introduced)

@MatthD
Copy link
Author

MatthD commented Apr 29, 2024

The idea is that we are trying to build the best dev experience about workspace management for our team.
ATM bit show app. and bit status help a lot to identify what dependencies are needed or missing.
We don't want to manage manually by app/component the deps (and forgot to remove then etc..), because BIT can do it for us via git show (identified what is needed and missing) so that we have them globally inside workspace.jsonc .

The solution you do propose could be cool but how could we indicate that we want a specific version of dependencies ? We couldn't just get the latest one available ;) (and aliases etc)

We thought about something that bit can install only app/component we would want to limit the dependencies install for build production for eg.

@davidfirst
Copy link
Member

So if I understand correctly, you have like a pool of dependencies with specific versions that you want bit install to use.
This way, when a component needs one of these dependencies, it'll find the version in that pool instead of using the latest.
Specifying them in workspace.jsonc doesn't work for you because then all of them get installed and you want to install only the dependencies you use.

@GiladShoham @zkochan , maybe you guys have an idea.

@GiladShoham
Copy link
Member

You can do something like create a temp workspace and bring only the app component into it (bit import it). Then it will only install its dependencies.
Or you can even make a temp folder (non even bit workspace) and just npm install the app component itself.

@MatthD
Copy link
Author

MatthD commented Apr 30, 2024

Thanks for the suggestions !

  1. About the create a temp workspace and bring only the app component into it (bit import it) Yes but how to do that because we don't have package.json/lock per component for deps, they are only managed by our workspace.jsonc. And we need specific versions of dependencies, some with aliases ... So don't know how to not have the workspace.jsonc
  2. Not really possible with npm install because we don't want to manage our component manually via npm (we removed package.lock and deps inside package.json to benefit of bit dependencies detection !)

For the moment what I am trying on is:

  1. bit install the full deps
  2. Call a script that will get deps from bit show ${app} and generate the list of dependencies necessary (with sub-deps nedeed by all local packages)
  3. The script will filter dev dependencies and not use dependencies
  4. Export the list inside the property dependencies of workspace.jsonc
  5. Remove node-modulesand lock file,
  6. Reinstall

I think that this way I am cleaning the useless element for production build and have the minimum required.
Inside the script I am using chlid_process of node to call bit cli , I would have prefer to use the API of @teambit, do you guys can indicate if there is a way to call this command bit show/bit install.... programatically?

@GiladShoham
Copy link
Member

  1. when you create a new workspace and import the app component into it, the app component itself contains all the dependencies it needs. so you don't need to have any dependencies in the workspace.jsonc. they will be just installed correctly during the bit import process. if you do want to change some versions by force, you can run the bit show on the real workspace, then copy some dependencies to the policy in the workspace.jsonc of the temp one and modify the versions. (or even set overrides in the temp workspace)
  2. I was talking about running npm install on a temp empty folder, so in that case any way you don't need bit dependency detection. it's only a temp folder of your app for running/deploying it.

Your method that you are trying to do now, will not work.
even if you remove the dependencies from the policy in the worksapce.jsonc, when you do bit install, bit will recognize all dependencies used by the components in the workspace, and will install them.

@MatthD
Copy link
Author

MatthD commented May 4, 2024

Thanks for this explication !
In fact in devv we have all the app with their composent installed at the sale time.
For prod because we want the smalest build possible and have micro services build (but still use à monorepo) we are copying one app and the with a script and the help of bit show and version specific of out workspace we are keeping only what we need.
Will try to provide you a veru simple repo with the script

@MatthD
Copy link
Author

MatthD commented Jun 10, 2024

So here is a script I did for our project

  • it removed not needed deps based on bit show + dev deps
  • It update workspace.jsonc to keep only needed packages for a specific app
  • It list the sub deps of related packages, that means I need to first do a bit install , then run the script, then bit install

In the end by running you should get only needed deps for an app and re-running bit install you will get only the needed deps and sub-deps (of packages)

npx tsx scripts/clean-prod-deps.ts -p front

                "pino-pretty": "npm:[email protected]",
                "commander": "^12.0.0",
                "chalk": "^5.3.0",
                "redis": "^2.8.0",
                "redislatest": "npm:redis@latest",
                "twilio": "^4.11.2",
                "uuid": "^9.0.1",
                "wrtc": "npm:@avahq/[email protected]"

It's usefull because in the end it prevent you to have one package.json per apps/packages but manage them globally and in a prod container only get the needed deps per app , that means you can have a monorepo workspace with micro service

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants