Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improvement: install private packages from hexpm #157

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

zachallaun
Copy link
Collaborator

@zachallaun zachallaun commented Nov 15, 2024

Closes #155.

mix igniter.install org/package

Contributor checklist

  • Bug fixes include regression tests
  • Features include unit/acceptance tests

@zachallaun
Copy link
Collaborator Author

@hypno2000 If you wouldn't mind installing Igniter with {:igniter, github: "ash-project/igniter", ref: "za-install-hex-repo"} and making sure this handles your use-case, I'd greatly appreciate it!

@hypno2000
Copy link

@hypno2000 If you wouldn't mind installing Igniter with {:igniter, github: "ash-project/igniter", ref: "za-install-hex-repo"} and making sure this handles your use-case, I'd greatly appreciate it!

Thanks for that quick response! We need specifically organization (not repo), ie {:my_lib, "~> 1.0", organization: "acme"}

@zachallaun
Copy link
Collaborator Author

Ah, I misunderstood the spec :) I haven't used anything but the default hex orgs/repos before, so forgive my ignorance here. To make sure I understand: :repo describes where the thing is, defaulting to hex.pm, and repos can have many organizations.

So you all have private packages hosted on hex.pm under your organization, and need to install those, whereas others may have private packages hosted on a separate repo with the default or no organization, e.g. Oban Pro.

Assuming that understanding is correct, we may want to support both of these. Since Hex.pm presents private packages as org/package, we should use that slash syntax to mean organization, not repo. Perhaps : becomes the delimiter for repo? So if you were installing Oban Pro, you might use mix igniter.install oban:oban_pro, and if you were installing your organization's package from hosted on Hex, you'd use mix igniter.install my_org/package.

Sound reasonable?

@zachallaun zachallaun changed the title improvement: install from private hex repos with repo/package improvement: install from private hex repos and orgs Nov 15, 2024
@zachallaun
Copy link
Collaborator Author

@hypno2000 Made the changes described above, let me know if they work for you 🙂

@zachallaun zachallaun marked this pull request as draft November 15, 2024 19:48
@zachallaun
Copy link
Collaborator Author

By the way, I'm going to convert this PR to a draft for now because @zachdaniel is on vacation, but I'd like him to sign off on the repo:org/package syntax before merging it in. I believe he's back in action next week, but I want to make sure it doesn't accidentally get merged in before that. :)

@hypno2000
Copy link

hypno2000 commented Nov 15, 2024

The syntax makes sense to me.

So you all have private packages hosted on hex.pm under your organization, and need to install those

yes exactly

right now i am getting same result as before:

** (RuntimeError) Could not determine source for requested package acme/my_lib
    (igniter 0.4.7) lib/igniter/util/install.ex:31: anonymous fn/1 in Igniter.Util.Install.install/4
    (elixir 1.17.3) lib/enum.ex:1703: Enum."-map/2-lists^map/1-1-"/2
    (igniter 0.4.7) lib/igniter/util/install.ex:28: Igniter.Util.Install.install/4
    (mix 1.17.3) lib/mix/task.ex:495: anonymous fn/3 in Mix.Task.run_task/5
    (mix 1.17.3) lib/mix/cli.ex:96: Mix.CLI.run_task/2

i suspect that is because of my local hex auth is not passed on here:
https://github.com/ash-project/igniter/blob/za-install-hex-repo/lib/igniter/project/deps.ex#L342

@zachallaun
Copy link
Collaborator Author

Out of curiosity, does it work if you specify a version? mix igniter.install org/[email protected]

@zachallaun
Copy link
Collaborator Author

And you're definitely right re: the HTTP request. Perhaps we should be using hex_core here.

@hypno2000
Copy link

i am getting this with specific version:

** (ArgumentError) expected a keyword list, but an entry in the list is not a two-element tuple with an atom as its first element, got: {:my_lib, "~> 1.0", [organization: "acme"]}
    (elixir 1.17.3) lib/keyword.ex:656: Keyword.keys/1
    (igniter 0.4.7) lib/igniter/util/info.ex:39: Igniter.Util.Info.compose_install_and_validate!/6
    (igniter 0.4.7) lib/igniter/util/install.ex:77: Igniter.Util.Install.install/4
    (mix 1.17.3) lib/mix/task.ex:495: anonymous fn/3 in Mix.Task.run_task/5
    (mix 1.17.3) lib/mix/cli.ex:96: Mix.CLI.run_task/2

@djgoku
Copy link

djgoku commented Nov 16, 2024

Oh, dang it I just found this after completing my PR. #158

I'll go ahead and close mine.

@zachallaun zachallaun closed this Nov 20, 2024
@zachallaun zachallaun reopened this Nov 20, 2024
@zachallaun zachallaun changed the title improvement: install from private hex repos and orgs improvement: install private packages from hexpm Nov 20, 2024
@zachallaun
Copy link
Collaborator Author

(Oops, didn't mean to close this earlier.)

@hypno2000 I've pushed a commit that I believe should allow you to install packages from organizations now.

For now I've removed support for repos other than hex.pm. Hex is two parts: an API and a Repo. The API has richer metadata about versions, like which versions might be retired; the Repo is just a "dumb" store of tarballs and checksums. It's much easier to host a Repo, which can be as simple as a static file server, than it is the API.

I think repo support could be added back in, we just need to make the request a bit differently, but I want to make sure organization support works correctly first.

@hypno2000
Copy link

it works now when version is specified, but without it still fails due to the auth issue:

** (RuntimeError) Could not determine source for requested package org/package
    (igniter 0.4.7) lib/igniter/util/install.ex:31: anonymous fn/1 in Igniter.Util.Install.install/4
    (elixir 1.17.3) lib/enum.ex:1703: Enum."-map/2-lists^map/1-1-"/2
    (igniter 0.4.7) lib/igniter/util/install.ex:28: Igniter.Util.Install.install/4
    (mix 1.17.3) lib/mix/task.ex:495: anonymous fn/3 in Mix.Task.run_task/5
    (mix 1.17.3) lib/mix/cli.ex:96: Mix.CLI.run_task/2

@zachdaniel
Copy link
Contributor

Is hex_core perhaps just magically available like other local archives? If so that would be a great way to make sure we are doing the right thing here :)

@zachallaun
Copy link
Collaborator Author

Sorry, stalled on this a little bit.

@zachdaniel Sort of, but relying on it might be pretty icky. We do have Hex available (it powers all the mix hex.* tasks), and it vendors in a namespaced version of hex_core.

@zachdaniel
Copy link
Contributor

Interesting. Could we perhaps talk to the folks at hex and find out if any of this could reasonably be leveraged to these ends? Would it be bad primarily because these aren't public interfaces? Or are there other reasons it would be an issue?

@zachallaun
Copy link
Collaborator Author

I did chat with Wojtek when originally working on this, and my impression is that this stuff is rather stable and unlikely to change. He was the one, for instance, who suggested this.

If Hex stopped vendoring in hex_core, the fix is also rather simple: we start depending on it ourselves (hex_core itself has zero dependencies), or we vendor it ourselves (potentially using the same script they use).

@zachdaniel
Copy link
Contributor

Well all that sounds pretty positive to me in that case 😄 I'm happy to rely on it. The bits of code that use this shouldn't be run outside of a mix task, the only thing I could imagine us having to do is to prompt users to install hex and restart in rare scenarios where it hadn't been installed yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

support hex organizations for igniter.install
4 participants