From c380eb5647ec1bc2acccdf6936ab4aa5f8553d3e Mon Sep 17 00:00:00 2001 From: jdx <216188+jdx@users.noreply.github.com> Date: Wed, 27 Nov 2024 09:19:58 -0600 Subject: [PATCH] docs: document aqua better --- docs/dev-tools/backends/aqua.md | 51 +++++++++++++++++++++++++-- docs/dev-tools/backends/asdf.md | 27 ++++----------- docs/dev-tools/backends/ubi.md | 61 +++++++++++++++++++++++++++++++-- docs/registry.md | 16 +++++++++ scripts/render-registry.js | 16 +++++++++ 5 files changed, 146 insertions(+), 25 deletions(-) diff --git a/docs/dev-tools/backends/aqua.md b/docs/dev-tools/backends/aqua.md index 2349c81b9..1b877ac65 100644 --- a/docs/dev-tools/backends/aqua.md +++ b/docs/dev-tools/backends/aqua.md @@ -1,8 +1,25 @@ # Aqua Backend -[Aqua](https://aquaproj.github.io/) tools may be used natively in mise. Aqua is encouraged as a backend for new tools if they -cannot be used with ubi as aqua tools directly fetch tarballs from the vendor without requiring unsafe -code execution in a plugin. +[Aqua](https://aquaproj.github.io/) tools may be used natively in mise. aqua is the ideal backend +to use for new tools since they don't require plugins, they work on windows, they offer security +features like cosign/slsa verification in addition to checksums. aqua installs also show more progress +bars, which is nice. + +You do not need to separately install aqua. The aqua CLI is not used in mise at all. What is used is +the [aqua registry](https://github.com/aquaproj/aqua-registry) which is a bunch of yaml files that get compiled into the mise binary on release. +Here's an example of one of these files: [`aqua:hashicorp/terraform`](https://github.com/aquaproj/aqua-registry/blob/main/pkgs/hashicorp/terraform/registry.yaml). +mise has a reimplementation of aqua that knows how to work with these files to install tools. + +As of this writing, aqua is relatively new to mise and because a lot of tools are being converted from +asdf to aqua, there may be some configuration in aqua tools that need to be tightened up. I put some +common issues below and would strongly recommend contributing changes back to the aqua registry if you +notice problems. The maintainer is super responsive and great to work with. + +If all else fails, you can disable aqua entirely with [`MISE_DISABLE_BACKENDS=aqua`](/configuration/settings.html#disable_backends). + +Currently aqua plugins don't support setting environment variables or doing more than simply downloading +binaries though (and I'm not sure this functionality would ever get added), so some tools will likely +always require plugins like asdf/vfox. The code for this is inside the mise repository at [`./src/backend/aqua.rs`](https://github.com/jdx/mise/blob/main/src/backend/aqua.rs). @@ -32,3 +49,31 @@ to use the aqua backend. To see these tools, run `mise registry | grep aqua:`. import Settings from '/components/settings.vue'; + +## Common aqua issues + +Here's some common issues I've seen when working with aqua tools. + +### Supported env missing + +The aqua registry defines supported envs for each tool of the os/arch. I've noticed some of these +are simply missing os/arch combos that are in fact supported—possibly because it was added after +the registry was created for that tool. + +The fix is simple, just edit the `supported_envs` section of `registry.yaml` for the tool in question. + +### Using `version_filter` instead of `version_prefix` + +This is a weird one that doesn't causes weird issues in mise. In general in mise we like versions like +`1.2.3` with no decoration like `v1.2.3` or `cli-v1.2.3`. This consistency not only makes `mise.toml` +cleaner but, it also helps make things like `mise up` function right because it's able to parse it as +semver without dealing with a bunch of edge-cases. + +Really if you notice aqua tools are giving you versions that aren't simple triplets, it's worth fixing. + +One common thing I've seen is registries using a `version_filter` expression like `Version startsWith "Version startsWith "atlascli/""`. + +This ultimately causes the version to be `atlascli/1.2.3` which is not what we want. The fix is to use +`version_prefix` instead of `version_filter` and just put the prefix in the `version_prefix` field. +In this example, it would be `atlascli/`. mise will automatically strip this out and add it back in, +which it can't do with `version_filter`. diff --git a/docs/dev-tools/backends/asdf.md b/docs/dev-tools/backends/asdf.md index c9e2f364b..80ef19f42 100644 --- a/docs/dev-tools/backends/asdf.md +++ b/docs/dev-tools/backends/asdf.md @@ -1,25 +1,12 @@ # asdf Backend -asdf is the original backend for mise. It's the default if no backend is specified, -e.g.: `mise i ripgrep` will use [asdf](https://gitlab.com/wt0f/asdf-ripgrep) but `mise i cargo:ripgrep` -will use the [cargo](./cargo) backend. You can explicitly specify the asdf backend with `mise i asdf:ripgrep`. -If you wish. - -If choosing a backend to integrate a tool into mise, it's discouraged to use the asdf backend. ubi -would be the ideal choice if it can work as a single binary, otherwise aqua would be the next best choice -since it requires minimal configuration and doesn't require executing code in a plugin. Generally -vfox plugins can handle anything an asdf plugin might need to do while also being potentially able -to support windows. - -There are [hundreds of plugins](https://github.com/mise-plugins/registry) available in the -[mise registry](https://github.com/mise-plugins) and you can also install plugins from git -repos or local directories. - -::: warning -Take care using plugins as well as anything else you get from the internet. CLIs are -unfortunately capable of doing a lot of damage to your system if a bad actor manages to -get into your system through a plugin or other tool. -::: +asdf is the original backend for mise. It relies on asdf plugins for each tool. asdf plugins are +more risky to use because they're typically written by a single developer unrelated to the tool vendor +they also do not function on Windows. +Because of the extra complexity of asdf tools and security concerns we are actively moving tools in +the registry away from asdf where possible to backends like aqua and ubi which don't require plugins. +That said, not all tools can function with ubi/aqua if they have a unique installation process or +need to set env vars other than PATH. ## Writing asdf plugins for mise diff --git a/docs/dev-tools/backends/ubi.md b/docs/dev-tools/backends/ubi.md index 4d352cc56..62ecc34ab 100644 --- a/docs/dev-tools/backends/ubi.md +++ b/docs/dev-tools/backends/ubi.md @@ -1,8 +1,12 @@ # Ubi Backend You may install GitHub Releases and URL packages directly using [ubi](https://github.com/houseabsolute/ubi) backend. ubi is directly compiled into -the mise codebase so it does not need to be installed separately to be used. ubi is the preferred backend when it functions for tools as it is the -simplest and requires minimal configuration. +the mise codebase so it does not need to be installed separately to be used. ubi is preferred over +asdf/vfox for new tools since it doesn't require a plugin, supports Windows, and is really easy to use. + +ubi doesn't require plugins or even any configuration for each tool. What it does is try to deduce what +the proper binary/tarball is from GitHub releases and downloads the right one. As long as the vendor +uses a somewhat standard labeling scheme for their releases, ubi should be able to figure it out. The code for this is inside of the mise repository at [`./src/backend/ubi.rs`](https://github.com/jdx/mise/blob/main/src/backend/ubi.rs). @@ -59,3 +63,56 @@ then this will be ignored. - **GitHub shorthand for latest release version:** `ubi:goreleaser/goreleaser` - **GitHub shorthand for specific release version:** `ubi:goreleaser/goreleaser@1.25.1` - **URL syntax:** `ubi:https://github.com/goreleaser/goreleaser/releases/download/v1.16.2/goreleaser_Darwin_arm64.tar.gz` + +## Troubleshooting ubi + +### `ubi` resolver can't find os/arch + +Sometimes vendors use strange formats for their releases that ubi can't figure out, possibly for a +specific os/arch combination. For example this recently happend in [this ticket](https://github.com/houseabsolute/ubi/issues/79) because a vendor used +"mac" instead of the more common "macos" or "darwin" tags. + +Try using ubi by itself to see if the issue is related to mise or ubi: + +```sh +ubi -p jdx/mise +./bin/mise -v # yes this technically means you could do `mise use ubi:jdx/mise` though I don't know why you would +``` + +### `ubi` picks the wrong tarball + +Another issue is that a GitHub release may have a bunch of tarballs, some that don't contain the CLI +you want, you can use the `matching` field in order to specify a string to match against the release. + +```sh +mise use ubi:tamasfe/taplo[matching=full] +# or with ubi directly +ubi -p tamasfe/taplo -m full +``` + +### `ubi` can't find the binary in the tarball + +ubi assumes that the repo name is the same as the binary name, however that is often not the case. +For example, BurntSushi/ripgrep gives us a binary named `rg` not `ripgrep`. In this case, you can +specify the binary name with the `exe` field: + +```sh +mise use ubi:BurntSushi/ripgrep[exe=rg] +# or with ubi directly +ubi -p BurntSushi/ripgrep -e rg +``` + +### `ubi` uses weird versions + +This issue is actually with mise and not with ubi. mise needs to be able to list the available versions +of the tools so that "latest" points to whatever is the actual latest release of the CLI. What sometimes +happens is vendors will have GitHub releases for unrelated things. For example, `cargo-bins/cargo-binstall` +is the repo for cargo-binstall, however it has a bunch of releases for unrelated CLIs that are not +cargo-binstall. We need to filter these out and that can be specified with the `tag_regex` tool option: + +```sh +mise use 'ubi:cargo-bins/cargo-binstall[tag_regex=^\d+\.]' +``` + +Now when running `mise ls-remote ubi:cargo-bins/cargo-binstall[tag_regex=^\d+\.]` you should only see +versions starting with a number. Note that this command is cached so you likely will need to run `mise cache clear` first. diff --git a/docs/registry.md b/docs/registry.md index 4c4e25916..60de37a5d 100644 --- a/docs/registry.md +++ b/docs/registry.md @@ -4,6 +4,22 @@ editLink: false # Registry +In general, the preferred backend to use for new tools is the following: + +- [aqua](./dev-tools/backends/aqua.html) - offers the most features and security while not requiring plugins +- [ubi](./dev-tools/backends/ubi.html) - very simple to use +- [pipx](./dev-tools/backends/pipx.html) - only for python tools, requires python to be installed but this generally would always be the case for python tools +- [npm](./dev-tools/backends/npm.html) - only for node tools, requires node to be installed but this generally would always be the case for node tools +- [vfox](./dev-tools/backends/vfox.html) - only for tools that have unique installation requirements or need to modify env vars +- [asdf](./dev-tools/backends/asdf.html) - only for tools that have unique installation requirements or need to modify env vars, doesn't support windows +- [go](./dev-tools/backends/go.html) - only for go tools, requires go to be installed to compile. Because go tools can be distributed as a single binary, aqua/ubi are definitely preferred. +- [cargo](./dev-tools/backends/cargo.html) - only for rust tools, requires rust to be installed to compile. Because rust tools can be distributed as a single binary, aqua/ubi are definitely preferred. + +However, each tool can define its own priority if it has more than 1 backend it supports. You can disable a backend with `mise settings disable_backends=asdf`. +And it will be skipped. See [Aliases](/dev-tools/aliases.html) for a way to set a default backend for a tool. + +You can also specify the full name for a tool using `mise use aqua:1password/cli` if you want to use a specific backend. + | Short | Full | | ----------- | --------------- | | 1password | [asdf:NeoHsu/asdf-1password-cli](https://github.com/NeoHsu/asdf-1password-cli) [aqua:1password/cli](https://github.com/1password/cli) | diff --git a/scripts/render-registry.js b/scripts/render-registry.js index 66b0605f2..7938dc845 100755 --- a/scripts/render-registry.js +++ b/scripts/render-registry.js @@ -15,6 +15,22 @@ editLink: false --- # Registry + +In general, the preferred backend to use for new tools is the following: + +- [aqua](./dev-tools/backends/aqua.html) - offers the most features and security while not requiring plugins +- [ubi](./dev-tools/backends/ubi.html) - very simple to use +- [pipx](./dev-tools/backends/pipx.html) - only for python tools, requires python to be installed but this generally would always be the case for python tools +- [npm](./dev-tools/backends/npm.html) - only for node tools, requires node to be installed but this generally would always be the case for node tools +- [vfox](./dev-tools/backends/vfox.html) - only for tools that have unique installation requirements or need to modify env vars +- [asdf](./dev-tools/backends/asdf.html) - only for tools that have unique installation requirements or need to modify env vars, doesn't support windows +- [go](./dev-tools/backends/go.html) - only for go tools, requires go to be installed to compile. Because go tools can be distributed as a single binary, aqua/ubi are definitely preferred. +- [cargo](./dev-tools/backends/cargo.html) - only for rust tools, requires rust to be installed to compile. Because rust tools can be distributed as a single binary, aqua/ubi are definitely preferred. + +However, each tool can define its own priority if it has more than 1 backend it supports. You can disable a backend with \`mise settings disable_backends=asdf\`. +And it will be skipped. See [Aliases](/dev-tools/aliases.html) for a way to set a default backend for a tool. + +You can also specify the full name for a tool using \`mise use aqua:1password/cli\` if you want to use a specific backend. `, ];