Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaging solutions #497

Open
intergrav opened this issue Aug 6, 2023 · 92 comments
Open

Packaging solutions #497

intergrav opened this issue Aug 6, 2023 · 92 comments
Labels
app Relates to Modrinth App enhancement New feature or request

Comments

@intergrav
Copy link
Contributor

intergrav commented Aug 6, 2023

At the time of writing, there are only two official ways for using the Modrinth App on Linux - AppImages and Debian packages. While this is fine for people on Debian-based distributions, and those who like using AppImages, some may prefer to use a package manager such as flatpak or dnf as it's easier to manage packages that way. According to this comment, the current compiled Debian packages aren't very great right now because they don't include proper dependency info and leave out some important stuff.

Along with AppImages and Debian packages, there are various packaging solutions to consider. If there's other packaging types that you think would be a good idea, suggest them in this thread. I personally think that the Flatpak should be maintained by Modrinth, and the rest could be maintained by members of the community if they wanted to do so. I think this is especially important as the Modrinth App is now in an open beta.

This issue serves as a place to discuss packaging, and I'll also treat this as a sort of "unofficial download page" for Linux users with updated information on what packages to use and where to get them.

Indicators

  1. a package exists and works properly
  2. that package is considered ready for daily use
  3. that package is available on commonly used repositories, such as a Flatpak being on Flathub for example

🔃

  1. same as 1 and 2 in the above
  2. however, that package is not available on commonly used repositories, but is available on a developer's personal repository. this may not be applicable for some solutions and will use the checkmark icon instead, such as COPR

Unix/Unix-like packaging

Flathub logo Flatpak

Package is available and maintained on Flathub - thank you @getchoo and other contributors!

Package that can be used on most common Linux distributions, and also good for security as Flatpaks are sandboxed unlike other types. This will be the package that most Linux users are looking for.

Notable mentions/issues
Who's working on this?

Arch

Packages are available and maintained on the AUR - modrinth-app and modrinth-app-bin - thank you Antti!

Any ideas on this?
Notable mentions/issues
Who's working on this?
  • Antti has made two AUR packages
    • modrinth-app
    • modrinth-app-bin
  • Nixuge has made two AUR packages, however they could be broken as they miss many dependencies
    • modrinth-app-git
    • modrinth-cli-git

Nix logo Nix

Package is available and maintained on Nixpkgs - modrinth-app - thank you @getchoo and other contributors!

Nix can be used on both Linux and Darwin systems and is the package manager for NixOS.

Who's working on this?

Fedora logo Fedora Copr

User-maintained package repositories for Fedora, CentOS Stream, and RHEL. Can be enabled with dnf copr enable <author>/<repo>

Notable mentions/issues

Homebrew logo Homebrew cask

Homebrew is a package manager for MacOS (and Linux). Idea from #560. It would allow for easy installation on MacOS.

Windows packaging

🔃 🍨 Scoop

Package is available and maintained on brawaru/bucket - thank you @brawaru!

Scoop is a command-line installer for Windows. It has many benefits, such as installing packages in a portable way (by default, it installs packages to ~/scoop). Package manifests are just simple JSON files - this shouldn't be too hard to set up.

WinGet logo WinGet

Package is available and maintained on WinGet - thank you @TheBossMagnus and other contributors!

WinGet is Microsoft's own package manager.

Other information

Theseus is very complex to package and doing so for many different formats at a time would be a little chaotic. I think that the common package types like Flatpak, Arch and Nix packages should be done.

Leave your thoughts below!

For discussion, see the thread in the #development channel of the Modrinth Discord. You can also continue discussing here if you prefer that.
@intergrav intergrav added the enhancement New feature or request label Aug 6, 2023
@Hyphrio
Copy link

Hyphrio commented Aug 6, 2023

For Linux these two Tauri issues are related:
tauri-apps/tauri#4402
tauri-apps/tauri#3619

@offbeat-stuff
Copy link

At least make a PKGBUILD that extracts the deb package to be used as an archlinux pacman package.

@offbeat-stuff
Copy link

Here is a PKGBUILD for archlinux that extracts a .deb package

https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=linux-xanmod-linux-bin-x64v3

@getchoo
Copy link

getchoo commented Aug 6, 2023

At least make a PKGBUILD that extracts the deb package to be used as an archlinux pacman package.

this isn't a good method for the long term, especially as arch is known to make abi incompatible changes to packages frequently, or at the very least push versions of libraries usually incompatible with those found on debian/ubuntu based distros (there are also problems between those, but usually less so). here's a good example

for a temporary thing it would probable be fine? but ideally all community packages should be built against their distro's packages to ensure the final binaries actually work

@getchoo
Copy link

getchoo commented Aug 6, 2023

I personally think that the Flatpak should be maintained by Modrinth, and the rest could be maintained by members of the community if they wanted to do so.

i 100% agree with this. it's very hard and completely impractical for upstream to support the endless amounts of linux distributions out there; universal solutions should be the main focus here, along with basic files for distribution that can be used there and in downstream packages

Flatpak/Flathub

the biggest issue here is that networking may need to be enabled during building due to flatpak's build tools lacking support for pnpm lock files. this is a problem since flathub doesn't really like that and it could make it very difficult for it to be uploaded there

Nixpkgs

this is pretty much complete and fully working on any linux distro. macos support for is being worked on. the problem with pnpm and offline builds also showed up here, but since nix is a lot more flexible with it's build process, i was able to figure out a way to maintain reproducibility while still using pnpm's lock file. i will try to upstream this in nixpkgs once a stable release is made

Fedora Copr
Arch User Repository
Alpine Linux packages

like you said, these should probably be community packages. i would like to note though that some may be difficult (mainly fedora rpms) due to the issue with pnpm and offline builds as mentioned before. arch is very lose with this though so it should work fine; i can't say anything about alpine since i've never used/packaged for it personally

the only other thing i would like to note is that tauri's native package export feature really shouldn't be used here. the current debian package for example seems to strip out the other tools theseus provides such as it's cli, doesn't include dependency information (i'm assuming the direct dependencies of theseus are compiled statically...which isn't great but works ig) which can cause issues with even launching minecraft, and in general is very inflexible for the future where manual intervention for packaging may be required. i'm shocked the deb package works in it's current state, let alone if it can even actually start minecraft ootb. we should be following the standard of other linux packages (specifically packages for other launchers) by supporting the formats themselves instead of relying on a tool that in all honestly, doesn't produce good packages. most of this will fall on the community of individual distros, but for a download link on the upstream maintainer's website, poorly auto generated debs and appimages aren't really acceptable for "linux support" imo. the best solution to fix this is flatpak

edit: macos support for nix is available now. discussion about upstreaming to nixpkgs will be in NixOS/nixpkgs#247468 (thanks PC!)

@Hyphrio
Copy link

Hyphrio commented Aug 6, 2023

Tauri issue and PR for Arch Linux:
tauri-apps/tauri#3728
tauri-apps/tauri#4301
PR seems to be related for Pacman but seems to have stalled

@CompeyDev
Copy link

I'd be willing to draft PKGBUILDs for the AUR, just lmk.

@TheoCGaming
Copy link

If you do go for Flatpak, please, PLEASE include proper permissions. I'm tired of having to deal with broken permissions on Flatpak packages.

@MagmaBro123
Copy link

MagmaBro123 commented Aug 6, 2023

If you do go for Flatpak, please, PLEASE include proper permissions. I'm tired of having to deal with broken permissions on Flatpak packages.

Flatseal can help with fixing those, but agreed.

@intergrav
Copy link
Contributor Author

Flatseal can help with fixing those, but agreed.

The user shouldn't have to use Flatseal to fix broken permissions provided by the package, that's just bad design

@MagmaBro123
Copy link

Flatseal can help with fixing those, but agreed.

The user shouldn't have to use Flatseal to fix broken permissions provided by the package, that's just bad design

Again, I said I agreed

@blt-r
Copy link

blt-r commented Aug 7, 2023

There is a tool to generate flatpak builder sources definitions from package-lock.json or yarn.lock, but it doesn't support pnpm-lock.yaml. The only way I found to convert pnpm-lock.yaml to something that the tool can consume is this npm package, but I didn't manage to make it work. We need to get either yarn or npm lockfile with right dependency versions. (Or implement support for pnpm in that tool, but that's very not easy).

Besides that, we probably will need to disable auto updating for the flatpak package. If there is no better way, that could be done with some sed in the build script.

Since the whole app is bundled in a single executable, packaging it as a flatpak would probably be pretty easy.

@blt-r
Copy link

blt-r commented Aug 7, 2023

Actually, on a closer look, the resulting binary has quite a few dynamic dependencies. I'll try to see if I can put all of them into the flatpak

@intergrav
Copy link
Contributor Author

intergrav commented Aug 8, 2023

I just made a forum thread in the Modrinth Discord server #development channel for discussing about this: https://discord.com/channels/734077874708938864/1138510665947041876
I'll also edit the issue and place it around the top, you can continue discussing here aswell :)

@AnOpenSauceDev
Copy link

3 days and still no mention of snap...
/s

@MagmaBro123
Copy link

Ew, Snaps.

@getchoo
Copy link

getchoo commented Aug 9, 2023

https://discord.com/channels/734077874708938864/1138510665947041876

quoting a message i made here:

nix packaging for linux and darwin is basically complete, just needs a few little touchups. i have flatpak almost working, but as it stands it would only be able to be distributed through flathub - only the website. this is due to no flatpak build tools supporting pnpm’s lockfile format; it’s a similar case with nix, but there are a few diy methods already in nixpkgs i was able to use so not an issue there. for flatpak, networking needs to be enabled during the build but that’s a big nono

only other “issue” is that tauri (or what i’m hoping to be only the specific version of tauri being used) is using an old version of webkitgtk to build so…i need to compile all of webkitgtk for it to work, which causes my pc to oom

it seems that upstream is willing to switch to npm (source) which solves the first issues; everything else with the flatpak seems pretty good.

one thing i thought of though was making some packaging guidelines for the app, as currently some things such as actual naming can be unclear. maybe this could be done with upstream sources for stuff like desktop files?

@MagmaBro123
Copy link

MagmaBro123 commented Aug 9, 2023

https://discord.com/channels/734077874708938864/1138510665947041876

quoting a message i made here:

nix packaging for linux and darwin is basically complete, just needs a few little touchups. i have flatpak almost working, but as it stands it would only be able to be distributed through flathub - only the website. this is due to no flatpak build tools supporting pnpm’s lockfile format; it’s a similar case with nix, but there are a few diy methods already in nixpkgs i was able to use so not an issue there. for flatpak, networking needs to be enabled during the build but that’s a big nono

only other “issue” is that tauri (or what i’m hoping to be only the specific version of tauri being used) is using an old version of webkitgtk to build so…i need to compile all of webkitgtk for it to work, which causes my pc to oom

it seems that upstream is willing to switch to npm (source) which solves the first issues; everything else with the flatpak seems pretty good.

one thing i thought of though was making some packaging guidelines for the app, as currently some things such as actual naming can be unclear. maybe this could be done with upstream sources for stuff like desktop files?

You and the Modrinth team would probably be better at chatting about how this would work imo. I mostly broght up Flatpak due to it being more secure thanks to its sandboxed nature.

And, Snaps also suck, Debs are kinda the stone-age of package formats, and AppImages are kinda annoying due to you having to manually download each new AppImage per update. Also, both the Deb and AppImage Modrinth provides seem to be quite buggy. At least, afaik.

@AnOpenSauceDev
Copy link

And, Snaps also suck, Debs are kinda the stone-age of package formats, and AppImages are kinda annoying due to you having to manually download each new AppImage per update. Also, both the Deb and AppImage Modrinth provides seem to be quite buggy. At least, afaik.

I was joking about snaps (unfortunately Canonical isn't), and I've had tonnes of issues with installing the DEB version at all, plus packaging systems that bundle everything like AppImages and Flatpak tend to take up more space than native (10x more for the modrinth app), which might be an issue for some people.

@MagmaBro123
Copy link

And, Snaps also suck, Debs are kinda the stone-age of package formats, and AppImages are kinda annoying due to you having to manually download each new AppImage per update. Also, both the Deb and AppImage Modrinth provides seem to be quite buggy. At least, afaik.

I was joking about snaps (unfortunately Canonical isn't), and I've had tonnes of issues with installing the DEB version at all, plus packaging systems that bundle everything like AppImages and Flatpak tend to take up more space than native (10x more for the modrinth app), which might be an issue for some people.

Eh, true. But there's no real universal way of packaging apps easily.

@getchoo
Copy link

getchoo commented Aug 9, 2023

Flatpak tend to take up more space than native (10x more for the modrinth app),

this is basically a made up number. in fact, most of modrinth app's dependencies are already in the runtime - this means unlike some flatpaks, there are very few dependencies specific to it that need to be bundled. it should be a similar size to the deb + the size of the runtime

@MagmaBro123
Copy link

@AnOpenSauceDev
Copy link

AnOpenSauceDev commented Aug 9, 2023

Flatpak tend to take up more space than native (10x more for the modrinth app),

this is basically a made up number. in fact, most of modrinth app's dependencies are already in the runtime - this means unlike some flatpaks, there are very few dependencies specific to it that need to be bundled. it should be a similar size to the deb + the size of the runtime

I was more talking about the appimage, I'm aware that flatpak has measures to keep its size down.

@cryolithic
Copy link

I'll add that flatpak is what the Steam Deck is using, so that's a big gaming base there

@AnOpenSauceDev
Copy link

AnOpenSauceDev commented Aug 10, 2023

I'll add that flatpak is what the Steam Deck is using, so that's a big gaming base there

flatpak is supported by almost all modern distros in general, but the deck can run the AppImage.

@Danik1601
Copy link

Danik1601 commented Nov 13, 2023

It looks like, if you want to play minecraft on that distro you need to use flatpak, which I learned only after compiling theseus too many times 😅

They have been actively working on it, but the work has stalled being blocked by Tauri dependencies. It seems that the work on Flatpak version will continue after Tauri 2.0 is released (after Q1 2024)

@CeruleanDerpo
Copy link

The issue for tauri apps as RPM has been recently closed and compatibility has been added. Now modrinth can be packaged for Fedora.

@getchoo
Copy link

getchoo commented Feb 15, 2024

i've opened a PR for nixpkgs; flathub is next
NixOS/nixpkgs#289149

@RokeJulianLockhart
Copy link

RokeJulianLockhart commented Feb 19, 2024

#497 (comment)

@intergrav, don't merely package RPMs for Fedora — make them compatible with openSUSE (and others) too, using universal RPM packaging, as https://forums.opensuse.org/t/proton-bridge-installation-error/172223/5 explains. I'd also build them at the OBS (as chidiwilliams/buzz#466 (comment) explains).

@getchoo
Copy link

getchoo commented Feb 20, 2024

The issue for tauri-apps/tauri#4402 has been recently closed and compatibility has been added. Now modrinth can be packaged for Fedora.

this shouldn't be used. it will have the same issues as the current deb package, as tauri has no idea it's meant to also launch minecraft

make them compatible with openSUSE (and others) too, using universal RPM packaging

this can get messy very fast. i've attempted it before with prismlauncher, and while it did work, it made the spec files more much difficult to maintain with little benefit. i would highly advise against anyone doing this, especially if you are also targeting leap releases as well as tumbleweed or fedora + enterprise linux.

I'd also build them at the OBS (as chidiwilliams/buzz#466 (comment) explains).

this is a good idea though!...just not for fedora. copr is much better for fedora users with it's built in support, and likewise obs is great for suse users with it's built in support but not so great on fedora. in my experience, users of both distros also overwhelmingly prefer their "native" platforms

@getchoo
Copy link

getchoo commented Feb 20, 2024

i've opened a PR for nixpkgs; flathub is next NixOS/nixpkgs#289149

for anyone who's missed it: the flathub pr is up! you can find the link in the original post for this thread

@MagmaBro123
Copy link

i've opened a PR for nixpkgs; flathub is next NixOS/nixpkgs#289149

for anyone who's missed it: the flathub pr is up! you can find the link in the original post for this thread

Still has a bunch of unfixed stuff though

@getchoo
Copy link

getchoo commented Feb 22, 2024

Still has a bunch of unfixed stuff though

it works fine outside of bugs found in webkitgtk (so nothing specific to the package). anything mentioned there is only about meeting flathub's guidelines rather than getting a working flatpak period

@MagmaBro123
Copy link

MagmaBro123 commented Feb 22, 2024

Ah.

@intergrav
Copy link
Contributor Author

I did create a Alpine Linux package that works.

Thou minecraft won't run because of alpine not using java with gnu core utils. It looks like, if you want to play minecraft on that distro you need to use flatpak, which I learned only after compiling theseus too many times 😅

PS. I don't see any reason using alpine package at the current state of alpine java package, but I will include my efforts bellow 😁

Modrinth-App APKBUILD modrinth-app-0.6.1-r0.apk

@jptrzy you think I should just remove the mention of an Alpine package in this issue considering that it doesn't make sense to use it in it's current state?

I'll remove it for now but lmk if you want me to add it back to the issue.

@intergrav
Copy link
Contributor Author

I'm currently working on cleaning up and updating the issue with latest info

@Enderteck
Copy link

Hi why isn't the Flathub package on the Modrinth website ?

It still downloads the appimage by default and there isn't any kind of flatpak support outside of third party packages.

Is the flatpak support is official or not, and if yes, why is it hidden in third party packages ?
Why is it not the default considering it's better and more compatible than the appimage (it will update, it really installs, it has desktop integrations, etc) ?

@CompeyDev
Copy link

@CompeyDev is working on a package

@intergrav This isn't true - I said I would be willing to work on one if required.

@intergrav
Copy link
Contributor Author

@CompeyDev fixed :)

@getchoo
Copy link

getchoo commented Jul 4, 2024

Is the flatpak support is official or not, and if yes, why is it hidden in third party packages ?

it is not, which is why it's placed there

@Bobrokus
Copy link

Bobrokus commented Jul 9, 2024

I'm suggesting my improved version of @brawaru 's Scoop package manifest: https://github.com/Bobrokus/grain/blob/master/bucket%2Fmodrinth-app.json

I might also ask for merging into the official Games Scoop bucket

@brawaru
Copy link
Contributor

brawaru commented Jul 9, 2024

"Remove-Item $env:Appdata/com.modrinth.theseus, $env:LocalAppdata/com.modrinth.theseus -Force -Recurse -ErrorAction Ignore"

This is highly concerning. You should not get rid of %AppData%\com.modrinth.theseus if you don't use it. It may actually contain user's profiles.

"THESEUS_CONFIG_DIR": "$dir/data"

This will have an unwanted side of effect of forcing every Modrinth App instance running on your Windows account to use the persist directory for Scoop installation of Modrinth App.

@Bobrokus
Copy link

Bobrokus commented Jul 9, 2024

"Remove-Item $env:Appdata/com.modrinth.theseus, $env:LocalAppdata/com.modrinth.theseus -Force -Recurse -ErrorAction Ignore"

This is highly concerning. You should not get rid of %AppData%\com.modrinth.theseus if you don't use it. It may actually contain user's profiles.

"THESEUS_CONFIG_DIR": "$dir/data"

This will have an unwanted side of effect of forcing every Modrinth App instance running on your Windows account to use the persist directory for Scoop installation of Modrinth App.

  1. I completely understand your concern. Maybe we can either ask the user if he would like to migrate? Or we can just keep the data.
  2. I don't think this is a very likely scenario imo but if it's a big deal then we can warn the user about the change and show him how to change it back. I don't like the .bat approach. Just seems too complicated.

Either way, thanks for the useful feedback, and let me know what you think!

@Sid127
Copy link

Sid127 commented Jul 25, 2024

I have packaged modrinth-app in a Fedora Copr, available here: https://copr.fedorainfracloud.org/coprs/sid127/packages and am willing to continue to maintain it

the .spec is adapted from the AUR PKGBUILD, but I run into an error where the app is unable to find index.html. Any help would be appreciated :)

** (modrinth-app:10189): WARNING **: 12:00:30.209: webkit_settings_set_enable_offline_web_application_cache is deprecated and does nothing.
/usr/bin/xdg-mime: line 885: qtpaths: command not found
Asset `index.html` not found; fallback to index.html.html

@Prospector Prospector added the web Relates to Modrinth.com web frontend label Aug 3, 2024
@mirai-toto
Copy link

Hello, I have read the entire discussion, but I am still unsure why Flatpak is not officially supported. Is it due to a technical issue or a lack of resources?

@Minenash Minenash added app Relates to Modrinth App and removed web Relates to Modrinth.com web frontend labels Aug 22, 2024
@ArduFish
Copy link

What about snaps?

@MagmaBro123
Copy link

What about snaps?

Snaps are not being dealt with by anyone as far as I know.

@CompeyDev
Copy link

Ew, Snaps.

Geometrically added a commit that referenced this issue Oct 16, 2024
* Fix statuses again

* Make it compile
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
app Relates to Modrinth App enhancement New feature or request
Development

No branches or pull requests