[RFC 0182] I'm Gonna Build My Own Home Tool With Blackjack and Modules! #182
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,306 @@ | ||
| --- | ||
| feature: I'm Gonna Build My Own Home Tool With Blackjack and Modules! | ||
| start-date: 2024-09-20 | ||
| author: Anderson Torres | ||
| co-authors: @nyabinary | ||
| shepherd-team: (names, to be nominated and accepted by RFC steering committee) | ||
| shepherd-leader: (name to be appointed by RFC steering committee) | ||
| related-issues: (will contain links to implementation PRs) | ||
| --- | ||
|
|
||
| # Summary | ||
| [summary]: #summary | ||
|
|
||
| Enhance Nixpkgs monorepo with a declarative management system for basic users. | ||
|
|
||
| # Terminology | ||
| [terminology]: #terminology | ||
|
|
||
| Henceforth, | ||
|
|
||
| - The typical unprivileged user of a system will be called _basic user_; | ||
| - The typical privileged user of a system will be called _superuser_; | ||
|
|
||
| # Motivation | ||
| [motivation]: #motivation | ||
|
|
||
| [Nixpkgs](https://github.com/NixOS/nixpkgs) is by a far margin the largest | ||
| packageset in this existence, according to | ||
| [Repology](https://repology.org/repository/nix_unstable). | ||
|
|
||
| In principle, currently the Nix interpreter and evaluator already leverage | ||
| Nixpkgs for basic users. However, this raw usage is not very ergonomical, | ||
| especially when compared to the structured model provided by NixOS. | ||
|
|
||
| In this panorama, the average user has two extreme options here: | ||
|
|
||
| - system-wide configuration via NixOS, requiring superuser privileges; | ||
| - ad-hoc management via raw Nix commands, especially the not-recommended | ||
| `nix-env`. | ||
|
|
||
| This RFC proposes to fill this gap: a structured model of system management for | ||
| basic users that does not require superuser privileges to be deployed. | ||
|
|
||
| # Detailed Design | ||
| [design]: #detailed-design | ||
|
|
||
| In a dedicated directory inside the Nixpkgs monorepo a well-crafted set of core | ||
| modules, plus some companion modules, will be created and maintained. | ||
|
|
||
| Further, a companion toolset (henceforth called `hometool`) to manipulate and | ||
| deploy the home environment will be provided. | ||
|
|
||
| This toolset plus module set should provide the following properties: | ||
|
|
||
| - Declarativeness | ||
|
|
||
| Users can specify the configuration of their systems in Nix language, by a set | ||
| of detailed descriptions. | ||
|
|
||
| - Immutability | ||
|
|
||
| As a consequence of declarativeness, the resulting descriptions are immutable. | ||
| It allows comparing them by knowing precisely what changed between | ||
| configurations. | ||
|
|
||
| - Customizability | ||
|
|
||
| Users can derive specialized module configurations from current module set. | ||
|
There was a problem hiding this comment. This is very similar to the next point. How would a user go about this? There was a problem hiding this comment. Exempli gratia, the user should be able to override the (This is a recent concern in NixOS modules; many of them hardcode the main program.) There was a problem hiding this comment. That doesn't sound much like a "specialized module configuration" to me, just ensuring support for what should be basic options in a module. The wording here implies extending/editing the base modules to fit individual needs; simply introducing a There was a problem hiding this comment.
This would be interesting too. There was a problem hiding this comment. That'd be awesome even in NixOS honestly, it'd be nice to have a slightly nicer interface to extend modules, particularly the ones that use submodule types (sure its not hard, but it could be made simpler maybe) |
||
|
|
||
| - Extensibility | ||
|
|
||
| Users can extend the module set by writing their own, leveraged by overlay and | ||
| other Nix language constructs. | ||
|
There was a problem hiding this comment. I feel this is a bit vague. Writing your own modules and importing them is of course a mainstay in the NixOS module system -- but I'm not sure how overlays and "other Nix language constructs" would fit here. Is this referring to the customization of packages used by a module in an overlay rather than extending the module itself? What language constructs would assist in extending an existing module for example? |
||
|
|
||
| - Scalability | ||
|
|
||
| `hometool` should be scalable from the simplest to the more complex user | ||
| environment definitions. | ||
|
Comment on lines
+90
to
+91
|
||
|
|
||
| - Documentation | ||
|
|
||
| Both the `hometool` and the moduleset should be well documented. | ||
|
There was a problem hiding this comment. We should be more specific as to what "well documented" means. Should this have a dedicated manual similar to nixpkgs/NixOS? What about support for these modules on |
||
|
|
||
| - Human Friendliness | ||
|
|
||
| This `hometool` should be approachable with clear and understandable logging | ||
| messages, plus the aforementioned documentation. | ||
|
Comment on lines
+99
to
+100
There was a problem hiding this comment. This should probably also include the UX of the tool, as documentation and logging is really only one step in being "user friendly" (especially in the context of the CLI) |
||
|
|
||
| # Examples and Interactions | ||
| [examples-and-interactions]: #examples-and-interactions | ||
|
|
||
| Let's call `hometool` the tool used by the basic users to deploy and deal with | ||
| their configurations. | ||
|
|
||
| ```shell | ||
| $> ### help message, possibly pointing to further detailed and/or specialized help, like `--all`, `--modules` | ||
| $> hometool help | ||
| $> ### generate a sample config; can be enhanced with a wizard | ||
| $> hometool sample-config > user-configuration.nix | ||
| $> ### because we like to tweak things a bit | ||
| $> emacs user-configuration.nix | ||
| $> ### build it first without deploying it yet | ||
| $> hometool build | ||
| $> ### a container to test it before deploying - for us paranoids! | ||
| $> hometool container | ||
AndersonTorres marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| $> ### now install it! | ||
| $> hometool install | ||
AndersonTorres marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| $> ### list the generations | ||
| $> hometool generations list | ||
| $> ### list differences between generations | ||
| $> hometool generations diff 9 10 | ||
|
There was a problem hiding this comment. I'm not against a feature like this, but what differences would be shown? Would it be similar to There was a problem hiding this comment. If a tool already provides this, then we can push the tool as an optional dependency. |
||
| $> ### select a specific generation | ||
| $> hometool generations select 10 | ||
AndersonTorres marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| $> ### remove older generations, marking them to be garbage-collected | ||
| $> hometool generations remove 1-8 | ||
| ``` | ||
|
|
||
| # Drawbacks | ||
| [drawbacks]: #drawbacks | ||
|
|
||
| - Why not to keep this tool outside Nixpkgs monorepo? | ||
|
|
||
| It can be argued that overlays and other facilities provided by Nix language | ||
| allow to keep entire projects outside the Nixpkgs monorepo. | ||
|
|
||
| However, any of those projects suffer of an unpleasant phenomenon: the second | ||
| class citizen repo. | ||
|
|
||
| Basically, a repository outside the monorepo will not receive the same level | ||
| of care and attention of a project inside the monorepo. It happens because, | ||
| among other things, | ||
|
|
||
| - it is harder to pay attention in multiple repositories at the same time slot | ||
| than only one; | ||
| - it is harder to deal with pieces of code contained in distinct repositories; | ||
| - it is harder to synchronize multiple repositories; | ||
|
|
||
| Because of those hurdles, it becomes easier to ignore the repos outside the | ||
| main monorepo. | ||
|
|
||
| As a consequence, breaking changes in the monorepo are prone to be decided | ||
| without taking the satellite repos into consideration. This situation leads | ||
| the satellite repos to deal with the extra work of synchronizing with the | ||
| recent changes. | ||
|
|
||
| By living inside the main monorepo, the problems exposed above diminishes | ||
| drastically. | ||
|
|
||
| Further, having this home management toolset inside the monorepo brings many | ||
| advantages: | ||
|
|
||
| - No longer dealing with synchronization among projects | ||
| - Better strategies for code deduplication, sharing and refactoring | ||
| - Direct access to the most recent enhancements available | ||
|
|
||
| Indeed this is precisely what happens in NixOS already: when a program is | ||
| added, a NixOS module and test suite can be plugged in the same pull | ||
| request, with few to no bureaucracy. | ||
|
|
||
| - Reputation | ||
|
|
||
| A home management toolset kept in the monorepo has the social add-on of | ||
| being better regarded by the Nixpkgs community as a whole, since there will | ||
| be no barriers for contribution from this same community. | ||
|
|
||
| - The monorepo will increase in size. | ||
|
|
||
| The increase in size is expected to be small. | ||
|
There was a problem hiding this comment. Using a similar method as below and home-manager as a reference point $ du -sh nixpkgs/nixos/modules home-manager/modules
17M nixpkgs/nixos/modules
3.0M home-manager/modulesThis is ~1/5th the size of our NixOS modules. Even with the refactors mentioned below, I'm not sure I would call this "small". Furthermore, this glosses over the changes required to make the refactors and code sharing possible, as well as the tools, testing, documentation, etc associated with this There was a problem hiding this comment. Well, my estimation is way worse in this prospect. There was a problem hiding this comment. So the idea is that this will provide the same functionality as Home Manager but with a tenth of the code? How did you derive that number? There was a problem hiding this comment.
No.
From what getchoo provided. Not sure if I understood your question here. |
||
|
|
||
| Since we are building a set of modules to deal with the Nixpkgs packageset, a | ||
| good heuristic approximation of the worst case would be the current size of | ||
| `nixos` directory - after all, we want NixOS without superuser privileges. | ||
|
|
||
| A quick `du` returns something like this. | ||
|
|
||
| ```shell | ||
| $> du -sh nixos/ pkgs/ | ||
| 024572 nixos/ | ||
| 379536 pkgs/ | ||
| ``` | ||
|
|
||
| By the numbers above, obtained locally at 2024-09-13, `nixos` occupies less | ||
| than 7% of the sum of both `nixos + pkgs`. A quick and dirty calculation would | ||
| bring a similar 7% increase in size. | ||
|
|
||
| This is certainly a crude calculation, however we are ignoring many factors | ||
| that will bring these numbers down. Namely: | ||
|
|
||
| - Refactorings | ||
|
|
||
| Similar code can be factored and abstracted. | ||
|
There was a problem hiding this comment. "Factored" seems very close to the next point of code sharing. And what are some examples of abstractions to help here? There was a problem hiding this comment. The most obvious to me is There was a problem hiding this comment.
Actually there's this |
||
|
|
||
| - Code sharing | ||
|
|
||
| Code that was initially projected for basic user management can be found | ||
| useful for superuser management too. | ||
|
There was a problem hiding this comment. How would we go about this? AFAIK, there isn't much in the modules system to really share/extend options from other modules -- let alone those that aren't technically apart of the current evaluation. I'd imagine this would need a good bit of work on There was a problem hiding this comment. There are many duplicated pieces of code between home-manager and nixos. There was a problem hiding this comment. Definitely! ...but that isn't what I'm asking here: it's how you would share code between a home-manager and NixOS module for Chromium. This is repeatedly mentioned in this RFC, but without any actual methods as to how to do it The only one I can think of is what we currently do with NGINX's virtual host options at times, but using primitive language constructs like this for many shared options isn't a very good idea IMO. Again, this would probably need a good amount of work to specifically implement well -- unless you have some other ideas There was a problem hiding this comment. I do not expect this to be made methodically or automatically - my guts suggests this is a problem on the fringes of undecidable. In the specific case of Chromium, I can think of creating smaller modules for structuring things like
In the end, the NixOS and Hometool modules would use the moduleset defined above and extend it according to the particularities of their respective use cases. |
||
|
|
||
| - Code duplications. | ||
|
|
||
| Code duplications can be refactored as they appear. | ||
|
|
||
| Indeed it is way easier to deal with code duplication in a monorepo: since the | ||
| barrier of communication between multiple repositories disappears, a | ||
| duplicated code can be factored more easily, requiring just one pull request | ||
| instead of one per repo. | ||
|
Comment on lines
+212
to
+217
There was a problem hiding this comment. See above. In the context of modules, this isn't something we really have good tools for ATM. How could we fix this? |
||
|
|
||
| - Evaluation times of monorepo will increase. | ||
|
|
||
| The increase in evaluation time is a bit harder to measure. However, | ||
|
|
||
| - "increase in evaluation time" was not an argument strong enough for undoing | ||
| the NixOS assimilation; | ||
| - arguably, the increase in evaluation time will be felt by those effectively | ||
| using the hometool; the remaining users will not be signficatively affected. | ||
| - a similar argument for the increase in size can be crafted here. | ||
|
|
||
| The perceived advantages of having this module system surpasses these small | ||
| disadvantages | ||
|
|
||
| # Alternatives | ||
| [alternatives]: #alternatives | ||
|
|
||
| - The trivial "do nothing". | ||
|
|
||
| - Promote an alternative toolset. | ||
|
|
||
| The most viable alternative in this field is home-manager. However it is not | ||
| so easy to assimilate a consolidated project within another consolidated | ||
| project, not only in terms of source code, but also in terms of community and | ||
| decision-making. | ||
|
|
||
| A better approach would be promote some form of a healthy competition, in | ||
| which home-manager and hometool are different projects competing for the same | ||
| niche, each with their own teams, communities, design projects etc.; further, | ||
| both teams are free to share code between them, provided they follow their | ||
| corresponding licenses in doing so. | ||
|
There was a problem hiding this comment. I don't think this point is well made. You just say it's not easy, without elaborating on why. IMO creating this from scratch is substantially less easy, but that's not the point. You just go on to talk about competition rather than explaining what's hard about bringing in home-manager and how starting from scratch is easier. |
||
|
|
||
| - Promote `guix home` | ||
|
|
||
| What? Lisp is cool! | ||
|
|
||
| # Prior art | ||
| [prior-art]: #prior-art | ||
|
There was a problem hiding this comment. I would like to see There was a problem hiding this comment. I believe NixOS/nixpkgs#9250 really deserves a mention here. FWICT it is the closest thing to what is proposed in this RFC. |
||
|
|
||
| ## Guix Home | ||
|
|
||
| As an example of prior art, there is our Scheme-based cousin, Guix Software | ||
| Distribution. Since at least 2022 AD they bring a similar tool, conveniently | ||
| called Guix Home. | ||
|
|
||
| The nicest thing about this tool is that it is tightly integrated with Guix, to | ||
| the point of `home` being a mere subcommand of `guix`. | ||
|
|
||
| ## Home Manager | ||
|
|
||
| Home Manager is a well-established toolset that leverages the Nixpkgs module | ||
| system to allow basic users to manage their user-specific environment in a | ||
| declarative way. | ||
|
|
||
| ## Wrapper Manager | ||
|
|
||
| Wrapper-manager is a Nix library that allows the user to configure your favorite | ||
| applications without adding files into `~/.config`. This is done by creating | ||
| wrapper scripts that set the appropriate environment set - variables, flags, | ||
| configuration files etc. - to the wrapped program. | ||
|
Comment on lines
+298
to
+303
There was a problem hiding this comment. While this is prior art, I am a bit confusion by its inclusion. Is this implying the modules should take inspiration from this as opposed to using Also linking the author's blogpost on replacing home-manager with this, as the project is relatively new and the post gives some background to readers who are unfamiliar (like I was until a few days ago :p) |
||
|
|
||
| # Unresolved questions | ||
| [unresolved]: #unresolved-questions | ||
|
|
||
| Given that this tool will live inside Nixpkgs monorepo, it is expected that | ||
| future packages will interact with this new tool. How those interactions should | ||
| be dealt? | ||
|
|
||
| # Future work | ||
| [future]: #future-work | ||
|
|
||
| - Update an extend the CI | ||
AndersonTorres marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - Set expectations on portability among present and future platforms Nixpkgs | ||
| supports | ||
| - Especially outside NixOS | ||
| - Especially outside Linux | ||
|
There was a problem hiding this comment. I feel this is a bit out of scope for this RFC. Expanding platform support in nixpkgs is important of course, but I don't think it has much to do with user management There was a problem hiding this comment. I disagree. Home management would be a part of the package manager, not NixOS. I don't see why derivations would work on other systems but home management won't, it feels like a regression to me. Also, #163 should pave way for the implementation of things detailed in this RFC if/when accepted, making support for other platforms not so out of scope, IMO. There was a problem hiding this comment.
My interpretation of this was moreso that it was encouraging better support for platforms in derivations, not this tool specifically. If this is speaking about If I was correct in my initial reading though, I think my point still stands. Allowing derivations to build for more targets doesn't have much to do with introducing ways to configure them There was a problem hiding this comment. Well, if you're correct on that, then nixpkgs already supports more targets even if not formally listed in some policy. If it's regarding There was a problem hiding this comment. Well, in my experience different distros do not share There was a problem hiding this comment.
I'm a bit confused what this has to do with the current thread. I don't believe sharing There was a problem hiding this comment. I think the wording here is ambiguous. @AndersonTorres by "platform" do you mean architectures or OS/distributions There was a problem hiding this comment.
Anecdotal evidence ahead. In the old times I was switching from Slackware to Arch, my
Operating system - and everything it implies but we usually abstract away (libc, userland etc.). There was a problem hiding this comment.
I'm well aware of these types of issues, but again I'm not sure what it has to do with the current thread. Are you implying this tool should act similar to sharing a /home but not have the typical issues associated with it?
This needs to be specified in the RFC. The current wording is (once again) vague
Comment on lines
+322
to
+323
There was a problem hiding this comment. I think both of these points are going to become a real burden for future use, and I'd honestly somewhat discourage their practical application. home-manager already somewhat suffers from this "one shoe fits all" philosophy by duplicating NixOS module logic all over. Of course, for home-manager specifically this is not a problem, because they never aimed to be a NixOS only tool anyways, but if we are going to bring in first class support, we have to decide to what extent are we willing to duplicate and re-use modules from NixOS if we want to consider support for a broader ecosystem. It's not like NixOS modules can be used outside of NixOS (future RFC pending?) and the guarantees we can make in an environment outside of the module system is slim at best. It would also be practically unfeasible to attempt what home-manager does and reinvent the wheel everywhere, because that leads to duplicated logic and maintenance burden. To be completely fair, I actually would love the idea of a world where I could declare any Linux system OOTB using Nix/NixOS modules and make it work, and optionally declare a home environment in much the same way, but practically those are hard to implement, especially at our level. The most I think we should ever realistically aim for is first-class support for NixOS only. Tighter integration allows us more flexibility in what we can re-use, more testing and applicable user experience to improve the tooling from, and although not necessarily the strongest point: makes NixOS a more appealing distribution for reasons beyond just system level configuration. There was a problem hiding this comment. If support for the non‐NixOS Home Manager platforms is dropped, then this will not result in merging Home Manager into the Nixpkgs monorepo or usurping Home Manager by something better; it will just result in an ecosystem fracturing between the home management tooling in the Nixpkgs monorepo and the Home Manager that will continue to exist. It is very popular to use Home Manager to manage user environments on non‐NixOS Linux and macOS, and a significant new user funnel for NixOS – besides which, the utility of Home Manager is decreased if it has to be integrated into a monolithic NixOS configuration and doesn’t allow individual users autonomy anyway, as it makes the multi‐user case a pain, and the single‐user case is already served fairly decently by NixOS already. |
||
|
|
||
| # References | ||
| [references]: #references | ||
|
|
||
| - [Home Manager](https://nix-community.github.io/home-manager/) | ||
| - [Keeping One's Home | ||
| Tidy](https://guix.gnu.org/en/blog/2022/keeping-ones-home-tidy/), by Ludovic | ||
| Courtès | ||
| - [Guix Home | ||
| Configuration](https://guix.gnu.org/manual/devel/en/html_node/Home-Configuration.html) | ||
| - [Wrapper Manager](https://github.com/viperML/wrapper-manager) | ||
| - [Stop Using nix-env](https://stop-using-nix-env.privatevoid.net/) | ||
| - [Overlay for User Packages](https://gist.github.com/LnL7/570349866bb69467d0caf5cb175faa74) by LnL7 | ||
| - [BuilEnv-based Declarative User | ||
| Environment](https://gist.github.com/lheckemann/402e61e8e53f136f239ecd8c17ab1deb) | ||
| by lheckellman | ||
| - [Nix Home](https://github.com/museoa/nix-home/) | ||
| - Forked and archived from [Nix Home](https://github.com/sheenobu/nix-home/) | ||
| - with patches saved from other forks | ||
| - [nix-config from akavel](https://github.com/akavel/nix-config/tree/master/.nixpkgs) | ||
| - https://github.com/sheenobu/nix-home/issues/16 | ||
| - https://github.com/akavel/nix-config/blob/510f36861cc4a641bd976ad25dd339949b47339d/.nixpkgs/nix-home.nix | ||
| - https://github.com/akavel/nix-config/blob/510f36861cc4a641bd976ad25dd339949b47339d/.nixpkgs/nix-home.sh | ||
| - https://github.com/akavel/nix-config/blob/510f36861cc4a641bd976ad25dd339949b47339d/.nixpkgs/config.nix#L57 | ||
| - [GNU Stow](https://www.gnu.org/software/stow/) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In order to keep the conversation more discoverable, I am copying and answering here:
@paperdigits - #182 (comment)
Not necessarily. The code does not need to be written ex nihilo.
In the worst case, ignoring all considerations over ethics, we could just merge the current state of home-manager and start working right now.
But I feel this is not correct.
Assimilate HM would be veryi nice.
Indeed I kept the previous proposal for assimilation here, but preliminary conversations on Discourse and Matrix led me to cogitate this approach.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I definitely agree that everything does not need a rewrite, but I'd also like to highlight that many home-manager modules duplicate functionality that already exists in NixOS in some form (activation logic, systemd units, programs.* services.*). Large scale cleanup is going to be required to have a smooth transition that doesn't break in little places everywhere.