[RFC 0182] I'm Gonna Build My Own Home Tool With Blackjack and Modules! #182
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,350 @@ | ||
| --- | ||
| feature: I'm Gonna Build My Own Home Tool With Blackjack and Modules! | ||
| start-date: 2024-09-20 | ||
| author: Anderson Torres | ||
| co-authors: `@nyabinary` | ||
| shepherd-team: (names, to be nominated and accepted by RFC steering committee) | ||
| shepherd-leader: (name to be appointed by RFC steering committee) | ||
| related-issues: (will contain links to implementation PRs) | ||
| --- | ||
|
|
||
| # Summary | ||
| [summary]: #summary | ||
|
|
||
| Enhance Nixpkgs monorepo with a declarative management system for basic users. | ||
|
|
||
| # Terminology | ||
| [terminology]: #terminology | ||
|
|
||
| Henceforth, | ||
|
|
||
| - The typical unprivileged user of a system will be called _basic user_; | ||
| - The typical privileged user of a system will be called _superuser_; | ||
|
|
||
| # Motivation | ||
| [motivation]: #motivation | ||
|
|
||
| [Nixpkgs](https://github.com/NixOS/nixpkgs) is by a far margin the largest | ||
| packageset in this existence, according to | ||
| [Repology](https://repology.org/repository/nix_unstable). | ||
|
|
||
| In principle, currently the Nix interpreter and evaluator already leverage | ||
| Nixpkgs for basic users. However, this raw usage is not very ergonomical, | ||
| especially when compared to the structured model provided by NixOS. | ||
|
|
||
| In this panorama, the average user has two extreme options here: | ||
|
|
||
| - system-wide configuration via NixOS, requiring superuser privileges; | ||
| - ad-hoc management via raw Nix commands, especially the not-recommended | ||
| `nix-env`. | ||
|
|
||
| This RFC proposes to fill this gap: a structured model of system management for | ||
| basic users that does not require superuser privileges to be deployed. | ||
|
|
||
| Let's call it `hometool`. | ||
|
|
||
| # Detailed Design | ||
| [design]: #detailed-design | ||
|
|
||
| `hometool` has two components: | ||
|
|
||
| 1. A driver program, hereinafter called `hometool`. | ||
|
|
||
| Among other related tasks, this tool has the role of realizing the | ||
| description of a home environment. | ||
|
|
||
| As expected from a driver, this tool will rely on other programs to execute | ||
| its intents; the most immediate one being the Nix evaluator. | ||
|
|
||
| 2. A set of carefully crafted modules, leveraged by Nix Module System. | ||
|
|
||
| The moduleset should reside in `hometool/` directory at the root of Nixpkgs | ||
| monorepo, similar to how it happens to `nixos/` nowadays. | ||
|
|
||
| # Expectations | ||
| [expectations]: #expectations | ||
|
|
||
| The set of components above should provide the following properties: | ||
|
|
||
| - Declarativeness | ||
|
|
||
| Users can specify the configuration of their systems in Nix language, by a set | ||
| of detailed descriptions. | ||
|
|
||
| - Immutability | ||
|
|
||
| As a consequence of declarativeness, the resulting descriptions are immutable. | ||
| It allows comparing them by knowing precisely what changed between | ||
| configurations. | ||
|
|
||
| - Customizability | ||
|
|
||
| Users can derive specialized module configurations from current module set. | ||
|
There was a problem hiding this comment. This is very similar to the next point. How would a user go about this? There was a problem hiding this comment. Exempli gratia, the user should be able to override the (This is a recent concern in NixOS modules; many of them hardcode the main program.) There was a problem hiding this comment. That doesn't sound much like a "specialized module configuration" to me, just ensuring support for what should be basic options in a module. The wording here implies extending/editing the base modules to fit individual needs; simply introducing a There was a problem hiding this comment.
This would be interesting too. There was a problem hiding this comment. That'd be awesome even in NixOS honestly, it'd be nice to have a slightly nicer interface to extend modules, particularly the ones that use submodule types (sure its not hard, but it could be made simpler maybe) |
||
|
|
||
| - Extensibility | ||
|
|
||
| Users can extend the module set by writing their own. | ||
|
|
||
| - Scalability | ||
|
|
||
| `hometool` should be scalable from the simplest to the more complex user | ||
| environment definitions. | ||
|
Comment on lines
+90
to
+91
|
||
|
|
||
| - Documentation | ||
|
|
||
| Both the `hometool` and the moduleset should be well documented. | ||
|
There was a problem hiding this comment. We should be more specific as to what "well documented" means. Should this have a dedicated manual similar to nixpkgs/NixOS? What about support for these modules on |
||
|
|
||
| - Human Friendliness | ||
|
|
||
| This `hometool` should be approachable with clear and understandable logging | ||
| messages, plus the aforementioned documentation. | ||
|
Comment on lines
+99
to
+100
There was a problem hiding this comment. This should probably also include the UX of the tool, as documentation and logging is really only one step in being "user friendly" (especially in the context of the CLI) |
||
|
|
||
| # Examples and Interactions | ||
| [examples-and-interactions]: #examples-and-interactions | ||
|
|
||
| Here is a small example of what is expected | ||
|
|
||
| ```console | ||
| $> ### help message, possibly pointing to further detailed and/or specialized help, like `--all`, `--modules` | ||
| $> hometool help | ||
| $> ### generate a sample config | ||
| $> hometool sample-config > user-configuration.nix | ||
| $> ### because we like to tweak things a bit | ||
| $> emacs user-configuration.nix | ||
| $> ### build it first without deploying it yet | ||
| $> hometool build | ||
| $> ### a VM to test it before deploying - for us paranoids! | ||
| $> hometool build-vm | ||
| $> ### now install it! | ||
| $> hometool switch | ||
| $> ### list the generations | ||
| $> hometool generations list | ||
| $> ### list differences between generations | ||
| $> hometool generations diff 9 10 | ||
|
There was a problem hiding this comment. I'm not against a feature like this, but what differences would be shown? Would it be similar to There was a problem hiding this comment. If a tool already provides this, then we can push the tool as an optional dependency. |
||
| $> ### select a specific generation | ||
| $> hometool generations switch 10 | ||
| $> ### remove older generations | ||
| $> hometool generations remove 1-8 | ||
| ``` | ||
|
|
||
| # Drawbacks | ||
| [drawbacks]: #drawbacks | ||
|
|
||
| - Why not to keep this tool outside Nixpkgs monorepo? | ||
|
|
||
| It can be argued that overlays and other facilities provided by Nix language | ||
| allow to keep entire projects outside the Nixpkgs monorepo. | ||
|
|
||
| However, any of those projects suffer of an unpleasant phenomenon: the second | ||
| class citizen repo. | ||
|
|
||
| Basically, a repository outside the monorepo will not receive the same level | ||
| of care and attention of a project inside the monorepo. It happens because, | ||
| among other things, | ||
|
|
||
| - it is harder to pay attention in multiple repositories at the same time slot | ||
| than only one; | ||
| - it is harder to deal with pieces of code contained in distinct repositories; | ||
| - it is harder to synchronize multiple repositories; | ||
|
|
||
| Because of those hurdles, it becomes easier to ignore the repos outside the | ||
| main monorepo. | ||
|
|
||
| As a consequence, breaking changes in the monorepo are prone to be decided | ||
| without taking the satellite repos into consideration. This situation leads | ||
| the satellite repos to deal with the extra work of synchronizing with the | ||
| recent changes. | ||
|
|
||
| By living inside the main monorepo, the problems exposed above diminishes | ||
| drastically. | ||
|
|
||
| Further, having this home management toolset inside the monorepo brings many | ||
| advantages: | ||
|
|
||
| - No longer dealing with synchronization among projects | ||
| - Better strategies for code deduplication, sharing and refactoring | ||
| - Direct access to the most recent enhancements available | ||
|
|
||
| Indeed this is precisely what happens in NixOS already: when a program is | ||
| added, a NixOS module and test suite can be plugged in the same pull | ||
| request, with few to no bureaucracy. | ||
|
|
||
| - Reputation | ||
|
|
||
| A home management toolset kept in the monorepo has the social add-on of | ||
| being better regarded by the Nixpkgs community as a whole, since there will | ||
| be no barriers for contribution from this same community. | ||
|
|
||
| - The monorepo will increase in size. | ||
|
|
||
| The increase in size is expected to be small. | ||
|
There was a problem hiding this comment. Using a similar method as below and home-manager as a reference point $ du -sh nixpkgs/nixos/modules home-manager/modules
17M nixpkgs/nixos/modules
3.0M home-manager/modulesThis is ~1/5th the size of our NixOS modules. Even with the refactors mentioned below, I'm not sure I would call this "small". Furthermore, this glosses over the changes required to make the refactors and code sharing possible, as well as the tools, testing, documentation, etc associated with this There was a problem hiding this comment. Well, my estimation is way worse in this prospect. There was a problem hiding this comment. So the idea is that this will provide the same functionality as Home Manager but with a tenth of the code? How did you derive that number? There was a problem hiding this comment.
No.
From what getchoo provided. Not sure if I understood your question here. |
||
|
|
||
| Since we are building a set of modules to deal with the Nixpkgs packageset, a | ||
| good heuristic approximation of the worst case would be the current size of | ||
| `nixos` directory - after all, we want NixOS without superuser privileges. | ||
|
|
||
| A quick `du` returns something like this. | ||
|
|
||
| ```shell | ||
| $> du -sh nixos/ pkgs/ | ||
| 024572 nixos/ | ||
| 379536 pkgs/ | ||
| ``` | ||
|
|
||
| By the numbers above, obtained locally at 2024-09-13, `nixos` occupies less | ||
| than 7% of the sum of both `nixos + pkgs`. A quick and dirty calculation would | ||
| bring a similar 7% increase in size. | ||
|
|
||
| This is certainly a crude calculation, however we are ignoring many factors | ||
| that will bring these numbers down. Namely: | ||
|
|
||
| - Refactorings | ||
|
|
||
| Similar code can be factored and abstracted. | ||
|
There was a problem hiding this comment. "Factored" seems very close to the next point of code sharing. And what are some examples of abstractions to help here? There was a problem hiding this comment. The most obvious to me is There was a problem hiding this comment.
Actually there's this |
||
|
|
||
| - Code sharing | ||
|
|
||
| Code that was initially projected for basic user management can be found | ||
| useful for superuser management too; and vice-versa. | ||
|
|
||
| - Code duplications. | ||
|
|
||
| Code duplications can be refactored as they appear. | ||
|
|
||
| Indeed it is way easier to deal with code duplication in a monorepo: since the | ||
| barrier of communication between multiple repositories disappears, a | ||
| duplicated code can be factored more easily, requiring just one pull request | ||
| instead of one per repo. | ||
|
Comment on lines
+212
to
+217
There was a problem hiding this comment. See above. In the context of modules, this isn't something we really have good tools for ATM. How could we fix this? |
||
|
|
||
| - Evaluation times of monorepo will increase. | ||
|
|
||
| The increase in evaluation time is a bit harder to measure. However, | ||
|
|
||
| - "increase in evaluation time" was not an argument strong enough for undoing | ||
| the NixOS assimilation; | ||
| - arguably, the increase in evaluation time will be felt by those effectively | ||
| using the hometool; the remaining users will not be signficatively affected. | ||
| - a similar argument for the increase in size can be crafted here. | ||
|
|
||
| The perceived advantages of having this module system surpasses these small | ||
| disadvantages about size and evaluation time. | ||
|
|
||
| Further, outside the scope of Nixpkgs, there are initiatives focused on | ||
| optimizing the Nix evaluators. | ||
|
|
||
| # Alternatives | ||
| [alternatives]: #alternatives | ||
|
|
||
| ## The trivial "do nothing" | ||
|
|
||
| Trivially keeping the status quo. | ||
|
|
||
| ## Promote `guix-home` | ||
|
|
||
| What? Lisp is cool! | ||
|
|
||
| ## Promote `home-manager` | ||
|
|
||
| Now a serious alternative worthy of consideration. | ||
|
|
||
| Home Manager has the most obvious and the most powerful advantage: | ||
|
|
||
| It exists and is working well. Further, it is battle-tested and encodes a lot of | ||
| knowledge in its 8 years lifespan and more than 3.7k commits. | ||
|
|
||
| However, it has some non-negligible disadvantages: | ||
|
|
||
| - The current code does not follow the same standards of the current Nixpkgs | ||
| monorepo | ||
|
|
||
| - many modules rely on stringly `extraConfig` instead of structured | ||
| `settings`, contra RFC 0042; | ||
|
|
||
| - Many instances of technical debt, something expected in such a long-lived | ||
| project: | ||
|
|
||
| - excessive uses of `with lib;` | ||
|
|
||
| - hacky solutions like the dangerous `mkOutOfStoreSymlink`: | ||
| https://github.com/nix-community/home-manager/issues/3032 | ||
|
|
||
| - under-documented workarounds | ||
|
|
||
| - too much reliance on Bash and its nasty idiosyncrasies | ||
|
|
||
| On the other hand, starting from a cleaner slate has the opposite set of | ||
| advantages and disadvantages: the kickstart of a project is too wide and it is | ||
| easier to get lost. On the other hand, there is considerably more freedom to | ||
| prototype and test ideas, older and newer. | ||
|
|
||
| # Prior art | ||
| [prior-art]: #prior-art | ||
|
There was a problem hiding this comment. I would like to see There was a problem hiding this comment. I believe NixOS/nixpkgs#9250 really deserves a mention here. FWICT it is the closest thing to what is proposed in this RFC. |
||
|
|
||
| ## Guix Home | ||
|
|
||
| As an example of prior art, there is our Scheme-based cousin, Guix Software | ||
| Distribution. Since at least 2022 AD they bring a similar tool, conveniently | ||
| called Guix Home. | ||
|
|
||
| The nicest thing about this tool is its tight integration with Guix as a whole, | ||
| to the point of `home` being a mere subcommand of `guix`. | ||
|
|
||
| ## Home Manager | ||
|
|
||
| Home Manager is a well-established toolset that leverages the Nixpkgs module | ||
| system to allow basic users to manage their user-specific environment in a | ||
| declarative way. | ||
|
|
||
| ## Wrapper Manager | ||
|
|
||
| Wrapper-manager is a Nix library that allows the user to configure your favorite | ||
| applications without adding files into `~/.config`. This is done by creating | ||
| wrapper scripts that set the appropriate environment set - variables, flags, | ||
| configuration files etc. - to the wrapped program. | ||
|
Comment on lines
+298
to
+303
There was a problem hiding this comment. While this is prior art, I am a bit confusion by its inclusion. Is this implying the modules should take inspiration from this as opposed to using Also linking the author's blogpost on replacing home-manager with this, as the project is relatively new and the post gives some background to readers who are unfamiliar (like I was until a few days ago :p) |
||
|
|
||
| ## nixos-rebuild | ||
|
|
||
| In the Nix world, nixos-rebuild is the master tool in terms of deployment of a | ||
| Nix-based configuration, albeit being system-wide. | ||
|
|
||
| # Unresolved questions | ||
| [unresolved]: #unresolved-questions | ||
|
|
||
| Given that this tool will live inside Nixpkgs monorepo, it is expected that | ||
| future packages will interact with this new tool. How those interactions should | ||
| be dealt? | ||
|
|
||
| # Future work | ||
| [future]: #future-work | ||
|
|
||
| - Update and extend the CI | ||
| - Set expectations on portability | ||
| - Especially outside NixOS | ||
| - Especially outside Linux | ||
|
Comment on lines
+322
to
+323
There was a problem hiding this comment. I think both of these points are going to become a real burden for future use, and I'd honestly somewhat discourage their practical application. home-manager already somewhat suffers from this "one shoe fits all" philosophy by duplicating NixOS module logic all over. Of course, for home-manager specifically this is not a problem, because they never aimed to be a NixOS only tool anyways, but if we are going to bring in first class support, we have to decide to what extent are we willing to duplicate and re-use modules from NixOS if we want to consider support for a broader ecosystem. It's not like NixOS modules can be used outside of NixOS (future RFC pending?) and the guarantees we can make in an environment outside of the module system is slim at best. It would also be practically unfeasible to attempt what home-manager does and reinvent the wheel everywhere, because that leads to duplicated logic and maintenance burden. To be completely fair, I actually would love the idea of a world where I could declare any Linux system OOTB using Nix/NixOS modules and make it work, and optionally declare a home environment in much the same way, but practically those are hard to implement, especially at our level. The most I think we should ever realistically aim for is first-class support for NixOS only. Tighter integration allows us more flexibility in what we can re-use, more testing and applicable user experience to improve the tooling from, and although not necessarily the strongest point: makes NixOS a more appealing distribution for reasons beyond just system level configuration. There was a problem hiding this comment. If support for the non‐NixOS Home Manager platforms is dropped, then this will not result in merging Home Manager into the Nixpkgs monorepo or usurping Home Manager by something better; it will just result in an ecosystem fracturing between the home management tooling in the Nixpkgs monorepo and the Home Manager that will continue to exist. It is very popular to use Home Manager to manage user environments on non‐NixOS Linux and macOS, and a significant new user funnel for NixOS – besides which, the utility of Home Manager is decreased if it has to be integrated into a monolithic NixOS configuration and doesn’t allow individual users autonomy anyway, as it makes the multi‐user case a pain, and the single‐user case is already served fairly decently by NixOS already. |
||
|
|
||
| # References | ||
| [references]: #references | ||
|
|
||
| - [Home Manager](https://nix-community.github.io/home-manager/) | ||
| - [Keeping One's Home | ||
| Tidy](https://guix.gnu.org/en/blog/2022/keeping-ones-home-tidy/), by Ludovic | ||
| Courtès | ||
| - [Guix Home | ||
| Configuration](https://guix.gnu.org/manual/devel/en/html_node/Home-Configuration.html) | ||
| - [Wrapper Manager](https://github.com/viperML/wrapper-manager) | ||
| - [Stop Using nix-env](https://stop-using-nix-env.privatevoid.net/) | ||
| - [Overlay for User Packages](https://gist.github.com/LnL7/570349866bb69467d0caf5cb175faa74) by LnL7 | ||
| - [BuilEnv-based Declarative User | ||
| Environment](https://gist.github.com/lheckemann/402e61e8e53f136f239ecd8c17ab1deb) | ||
| by lheckellman | ||
| - [Nix Home](https://github.com/museoa/nix-home/) | ||
| - Forked and archived from [Nix Home](https://github.com/sheenobu/nix-home/) | ||
| - with patches saved from other forks | ||
| - [nix-config from akavel](https://github.com/akavel/nix-config/tree/master/.nixpkgs) | ||
| - https://github.com/sheenobu/nix-home/issues/16 | ||
| - https://github.com/akavel/nix-config/blob/510f36861cc4a641bd976ad25dd339949b47339d/.nixpkgs/nix-home.nix | ||
| - https://github.com/akavel/nix-config/blob/510f36861cc4a641bd976ad25dd339949b47339d/.nixpkgs/nix-home.sh | ||
| - https://github.com/akavel/nix-config/blob/510f36861cc4a641bd976ad25dd339949b47339d/.nixpkgs/config.nix#L57 | ||
| - [GNU Stow](https://www.gnu.org/software/stow/) | ||
| - [Dropping Home Manager](https://ayats.org/blog/no-home-manager), an article from ViperML | ||
| containing some criticism about home-manager | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In order to keep the conversation more discoverable, I am copying and answering here:
@paperdigits - #182 (comment)
Not necessarily. The code does not need to be written ex nihilo.
In the worst case, ignoring all considerations over ethics, we could just merge the current state of home-manager and start working right now.
But I feel this is not correct.
Assimilate HM would be veryi nice.
Indeed I kept the previous proposal for assimilation here, but preliminary conversations on Discourse and Matrix led me to cogitate this approach.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I definitely agree that everything does not need a rewrite, but I'd also like to highlight that many home-manager modules duplicate functionality that already exists in NixOS in some form (activation logic, systemd units, programs.* services.*). Large scale cleanup is going to be required to have a smooth transition that doesn't break in little places everywhere.