-
-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
doc: Improve overlays section #255132
Draft
infinisil
wants to merge
3
commits into
NixOS:master
Choose a base branch
from
tweag:doc-overlays-rewrite
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+132
−35
Draft
doc: Improve overlays section #255132
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -1,69 +1,166 @@ | ||||||
# Overlays {#chap-overlays} | ||||||
|
||||||
This chapter describes how to extend and change Nixpkgs using overlays. Overlays are used to add layers in the fixed-point used by Nixpkgs to compose the set of all packages. | ||||||
This chapter describes how to extend and change Nixpkgs using overlays. | ||||||
|
||||||
Nixpkgs can be configured with a list of overlays, which are applied in order. This means that the order of the overlays can be significant if multiple layers override the same package. | ||||||
TODO: Maybe add link to `lib.extends`/`lib.fix` | ||||||
|
||||||
## Installing overlays {#sec-overlays-install} | ||||||
## What are overlays {#sec-overlays-definition} | ||||||
|
||||||
The list of overlays can be set either explicitly in a Nix expression, or through `<nixpkgs-overlays>` or user configuration files. | ||||||
Overlays are Nix functions which accept two arguments, conventionally called `self` and `super`, and return a set of attributes. For example, the following is a valid overlay. | ||||||
|
||||||
### Set overlays in NixOS or Nix expressions {#sec-overlays-argument} | ||||||
```nix | ||||||
self: super: | ||||||
|
||||||
On a NixOS system the value of the `nixpkgs.overlays` option, if present, is passed to the system Nixpkgs directly as an argument. Note that this does not affect the overlays for non-NixOS operations (e.g. `nix-env`), which are [looked up](#sec-overlays-lookup) independently. | ||||||
{ | ||||||
boost = super.boost.override { | ||||||
python = self.python3; | ||||||
}; | ||||||
rr = super.callPackage ./pkgs/rr { | ||||||
stdenv = self.stdenv_32bit; | ||||||
}; | ||||||
} | ||||||
``` | ||||||
|
||||||
The list of overlays can be passed explicitly when importing nixpkgs, for example `import <nixpkgs> { overlays = [ overlay1 overlay2 ]; }`. | ||||||
The first argument (`self`) corresponds to the final package set. You should use this set for the dependencies of all packages specified in your overlay. For example, all the dependencies of `rr` in the example above come from `self`, as well as the overridden dependencies used in the `boost` override. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
NOTE: DO NOT USE THIS in nixpkgs. Further overlays can be added by calling the `pkgs.extend` or `pkgs.appendOverlays`, although it is often preferable to avoid these functions, because they recompute the Nixpkgs fixpoint, which is somewhat expensive to do. | ||||||
The second argument (`super`) corresponds to the result of the evaluation of the previous stages of Nixpkgs. It does not contain any of the packages added by the current overlay, nor any of the following overlays. This set should be used either to refer to packages you wish to override, or to access functions defined in Nixpkgs. For example, the original recipe of `boost` in the above example, comes from `super`, as well as the `callPackage` function. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
### Install overlays via configuration lookup {#sec-overlays-lookup} | ||||||
The value returned by this function should be a set similar to `pkgs/top-level/all-packages.nix`, containing overridden and/or new packages. | ||||||
|
||||||
The list of overlays is determined as follows. | ||||||
Overlays are similar to other methods for customizing Nixpkgs, in particular the `packageOverrides` attribute described in [](#sec-modify-via-packageOverrides). Indeed, `packageOverrides` acts as an overlay with only the `super` argument. It is therefore appropriate for basic use, but overlays are more powerful and easier to distribute. | ||||||
|
||||||
1. First, if an [`overlays` argument](#sec-overlays-argument) to the Nixpkgs function itself is given, then that is used and no path lookup will be performed. | ||||||
Note that the order of the overlays can matter if multiple overlays set the same attributes. | ||||||
|
||||||
2. Otherwise, if the Nix path entry `<nixpkgs-overlays>` exists, we look for overlays at that path, as described below. | ||||||
## Applying overlays {#sec-overlays-applying} | ||||||
|
||||||
See the section on `NIX_PATH` in the Nix manual for more details on how to set a value for `<nixpkgs-overlays>.` | ||||||
In order to define overlays, you need to know where to put them so they get applied. | ||||||
See [here](#sec-overlays-definition) for how to define overlays. | ||||||
|
||||||
3. If one of `~/.config/nixpkgs/overlays.nix` and `~/.config/nixpkgs/overlays/` exists, then we look for overlays at that path, as described below. It is an error if both exist. | ||||||
There are various ways of applying overlays, but by default none are applied. | ||||||
|
||||||
If we are looking for overlays at a path, then there are two cases: | ||||||
### Using nix expression {#d} | ||||||
|
||||||
- If the path is a file, then the file is imported as a Nix expression and used as the list of overlays. | ||||||
#### Applying overlays when importing Nixpkgs {#sec-overlays-applying-nixpkgs} | ||||||
|
||||||
- If the path is a directory, then we take the content of the directory, order it lexicographically, and attempt to interpret each as an overlay by: | ||||||
When importing Nixpkgs, a list of overlays can be specified using the `overlays` attribute in the argument: | ||||||
```nix | ||||||
pkgs = import <nixpkgs> { | ||||||
overlays = [ | ||||||
firstOverlay | ||||||
secondOverlay | ||||||
]; | ||||||
} | ||||||
``` | ||||||
|
||||||
- Importing the file, if it is a `.nix` file. | ||||||
If the `overlays` argument is not passed, these defaults are used: | ||||||
|
||||||
- Importing a top-level `default.nix` file, if it is a directory. | ||||||
- If the [path expression](https://nixos.org/manual/nix/stable/language/values#type-path) `<nixpkgs-overlays>` points to an existing path, we look for overlays there, as described in [the below section](#sec-overlays-applying-nixpkgs-paths). | ||||||
|
||||||
Because overlays that are set in NixOS configuration do not affect non-NixOS operations such as `nix-env`, the `overlays.nix` option provides a convenient way to use the same overlays for a NixOS system configuration and user configuration: the same file can be used as `overlays.nix` and imported as the value of `nixpkgs.overlays`. | ||||||
Where such path expressions point is defined with the [`NIX_PATH`](https://nixos.org/manual/nix/stable/command-ref/env-common.html#env-NIX_PATH) environment variable. | ||||||
|
||||||
On a NixOS system, this variable can be set using the [`nix.nixPath` option](https://search.nixos.org/options?show=nix.nixPath&query=nix.nixPath). | ||||||
On non-NixOS systems, see [here](https://wiki.archlinux.org/title/Environment_variables) for how to set environment variables. | ||||||
|
||||||
- If either `~/.config/nixpkgs/overlays.nix` or `~/.config/nixpkgs/overlays/` exist, then we look for overlays at that path, as described in [the below section](#sec-overlays-applying-nixpkgs-paths). | ||||||
It is an error if both exist. | ||||||
|
||||||
##### How paths get turned into overlays {#sec-overlays-applying-nixpkgs-paths} | ||||||
|
||||||
- If the path is a file, then the file is imported as a Nix expression and used as the list of overlays. | ||||||
Here is an example: | ||||||
```nix | ||||||
# overlays.nix | ||||||
[ | ||||||
# first overlay | ||||||
(final: prev: { ... }) | ||||||
|
||||||
# second overlay | ||||||
(final: prev: { ... }) | ||||||
] | ||||||
``` | ||||||
|
||||||
- If the path is a directory, then we take the entries of the directory, order them alphabetically, and attempt to interpret each as an overlay by: | ||||||
|
||||||
## Defining overlays {#sec-overlays-definition} | ||||||
- Importing the file, if it is a `.nix` file. | ||||||
|
||||||
Overlays are Nix functions which accept two arguments, conventionally called `self` and `super`, and return a set of packages. For example, the following is a valid overlay. | ||||||
- Importing a top-level `default.nix` file, if it is a directory. | ||||||
|
||||||
Here's an example of a suitable directory structure: | ||||||
``` | ||||||
overlays | ||||||
├── 50-firstOverlay.nix | ||||||
└── 60-secondOverlay | ||||||
└── default.nix | ||||||
``` | ||||||
|
||||||
#### Applying further overlays on top of an imported Nixpkgs {#c} | ||||||
|
||||||
If you already have an imported Nixpkgs (often in a variable called `pkgs`), | ||||||
you can apply a list of overlays on top using `pkgs.appendOverlays`: | ||||||
|
||||||
```nix | ||||||
self: super: | ||||||
pkgsModified = pkgs.appendOverlays [ | ||||||
firstAdditionalOverlay | ||||||
secondAdditionalOverlay | ||||||
] | ||||||
``` | ||||||
|
||||||
{ | ||||||
boost = super.boost.override { | ||||||
python = self.python3; | ||||||
}; | ||||||
rr = super.callPackage ./pkgs/rr { | ||||||
stdenv = self.stdenv_32bit; | ||||||
}; | ||||||
} | ||||||
Or just a single overlay using `pkgs.extend`: | ||||||
```nix | ||||||
pkgsModified = pkgs.extend additionalOverlay | ||||||
``` | ||||||
|
||||||
The first argument (`self`) corresponds to the final package set. You should use this set for the dependencies of all packages specified in your overlay. For example, all the dependencies of `rr` in the example above come from `self`, as well as the overridden dependencies used in the `boost` override. | ||||||
Both the overlays of original `pkgs` and the new ones will be applied, in this order. | ||||||
|
||||||
The second argument (`super`) corresponds to the result of the evaluation of the previous stages of Nixpkgs. It does not contain any of the packages added by the current overlay, nor any of the following overlays. This set should be used either to refer to packages you wish to override, or to access functions defined in Nixpkgs. For example, the original recipe of `boost` in the above example, comes from `super`, as well as the `callPackage` function. | ||||||
:::{.warning} | ||||||
The `pkgsModified` result cannot re-use any values already computed for the `pkgs` values, | ||||||
so these functions may be expensive. | ||||||
|
||||||
The value returned by this function should be a set similar to `pkgs/top-level/all-packages.nix`, containing overridden and/or new packages. | ||||||
This should not be used in Nixpkgs. | ||||||
::: | ||||||
|
||||||
### Commands implicitly importing Nixpkgs {#b} | ||||||
|
||||||
These CLI commands can implicitly import Nixpkgs without it being specified in a Nix expression. | ||||||
|
||||||
#### `nixos-rebuild` {#e} | ||||||
|
||||||
NixOS implicitly imports Nixpkgs for you [with the `overlays` argument](#sec-overlays-applying-nixpkgs) set to the value of the [`nixpkgs.overlays`](https://search.nixos.org/options?show=nixpkgs.overlays&query=nixpkgs.overlays) option. | ||||||
|
||||||
This option is empty by default. | ||||||
|
||||||
This is the only thing that the `nixpkgs.overlays` option affects. | ||||||
It does not propagate the overlays to the non-NixOS commands. | ||||||
|
||||||
Note that this does not affect the overlays for non-NixOS operations (e.g. `nix-env`), which are looked up independently, see the following sections. | ||||||
|
||||||
#### `nix-shell -p` {#f} | ||||||
|
||||||
`nix-shell -p` implicitly imports Nixpkgs without any arguments, such that the defaults from <link section> apply. | ||||||
|
||||||
#### `nix-build '<nixpkgs>'` / `nix-env -f '<nixpkgs>'` {#g} | ||||||
|
||||||
`nix-build` and `nix-env -f` with any Nixpkgs path, such as `<nixpkgs>` or `~/src/nixpkgs`, ... | ||||||
|
||||||
Will by default pass no arguments to the Nixpkgs imports, therefore the defaults from <link section> apply. | ||||||
|
||||||
The `overlays` argument can be overridden using the [`--arg`](<link to Nix manual>) option, for example to not use the default overlays: | ||||||
``` | ||||||
nix-build '<nixpkgs>' --arg overlays '[ ]' | ||||||
nix-env -f '<nixpkgs>' --arg overlays '[ ]' | ||||||
``` | ||||||
|
||||||
## Sharing overlays between a NixOS configuration and nix commands {#a} | ||||||
|
||||||
TODO: Rewrite | ||||||
Because overlays that are set in NixOS configuration do not affect non-NixOS operations such as `nix-env`, the `overlays.nix` option provides a convenient way to use the same overlays for a NixOS system configuration and user configuration: the same file can be used as `overlays.nix` and imported as the value of `nixpkgs.overlays`. | ||||||
|
||||||
```nix | ||||||
# configuration.nix | ||||||
|
||||||
nixpkgs.overlays = import ~/.config/nixpkgs/overlays.nix; | ||||||
``` | ||||||
|
||||||
Overlays are similar to other methods for customizing Nixpkgs, in particular the `packageOverrides` attribute described in [](#sec-modify-via-packageOverrides). Indeed, `packageOverrides` acts as an overlay with only the `super` argument. It is therefore appropriate for basic use, but overlays are more powerful and easier to distribute. | ||||||
|
||||||
## Using overlays to configure alternatives {#sec-overlays-alternatives} | ||||||
|
||||||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the important question is "what can i do with overlays". The technical correct definition is not as important in the first sentence.