From 57afc733fff56e9b08592cde63290eafeb622ca1 Mon Sep 17 00:00:00 2001 From: Silvan Mosberger Date: Thu, 14 Sep 2023 17:12:46 +0200 Subject: [PATCH 1/3] doc: Improve overlays section --- doc/using/overlays.chapter.md | 69 +++++++++++++++++++++++++---------- 1 file changed, 50 insertions(+), 19 deletions(-) diff --git a/doc/using/overlays.chapter.md b/doc/using/overlays.chapter.md index a51aa9ee8fc54..2ba8e39e84edb 100644 --- a/doc/using/overlays.chapter.md +++ b/doc/using/overlays.chapter.md @@ -2,47 +2,76 @@ This chapter describes how to extend and change Nixpkgs using overlays. Overlays are used to add layers in the fixed-point used by Nixpkgs to compose the set of all packages. -Nixpkgs can be configured with a list of overlays, which are applied in order. This means that the order of the overlays can be significant if multiple layers override the same package. +## Applying overlays -## Installing overlays {#sec-overlays-install} +There are various ways of applying overlays. -The list of overlays can be set either explicitly in a Nix expression, or through `` or user configuration files. +### Applying overlays when importing Nixpkgs -### Set overlays in NixOS or Nix expressions {#sec-overlays-argument} +When importing Nixpkgs, a list of overlays can be specified using the `overlays` attribute in the argument: +```nix +pkgs = import { + overlays = [ + firstOverlay + secondOverlay + ]; +} +``` -On a NixOS system the value of the `nixpkgs.overlays` option, if present, is passed to the system Nixpkgs directly as an argument. Note that this does not affect the overlays for non-NixOS operations (e.g. `nix-env`), which are [looked up](#sec-overlays-lookup) independently. +If the `overlays` argument is not passed, these defaults are used: +- If the Nix path entry `` exists, we look for overlays at that path, as described below. -The list of overlays can be passed explicitly when importing nixpkgs, for example `import { overlays = [ overlay1 overlay2 ]; }`. + See the section on `NIX_PATH` in the Nix manual for more details on how to set a value for `.` +- If one of `~/.config/nixpkgs/overlays.nix` and `~/.config/nixpkgs/overlays/` exists, then we look for overlays at that path, as described below. It is an error if both exist. -NOTE: DO NOT USE THIS in nixpkgs. Further overlays can be added by calling the `pkgs.extend` or `pkgs.appendOverlays`, although it is often preferable to avoid these functions, because they recompute the Nixpkgs fixpoint, which is somewhat expensive to do. +#### How paths get turned into overlays +- If the path is a file, then the file is imported as a Nix expression and used as the list of overlays. -### Install overlays via configuration lookup {#sec-overlays-lookup} +- If the path is a directory, then we take the content of the directory, order it lexicographically, and attempt to interpret each as an overlay by: -The list of overlays is determined as follows. + - Importing the file, if it is a `.nix` file. -1. First, if an [`overlays` argument](#sec-overlays-argument) to the Nixpkgs function itself is given, then that is used and no path lookup will be performed. + - Importing a top-level `default.nix` file, if it is a directory. -2. Otherwise, if the Nix path entry `` exists, we look for overlays at that path, as described below. +### Applying further overlays on top of an imported Nixpkgs - See the section on `NIX_PATH` in the Nix manual for more details on how to set a value for `.` +If you already have an imported Nixpkgs, you can apply further overlays on top using: +```nix +pkgs.appendOverlays [ + firstAdditionalOverlay + secondAdditionalOverlay +] +``` -3. If one of `~/.config/nixpkgs/overlays.nix` and `~/.config/nixpkgs/overlays/` exists, then we look for overlays at that path, as described below. It is an error if both exist. +Or for just a single overlay: +```nix +pkgs.extend additionalOverlay +``` -If we are looking for overlays at a path, then there are two cases: +:::{.warning} +These functions recompute the Nixpkgs fixpoint, which is somewhat expensive to do. +::: -- If the path is a file, then the file is imported as a Nix expression and used as the list of overlays. +### Setting overlays for a NixOS configuration -- If the path is a directory, then we take the content of the directory, order it lexicographically, and attempt to interpret each as an overlay by: +(link to nixpkgs.overlays option in the NixOS manual) - - Importing the file, if it is a `.nix` file. +Note that this does not affect the overlays for non-NixOS operations (e.g. `nix-env`), which are [looked up](#sec-overlays-lookup) independently. - - Importing a top-level `default.nix` file, if it is a directory. +## Sharing overlays between a NixOS configuration and nix commands +TODO: Rewrite Because overlays that are set in NixOS configuration do not affect non-NixOS operations such as `nix-env`, the `overlays.nix` option provides a convenient way to use the same overlays for a NixOS system configuration and user configuration: the same file can be used as `overlays.nix` and imported as the value of `nixpkgs.overlays`. +```nix +# configuration.nix + +nixpkgs.overlays = import ~/.config/nixpkgs/overlays.nix; +``` + ## Defining overlays {#sec-overlays-definition} -Overlays are Nix functions which accept two arguments, conventionally called `self` and `super`, and return a set of packages. For example, the following is a valid overlay. +Overlays are Nix functions which accept two arguments, conventionally called `self` and `super`, and return a set of attributes. For example, the following is a valid overlay. ```nix self: super: @@ -65,6 +94,8 @@ The value returned by this function should be a set similar to `pkgs/top-level/a Overlays are similar to other methods for customizing Nixpkgs, in particular the `packageOverrides` attribute described in [](#sec-modify-via-packageOverrides). Indeed, `packageOverrides` acts as an overlay with only the `super` argument. It is therefore appropriate for basic use, but overlays are more powerful and easier to distribute. +Note that the order of the overlays can matter if multiple overlays set the same attributes. + ## Using overlays to configure alternatives {#sec-overlays-alternatives} Certain software packages have different implementations of the same interface. Other distributions have functionality to switch between these. For example, Debian provides [DebianAlternatives](https://wiki.debian.org/DebianAlternatives). Nixpkgs has what we call `alternatives`, which are configured through overlays. From b80ff702ef512242440e1ceb3eaf0e9900e2ff06 Mon Sep 17 00:00:00 2001 From: Silvan Mosberger Date: Mon, 18 Sep 2023 17:34:17 +0200 Subject: [PATCH 2/3] Various improvements --- doc/using/overlays.chapter.md | 64 ++++++++++++++++++++++++++++------- 1 file changed, 51 insertions(+), 13 deletions(-) diff --git a/doc/using/overlays.chapter.md b/doc/using/overlays.chapter.md index 2ba8e39e84edb..c49d0bb208bf8 100644 --- a/doc/using/overlays.chapter.md +++ b/doc/using/overlays.chapter.md @@ -2,11 +2,14 @@ This chapter describes how to extend and change Nixpkgs using overlays. Overlays are used to add layers in the fixed-point used by Nixpkgs to compose the set of all packages. -## Applying overlays +## Applying overlays {#sec-overlays-applying} -There are various ways of applying overlays. +In order to define overlays, you need to know where to put them so they get applied. +See [here](#sec-overlays-definition) for how to define overlays. -### Applying overlays when importing Nixpkgs +There are various ways of applying overlays, but by default none are applied. + +### Applying overlays when importing Nixpkgs {#sec-overlays-applying-nixpkgs} When importing Nixpkgs, a list of overlays can be specified using the `overlays` attribute in the argument: ```nix @@ -19,41 +22,76 @@ pkgs = import { ``` If the `overlays` argument is not passed, these defaults are used: -- If the Nix path entry `` exists, we look for overlays at that path, as described below. - See the section on `NIX_PATH` in the Nix manual for more details on how to set a value for `.` -- If one of `~/.config/nixpkgs/overlays.nix` and `~/.config/nixpkgs/overlays/` exists, then we look for overlays at that path, as described below. It is an error if both exist. +- If the [path expression](https://nixos.org/manual/nix/stable/language/values#type-path) `` points to an existing path, we look for overlays there, as described in [the below section](#sec-overlays-applying-nixpkgs-paths). + + Where such path expressions point is defined with the [`NIX_PATH`](https://nixos.org/manual/nix/stable/command-ref/env-common.html#env-NIX_PATH) environment variable. + + On a NixOS system, this variable can be set using the [`nix.nixPath` option](https://search.nixos.org/options?query=nix.nixPath). + On non-NixOS systems, see [here](https://wiki.archlinux.org/title/Environment_variables) for how to set environment variables. + +- If either `~/.config/nixpkgs/overlays.nix` or `~/.config/nixpkgs/overlays/` exist, then we look for overlays at that path, as described in [the below section](#sec-overlays-applying-nixpkgs-paths). + It is an error if both exist. + +#### How paths get turned into overlays {#sec-overlays-applying-nixpkgs-paths} -#### How paths get turned into overlays - If the path is a file, then the file is imported as a Nix expression and used as the list of overlays. + Here is an example: + ```nix + # overlays.nix + [ + # first overlay + (final: prev: { ... }) + + # second overlay + (final: prev: { ... }) + ] + ``` -- If the path is a directory, then we take the content of the directory, order it lexicographically, and attempt to interpret each as an overlay by: +- If the path is a directory, then we take the entries of the directory, order them alphabetically, and attempt to interpret each as an overlay by: - Importing the file, if it is a `.nix` file. - Importing a top-level `default.nix` file, if it is a directory. + Here's an example of a suitable directory structure: + ``` + overlays + ├── 50-firstOverlay.nix + └── 60-secondOverlay + └── default.nix + ``` + ### Applying further overlays on top of an imported Nixpkgs -If you already have an imported Nixpkgs, you can apply further overlays on top using: +If you already have an imported Nixpkgs (often in a variable called `pkgs`), +you can apply a list of overlays on top using `pkgs.appendOverlays`: + ```nix -pkgs.appendOverlays [ +pkgsModified = pkgs.appendOverlays [ firstAdditionalOverlay secondAdditionalOverlay ] ``` -Or for just a single overlay: +Or just a single overlay using `pkgs.extend`: ```nix -pkgs.extend additionalOverlay +pkgsModified = pkgs.extend additionalOverlay ``` +Both the overlays of original `pkgs` and the new ones will be applied, in this order. + :::{.warning} -These functions recompute the Nixpkgs fixpoint, which is somewhat expensive to do. +The `pkgsModified` result cannot re-use any values already computed for the `pkgs` values, +so these functions may be expensive. + +This should not be used in Nixpkgs. ::: ### Setting overlays for a NixOS configuration +NixOS evaluations do not use + (link to nixpkgs.overlays option in the NixOS manual) Note that this does not affect the overlays for non-NixOS operations (e.g. `nix-env`), which are [looked up](#sec-overlays-lookup) independently. From de2b045fb766188c7d86329d5f4db722c4732ac0 Mon Sep 17 00:00:00 2001 From: Silvan Mosberger Date: Thu, 21 Sep 2023 16:50:43 +0200 Subject: [PATCH 3/3] Commands implicitly importing Nixpkgs bla --- doc/using/overlays.chapter.md | 94 +++++++++++++++++++++++------------ 1 file changed, 61 insertions(+), 33 deletions(-) diff --git a/doc/using/overlays.chapter.md b/doc/using/overlays.chapter.md index c49d0bb208bf8..ad1425d4379a3 100644 --- a/doc/using/overlays.chapter.md +++ b/doc/using/overlays.chapter.md @@ -1,6 +1,35 @@ # Overlays {#chap-overlays} -This chapter describes how to extend and change Nixpkgs using overlays. Overlays are used to add layers in the fixed-point used by Nixpkgs to compose the set of all packages. +This chapter describes how to extend and change Nixpkgs using overlays. + +TODO: Maybe add link to `lib.extends`/`lib.fix` + +## What are overlays {#sec-overlays-definition} + +Overlays are Nix functions which accept two arguments, conventionally called `self` and `super`, and return a set of attributes. For example, the following is a valid overlay. + +```nix +self: super: + +{ + boost = super.boost.override { + python = self.python3; + }; + rr = super.callPackage ./pkgs/rr { + stdenv = self.stdenv_32bit; + }; +} +``` + +The first argument (`self`) corresponds to the final package set. You should use this set for the dependencies of all packages specified in your overlay. For example, all the dependencies of `rr` in the example above come from `self`, as well as the overridden dependencies used in the `boost` override. + +The second argument (`super`) corresponds to the result of the evaluation of the previous stages of Nixpkgs. It does not contain any of the packages added by the current overlay, nor any of the following overlays. This set should be used either to refer to packages you wish to override, or to access functions defined in Nixpkgs. For example, the original recipe of `boost` in the above example, comes from `super`, as well as the `callPackage` function. + +The value returned by this function should be a set similar to `pkgs/top-level/all-packages.nix`, containing overridden and/or new packages. + +Overlays are similar to other methods for customizing Nixpkgs, in particular the `packageOverrides` attribute described in [](#sec-modify-via-packageOverrides). Indeed, `packageOverrides` acts as an overlay with only the `super` argument. It is therefore appropriate for basic use, but overlays are more powerful and easier to distribute. + +Note that the order of the overlays can matter if multiple overlays set the same attributes. ## Applying overlays {#sec-overlays-applying} @@ -9,7 +38,9 @@ See [here](#sec-overlays-definition) for how to define overlays. There are various ways of applying overlays, but by default none are applied. -### Applying overlays when importing Nixpkgs {#sec-overlays-applying-nixpkgs} +### Using nix expression {#d} + +#### Applying overlays when importing Nixpkgs {#sec-overlays-applying-nixpkgs} When importing Nixpkgs, a list of overlays can be specified using the `overlays` attribute in the argument: ```nix @@ -27,13 +58,13 @@ If the `overlays` argument is not passed, these defaults are used: Where such path expressions point is defined with the [`NIX_PATH`](https://nixos.org/manual/nix/stable/command-ref/env-common.html#env-NIX_PATH) environment variable. - On a NixOS system, this variable can be set using the [`nix.nixPath` option](https://search.nixos.org/options?query=nix.nixPath). + On a NixOS system, this variable can be set using the [`nix.nixPath` option](https://search.nixos.org/options?show=nix.nixPath&query=nix.nixPath). On non-NixOS systems, see [here](https://wiki.archlinux.org/title/Environment_variables) for how to set environment variables. - If either `~/.config/nixpkgs/overlays.nix` or `~/.config/nixpkgs/overlays/` exist, then we look for overlays at that path, as described in [the below section](#sec-overlays-applying-nixpkgs-paths). It is an error if both exist. -#### How paths get turned into overlays {#sec-overlays-applying-nixpkgs-paths} +##### How paths get turned into overlays {#sec-overlays-applying-nixpkgs-paths} - If the path is a file, then the file is imported as a Nix expression and used as the list of overlays. Here is an example: @@ -62,7 +93,7 @@ If the `overlays` argument is not passed, these defaults are used: └── default.nix ``` -### Applying further overlays on top of an imported Nixpkgs +#### Applying further overlays on top of an imported Nixpkgs {#c} If you already have an imported Nixpkgs (often in a variable called `pkgs`), you can apply a list of overlays on top using `pkgs.appendOverlays`: @@ -88,51 +119,48 @@ so these functions may be expensive. This should not be used in Nixpkgs. ::: -### Setting overlays for a NixOS configuration +### Commands implicitly importing Nixpkgs {#b} -NixOS evaluations do not use +These CLI commands can implicitly import Nixpkgs without it being specified in a Nix expression. -(link to nixpkgs.overlays option in the NixOS manual) +#### `nixos-rebuild` {#e} -Note that this does not affect the overlays for non-NixOS operations (e.g. `nix-env`), which are [looked up](#sec-overlays-lookup) independently. +NixOS implicitly imports Nixpkgs for you [with the `overlays` argument](#sec-overlays-applying-nixpkgs) set to the value of the [`nixpkgs.overlays`](https://search.nixos.org/options?show=nixpkgs.overlays&query=nixpkgs.overlays) option. -## Sharing overlays between a NixOS configuration and nix commands +This option is empty by default. -TODO: Rewrite -Because overlays that are set in NixOS configuration do not affect non-NixOS operations such as `nix-env`, the `overlays.nix` option provides a convenient way to use the same overlays for a NixOS system configuration and user configuration: the same file can be used as `overlays.nix` and imported as the value of `nixpkgs.overlays`. +This is the only thing that the `nixpkgs.overlays` option affects. +It does not propagate the overlays to the non-NixOS commands. -```nix -# configuration.nix +Note that this does not affect the overlays for non-NixOS operations (e.g. `nix-env`), which are looked up independently, see the following sections. -nixpkgs.overlays = import ~/.config/nixpkgs/overlays.nix; -``` +#### `nix-shell -p` {#f} -## Defining overlays {#sec-overlays-definition} +`nix-shell -p` implicitly imports Nixpkgs without any arguments, such that the defaults from apply. -Overlays are Nix functions which accept two arguments, conventionally called `self` and `super`, and return a set of attributes. For example, the following is a valid overlay. +#### `nix-build ''` / `nix-env -f ''` {#g} -```nix -self: super: +`nix-build` and `nix-env -f` with any Nixpkgs path, such as `` or `~/src/nixpkgs`, ... -{ - boost = super.boost.override { - python = self.python3; - }; - rr = super.callPackage ./pkgs/rr { - stdenv = self.stdenv_32bit; - }; -} +Will by default pass no arguments to the Nixpkgs imports, therefore the defaults from apply. + +The `overlays` argument can be overridden using the [`--arg`]() option, for example to not use the default overlays: +``` +nix-build '' --arg overlays '[ ]' +nix-env -f '' --arg overlays '[ ]' ``` -The first argument (`self`) corresponds to the final package set. You should use this set for the dependencies of all packages specified in your overlay. For example, all the dependencies of `rr` in the example above come from `self`, as well as the overridden dependencies used in the `boost` override. +## Sharing overlays between a NixOS configuration and nix commands {#a} -The second argument (`super`) corresponds to the result of the evaluation of the previous stages of Nixpkgs. It does not contain any of the packages added by the current overlay, nor any of the following overlays. This set should be used either to refer to packages you wish to override, or to access functions defined in Nixpkgs. For example, the original recipe of `boost` in the above example, comes from `super`, as well as the `callPackage` function. +TODO: Rewrite +Because overlays that are set in NixOS configuration do not affect non-NixOS operations such as `nix-env`, the `overlays.nix` option provides a convenient way to use the same overlays for a NixOS system configuration and user configuration: the same file can be used as `overlays.nix` and imported as the value of `nixpkgs.overlays`. -The value returned by this function should be a set similar to `pkgs/top-level/all-packages.nix`, containing overridden and/or new packages. +```nix +# configuration.nix -Overlays are similar to other methods for customizing Nixpkgs, in particular the `packageOverrides` attribute described in [](#sec-modify-via-packageOverrides). Indeed, `packageOverrides` acts as an overlay with only the `super` argument. It is therefore appropriate for basic use, but overlays are more powerful and easier to distribute. +nixpkgs.overlays = import ~/.config/nixpkgs/overlays.nix; +``` -Note that the order of the overlays can matter if multiple overlays set the same attributes. ## Using overlays to configure alternatives {#sec-overlays-alternatives}