From b3fbff053a38ee4981b52a2a128a45c16f131a43 Mon Sep 17 00:00:00 2001 From: John Ericson Date: Sun, 22 Oct 2023 20:01:01 -0400 Subject: [PATCH] Document JSON formats Good to document these formats separately from commands that happen to use them. --- doc/manual/src/SUMMARY.md.in | 3 + doc/manual/src/glossary.md | 2 +- doc/manual/src/json/derivation.md | 65 ++++++++++++++++ doc/manual/src/json/experimental-notice.md | 5 ++ doc/manual/src/json/store-object-info.md | 90 ++++++++++++++++++++++ src/libstore/globals.hh | 2 +- src/nix/derivation-add.md | 7 +- src/nix/derivation-show.md | 60 +-------------- 8 files changed, 172 insertions(+), 62 deletions(-) create mode 100644 doc/manual/src/json/derivation.md create mode 100644 doc/manual/src/json/experimental-notice.md create mode 100644 doc/manual/src/json/store-object-info.md diff --git a/doc/manual/src/SUMMARY.md.in b/doc/manual/src/SUMMARY.md.in index 794f78a07c27..6f772c1b0157 100644 --- a/doc/manual/src/SUMMARY.md.in +++ b/doc/manual/src/SUMMARY.md.in @@ -102,6 +102,9 @@ - [Channels](command-ref/files/channels.md) - [Default Nix expression](command-ref/files/default-nix-expression.md) - [Architecture and Design](architecture/architecture.md) +- [JSON Formats](json/json.md) + - [Store Object Info](json/store-object-info.md) + - [Derivation](json/derivation.md) - [Protocols](protocols/protocols.md) - [Serving Tarball Flakes](protocols/tarball-fetcher.md) - [Derivation "ATerm" file format](protocols/derivation-aterm.md) diff --git a/doc/manual/src/glossary.md b/doc/manual/src/glossary.md index 07891175a4e4..be574b173813 100644 --- a/doc/manual/src/glossary.md +++ b/doc/manual/src/glossary.md @@ -126,7 +126,7 @@ non-[fixed-output](#gloss-fixed-output-derivation) derivation. -- [output-addressed store object]{#gloss-output-addressed-store-object} +- [content-addressed store object]{#gloss-content-addressed-store-object} A [store object] whose [store path] is determined by its contents. This includes derivations, the outputs of [content-addressed derivations](#gloss-content-addressed-derivation), and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation). diff --git a/doc/manual/src/json/derivation.md b/doc/manual/src/json/derivation.md new file mode 100644 index 000000000000..50b98ef0275b --- /dev/null +++ b/doc/manual/src/json/derivation.md @@ -0,0 +1,65 @@ +{{#include ./experimental-notice.md}} + +The JSON serialization of a +[store derivation](@docroot@/glossary.md#gloss-store-derivation) +is a JSON object whose keys are the store paths of the derivations, and whose values are a JSON object with the following fields: + +* `name`: + The name of the derivation. + This is used when calculating the store paths of the derivation's outputs. + +* `outputs`: + Information about the output paths of the derivation. + This is a JSON object with one member per output, where the key is the output name and the value is a JSON object with these fields: + + * `path`: The output path. + + * `hashAlgo`: + For fixed-output derivations, the hashing algorithm (e.g. `sha256`), optionally prefixed by `r:` if `hash` denotes a NAR hash rather than a flat file hash. + + * `hash`: + For fixed-output derivations, the expected content hash in base-16. + + > **Example** + > + > ```json + > "outputs": { + > "out": { + > "path": "/nix/store/2543j7c6jn75blc3drf4g5vhb1rhdq29-source", + > "hashAlgo": "r:sha256", + > "hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62" + > } + > } + > ``` + +* `inputSrcs`: + A list of store paths on which this derivation depends. + +* `inputDrvs`: + A JSON object specifying the derivations on which this derivation depends, and what outputs of those derivations. + + > **Example** + > + > ```json + > "inputDrvs": { + > "/nix/store/6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"], + > "/nix/store/fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"] + > } + > ``` + + specifies that this derivation depends on the `dev` output of `curl`, and the `out` output of `unzip`. + +* `system`: + The system type on which this derivation is to be built + (e.g. `x86_64-linux`). + +* `builder`: + The absolute path of the program to be executed to run the build. + Typically this is the `bash` shell + (e.g. `/nix/store/r3j288vpmczbl500w6zz89gyfa4nr0b1-bash-4.4-p23/bin/bash`). + +* `args`: + The command-line arguments passed to the `builder`. + +* `env`: + The environment passed to the `builder`. diff --git a/doc/manual/src/json/experimental-notice.md b/doc/manual/src/json/experimental-notice.md new file mode 100644 index 000000000000..7779ca9eac51 --- /dev/null +++ b/doc/manual/src/json/experimental-notice.md @@ -0,0 +1,5 @@ +> **Warning** +> +> This format is currently +> [**experimental**](@docroot@/contributing/experimental-features.md#xp-feature-nix-command) +> and subject to change. diff --git a/doc/manual/src/json/store-object-info.md b/doc/manual/src/json/store-object-info.md new file mode 100644 index 000000000000..d5ef3e333f97 --- /dev/null +++ b/doc/manual/src/json/store-object-info.md @@ -0,0 +1,90 @@ +Info about a [store object]. + +{{#include ./experimental-notice.md}} + +* `path`: + + [Store path][store path] to the given store object. + +* `narHash`: + + Hash of the [file system object] part of the store object when serialized as a [Nix Archive](#gloss-nar). + +* `narSize`: + + Size of the [file system object] part of the store object when serialized as a [Nix Archive](#gloss-nar). + +* `references`: + + An array of [store paths][store path], possibly including this one. + +* `ca` (optional): + + Content address of this store object's file system object, used to compute its store path. + +[store path]: @docroot@/glossary.md#gloss-store-path +[file system object]: @docroot@/architecture/file-system-object.md + +## Impure fields + +These are not intrinsic properties of the store object. + +* `deriver` (optional): + + The path to the [derivation] which created this store object. + + [derivation]: @docroot@/glossary.md#gloss-store-derivation + +* `registrationTime` (optional): + + When this derivation was added to the store. + +* `ultimate` (optional): + + Whether this store object is trusted because we built it ourselves, rather than substituted a build product from elsewhere. + +* `signatures` (optional): + + Signatures claiming that this store object is what it claims to be. + Not relevant for [content-addressed] store objects, + but useful for [input-addressed] store objects. + + [content-addressed]: @docroot@/glossary.md#gloss-content-addressed-store-object + [input-addressed]: @docroot@/glossary.md#gloss-input-addressed-store-object + +### `.narinfo` extra fields + +This meta data is specific to the "binary cache" family of Nix store types. +This information is not intrinsic to the store object, but about how it is stored. + +* `url`: + + Where to download a compressed archive of the file system objects of this store object. + +* `compression`: + + The compression format that the archive is in. + +* `fileHash`: + + A digest for the compressed archive itself, as opposed to the data contained within. + +* `fileSize`: + + The size of the compressed archive itself. + +## Computed closure fields + +These fields are not stored at all, but computed by traverising the other other fields across all the store objects in a [closure]. + +* `closureSize`: + + The total size of the compressed archive itself for this object, and the compressed archive of every object in this object's [closure]. + +### `.narinfo` extra fields + +* `closureSize`: + + The total size of this store object and every other object in its [closure]. + +[closure]: @docroot@/glossary.md#gloss-closure diff --git a/src/libstore/globals.hh b/src/libstore/globals.hh index 27caf42c4eda..353c0a921e14 100644 --- a/src/libstore/globals.hh +++ b/src/libstore/globals.hh @@ -633,7 +633,7 @@ public: - the store object has been signed using a key in the trusted keys list - the [`require-sigs`](#conf-require-sigs) option has been set to `false` - - the store object is [output-addressed](@docroot@/glossary.md#gloss-output-addressed-store-object) + - the store object is [content-addressed](@docroot@/glossary.md#gloss-content-addressed-store-object) )", {"binary-cache-public-keys"}}; diff --git a/src/nix/derivation-add.md b/src/nix/derivation-add.md index f116681ab3a3..b76409dd79cf 100644 --- a/src/nix/derivation-add.md +++ b/src/nix/derivation-add.md @@ -9,10 +9,11 @@ Store derivations are used internally by Nix. They are store paths with extension `.drv` that represent the build-time dependency graph to which a Nix expression evaluates. -[store derivation]: ../../glossary.md#gloss-store-derivation -The JSON format is documented under the [`derivation show`] command. +[store derivation]: @docroot@/glossary.md#gloss-store-derivation -[`derivation show`]: ./nix3-derivation-show.md +# JSON Format + +{{#include ../../json/derivation.md}} )"" diff --git a/src/nix/derivation-show.md b/src/nix/derivation-show.md index 1296e2885f4c..c4b51281c805 100644 --- a/src/nix/derivation-show.md +++ b/src/nix/derivation-show.md @@ -5,8 +5,6 @@ R""( * Show the [store derivation] that results from evaluating the Hello package: - [store derivation]: ../../glossary.md#gloss-store-derivation - ```console # nix derivation show nixpkgs#hello { @@ -48,62 +46,10 @@ a Nix expression evaluates. By default, this command only shows top-level derivations, but with `--recursive`, it also shows their dependencies. -The JSON output is a JSON object whose keys are the store paths of the -derivations, and whose values are a JSON object with the following -fields: - -* `name`: The name of the derivation. This is used when calculating the - store paths of the derivation's outputs. - -* `outputs`: Information about the output paths of the - derivation. This is a JSON object with one member per output, where - the key is the output name and the value is a JSON object with these - fields: - - * `path`: The output path. - * `hashAlgo`: For fixed-output derivations, the hashing algorithm - (e.g. `sha256`), optionally prefixed by `r:` if `hash` denotes a - NAR hash rather than a flat file hash. - * `hash`: For fixed-output derivations, the expected content hash in - base-16. - - Example: - - ```json - "outputs": { - "out": { - "path": "/nix/store/2543j7c6jn75blc3drf4g5vhb1rhdq29-source", - "hashAlgo": "r:sha256", - "hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62" - } - } - ``` - -* `inputSrcs`: A list of store paths on which this derivation depends. - -* `inputDrvs`: A JSON object specifying the derivations on which this - derivation depends, and what outputs of those derivations. For - example, - - ```json - "inputDrvs": { - "/nix/store/6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"], - "/nix/store/fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"] - } - ``` - - specifies that this derivation depends on the `dev` output of - `curl`, and the `out` output of `unzip`. - -* `system`: The system type on which this derivation is to be built - (e.g. `x86_64-linux`). - -* `builder`: The absolute path of the program to be executed to run - the build. Typically this is the `bash` shell - (e.g. `/nix/store/r3j288vpmczbl500w6zz89gyfa4nr0b1-bash-4.4-p23/bin/bash`). +[store derivation]: @docroot@/glossary.md#gloss-store-derivation -* `args`: The command-line arguments passed to the `builder`. +# JSON Format -* `env`: The environment passed to the `builder`. +{{#include ../../json/derivation.md}} )""