Skip to content

Commit

Permalink
Document JSON formats
Browse files Browse the repository at this point in the history
Good to document these formats separately from commands that happen to
use them.
  • Loading branch information
Ericson2314 committed Nov 13, 2023
1 parent 2afe2e4 commit b3fbff0
Show file tree
Hide file tree
Showing 8 changed files with 172 additions and 62 deletions.
3 changes: 3 additions & 0 deletions doc/manual/src/SUMMARY.md.in
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,9 @@
- [Channels](command-ref/files/channels.md)
- [Default Nix expression](command-ref/files/default-nix-expression.md)
- [Architecture and Design](architecture/architecture.md)
- [JSON Formats](json/json.md)
- [Store Object Info](json/store-object-info.md)
- [Derivation](json/derivation.md)
- [Protocols](protocols/protocols.md)
- [Serving Tarball Flakes](protocols/tarball-fetcher.md)
- [Derivation "ATerm" file format](protocols/derivation-aterm.md)
Expand Down
2 changes: 1 addition & 1 deletion doc/manual/src/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@
non-[fixed-output](#gloss-fixed-output-derivation)
derivation.

- [output-addressed store object]{#gloss-output-addressed-store-object}
- [content-addressed store object]{#gloss-content-addressed-store-object}

A [store object] whose [store path] is determined by its contents.
This includes derivations, the outputs of [content-addressed derivations](#gloss-content-addressed-derivation), and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation).
Expand Down
65 changes: 65 additions & 0 deletions doc/manual/src/json/derivation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
{{#include ./experimental-notice.md}}

The JSON serialization of a
[store derivation](@docroot@/glossary.md#gloss-store-derivation)
is a JSON object whose keys are the store paths of the derivations, and whose values are a JSON object with the following fields:

* `name`:
The name of the derivation.
This is used when calculating the store paths of the derivation's outputs.

* `outputs`:
Information about the output paths of the derivation.
This is a JSON object with one member per output, where the key is the output name and the value is a JSON object with these fields:

* `path`: The output path.

* `hashAlgo`:
For fixed-output derivations, the hashing algorithm (e.g. `sha256`), optionally prefixed by `r:` if `hash` denotes a NAR hash rather than a flat file hash.

* `hash`:
For fixed-output derivations, the expected content hash in base-16.

> **Example**
>
> ```json
> "outputs": {
> "out": {
> "path": "/nix/store/2543j7c6jn75blc3drf4g5vhb1rhdq29-source",
> "hashAlgo": "r:sha256",
> "hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62"
> }
> }
> ```
* `inputSrcs`:
A list of store paths on which this derivation depends.
* `inputDrvs`:
A JSON object specifying the derivations on which this derivation depends, and what outputs of those derivations.
> **Example**
>
> ```json
> "inputDrvs": {
> "/nix/store/6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"],
> "/nix/store/fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"]
> }
> ```
specifies that this derivation depends on the `dev` output of `curl`, and the `out` output of `unzip`.
* `system`:
The system type on which this derivation is to be built
(e.g. `x86_64-linux`).
* `builder`:
The absolute path of the program to be executed to run the build.
Typically this is the `bash` shell
(e.g. `/nix/store/r3j288vpmczbl500w6zz89gyfa4nr0b1-bash-4.4-p23/bin/bash`).
* `args`:
The command-line arguments passed to the `builder`.
* `env`:
The environment passed to the `builder`.
5 changes: 5 additions & 0 deletions doc/manual/src/json/experimental-notice.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
> **Warning**
>
> This format is currently
> [**experimental**](@docroot@/contributing/experimental-features.md#xp-feature-nix-command)
> and subject to change.
90 changes: 90 additions & 0 deletions doc/manual/src/json/store-object-info.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
Info about a [store object].

{{#include ./experimental-notice.md}}

* `path`:

[Store path][store path] to the given store object.

* `narHash`:

Hash of the [file system object] part of the store object when serialized as a [Nix Archive](#gloss-nar).

* `narSize`:

Size of the [file system object] part of the store object when serialized as a [Nix Archive](#gloss-nar).

* `references`:

An array of [store paths][store path], possibly including this one.

* `ca` (optional):

Content address of this store object's file system object, used to compute its store path.

[store path]: @docroot@/glossary.md#gloss-store-path
[file system object]: @docroot@/architecture/file-system-object.md

## Impure fields

These are not intrinsic properties of the store object.

* `deriver` (optional):

The path to the [derivation] which created this store object.

[derivation]: @docroot@/glossary.md#gloss-store-derivation

* `registrationTime` (optional):

When this derivation was added to the store.

* `ultimate` (optional):

Whether this store object is trusted because we built it ourselves, rather than substituted a build product from elsewhere.

* `signatures` (optional):

Signatures claiming that this store object is what it claims to be.
Not relevant for [content-addressed] store objects,
but useful for [input-addressed] store objects.

[content-addressed]: @docroot@/glossary.md#gloss-content-addressed-store-object
[input-addressed]: @docroot@/glossary.md#gloss-input-addressed-store-object

### `.narinfo` extra fields

This meta data is specific to the "binary cache" family of Nix store types.
This information is not intrinsic to the store object, but about how it is stored.

* `url`:

Where to download a compressed archive of the file system objects of this store object.

* `compression`:

The compression format that the archive is in.

* `fileHash`:

A digest for the compressed archive itself, as opposed to the data contained within.

* `fileSize`:

The size of the compressed archive itself.

## Computed closure fields

These fields are not stored at all, but computed by traverising the other other fields across all the store objects in a [closure].

* `closureSize`:

The total size of the compressed archive itself for this object, and the compressed archive of every object in this object's [closure].

### `.narinfo` extra fields

* `closureSize`:

The total size of this store object and every other object in its [closure].

[closure]: @docroot@/glossary.md#gloss-closure
2 changes: 1 addition & 1 deletion src/libstore/globals.hh
Original file line number Diff line number Diff line change
Expand Up @@ -633,7 +633,7 @@ public:
- the store object has been signed using a key in the trusted keys list
- the [`require-sigs`](#conf-require-sigs) option has been set to `false`
- the store object is [output-addressed](@docroot@/glossary.md#gloss-output-addressed-store-object)
- the store object is [content-addressed](@docroot@/glossary.md#gloss-content-addressed-store-object)
)",
{"binary-cache-public-keys"}};

Expand Down
7 changes: 4 additions & 3 deletions src/nix/derivation-add.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,11 @@ Store derivations are used internally by Nix. They are store paths with
extension `.drv` that represent the build-time dependency graph to which
a Nix expression evaluates.

[store derivation]: ../../glossary.md#gloss-store-derivation

The JSON format is documented under the [`derivation show`] command.
[store derivation]: @docroot@/glossary.md#gloss-store-derivation

[`derivation show`]: ./nix3-derivation-show.md
# JSON Format

{{#include ../../json/derivation.md}}

)""
60 changes: 3 additions & 57 deletions src/nix/derivation-show.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,6 @@ R""(
* Show the [store derivation] that results from evaluating the Hello
package:

[store derivation]: ../../glossary.md#gloss-store-derivation

```console
# nix derivation show nixpkgs#hello
{
Expand Down Expand Up @@ -48,62 +46,10 @@ a Nix expression evaluates.
By default, this command only shows top-level derivations, but with
`--recursive`, it also shows their dependencies.

The JSON output is a JSON object whose keys are the store paths of the
derivations, and whose values are a JSON object with the following
fields:

* `name`: The name of the derivation. This is used when calculating the
store paths of the derivation's outputs.

* `outputs`: Information about the output paths of the
derivation. This is a JSON object with one member per output, where
the key is the output name and the value is a JSON object with these
fields:

* `path`: The output path.
* `hashAlgo`: For fixed-output derivations, the hashing algorithm
(e.g. `sha256`), optionally prefixed by `r:` if `hash` denotes a
NAR hash rather than a flat file hash.
* `hash`: For fixed-output derivations, the expected content hash in
base-16.

Example:

```json
"outputs": {
"out": {
"path": "/nix/store/2543j7c6jn75blc3drf4g5vhb1rhdq29-source",
"hashAlgo": "r:sha256",
"hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62"
}
}
```

* `inputSrcs`: A list of store paths on which this derivation depends.

* `inputDrvs`: A JSON object specifying the derivations on which this
derivation depends, and what outputs of those derivations. For
example,

```json
"inputDrvs": {
"/nix/store/6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"],
"/nix/store/fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"]
}
```

specifies that this derivation depends on the `dev` output of
`curl`, and the `out` output of `unzip`.

* `system`: The system type on which this derivation is to be built
(e.g. `x86_64-linux`).

* `builder`: The absolute path of the program to be executed to run
the build. Typically this is the `bash` shell
(e.g. `/nix/store/r3j288vpmczbl500w6zz89gyfa4nr0b1-bash-4.4-p23/bin/bash`).
[store derivation]: @docroot@/glossary.md#gloss-store-derivation

* `args`: The command-line arguments passed to the `builder`.
# JSON Format

* `env`: The environment passed to the `builder`.
{{#include ../../json/derivation.md}}

)""

0 comments on commit b3fbff0

Please sign in to comment.