docs/concept: working with incomplete CUE

This adds a concept guide explaining the basics of working with
incomplete CUE using both the cue command and the Go API.

Fixes cue-lang/docs-and-content#180.

Preview-Path: /docs/concept/working-with-incomplete-cue/
Signed-off-by: Paul Jolly <[email protected]>
Change-Id: If2fdd49e42c96cd098a19464038827ad2000e951
Dispatch-Trailer: {"type":"trybot","CL":1203092,"patchset":9,"ref":"refs/changes/92/1203092/9","targetBranch":"master"}

content/docs/concept/working-with-incomplete-cue/en.md

@@ -0,0 +1,150 @@
+---
+title: "Working with incomplete CUE"
+toc_hide: true
+authors: [myitcv, jpluscplusm]
+tags: [go api]
+---
+
+{{{with _script_ "en" "set caches to speed up re-running"}}}
+export GOMODCACHE=/caches/gomodcache
+export GOCACHE=/caches/gobuild
+{{{end}}}
+
+In general, CUE can handle references to fields that don't yet exist, or where
+a value can't be calculated because there's insufficient information.
+Otherwise valid CUE that contains such references or values is referred to as
+*incomplete* CUE.
+
+Here's an example of incomplete CUE. In `example.cue`, field `A.x` doesn't
+exist, which means that the value of field `B` can't be calculated given the
+information in this file alone:
+
+{{{with upload "en" "example.cue"}}}
+-- example.cue --
+package example
+
+A: b: 2
+B: A.x + 10
+{{{end}}}
+
+Because CUE permits data to be added into an evaluation
+(through {{<linkto/inline "tour/basics/unification" >}}),
+the value of field `B` might be able to be calculated in the future - but only
+if the right information is introduced.
+Until then, this CUE is still *valid*, but is referred to as "incomplete"
+because there isn't sufficient information to perform a complete evaluation.
+
+Because incomplete CUE is valid CUE it *can* be evaluated ...
+
+{{{with script "en" "cue eval"}}}
+cue eval example.cue
+{{{end}}}
+
+... but it *can't* be exported:
+
+{{{with script "en" "cue export"}}}
+! cue export example.cue
+{{{end}}}
+
+A configuration that includes incomplete CUE can be made *complete* by unifying
+it with the right data. This means that every field that contributes to the
+configuration being emitted must be  able to be calculated.
+
+Here's some some YAML data that will do this for our example, by "filling in
+the gaps" in our incomplete CUE:
+
+{{{with upload "en" "data.yaml"}}}
+-- data.yaml --
+A:
+  x: 5
+{{{end}}}
+
+Unifying our incomplete CUE with this data makes the configuration complete,
+and allows us to export the result:
+
+{{{with script "en" "export example.cue data.yaml"}}}
+cue export example.cue data.yaml
+{{{end}}}
+
+### Using the Go API
+
+The Go API is also able to handle incomplete CUE.
+
+To demonstrate the Go API in action we start by initializing a Go module:
+
+{{{with script "en" "go mod init"}}}
+#ellipsis 0
+go mod init go.example
+{{{end}}}
+
+We place this example Go code in `main.go`.
+Its comments explain what is being done at each step.
+
+{{{with upload "en" "main.go"}}}
+-- main.go --
+package main
+
+import (
+	"fmt"
+	"log"
+
+	"cuelang.org/go/cue"
+	"cuelang.org/go/cue/cuecontext"
+	"cuelang.org/go/cue/load"
+	"cuelang.org/go/encoding/yaml"
+)
+
+func main() {
+	ctx := cuecontext.New()
+
+	// Step #1: load the CUE package in the current directory.
+	// It contains a single file - "example.cue", as shown above.
+	bis := load.Instances([]string{"."}, nil)
+	step1 := ctx.BuildInstance(bis[0])
+	fmt.Printf("step1: %v\n", step1)
+
+	// Step #2: load the "data.yaml" file shown above.
+	step2File, err := yaml.Extract("data.yaml", nil)
+	if err != nil {
+		log.Fatal(err)
+	}
+	step2 := ctx.BuildFile(step2File)
+	fmt.Printf("step2: %v\n", step2)
+
+	// Ensure that the result of unifying the two steps is both
+	// valid and concrete - and thus could be exported as data:
+	result := step1.Unify(step2)
+	if err := result.Validate(cue.Concrete(true)); err != nil {
+		log.Fatal(err)
+	}
+
+	// Display the resulting CUE:
+	fmt.Printf("result: %v\n", result)
+}
+{{{end}}}
+
+We fetch the latest version of CUE, and tidy our Go module:
+
+{{{with script "en" "go get etc"}}}
+#ellipsis 0
+go get cuelang.org/go@${CUELANG_CUE_LATEST}
+#ellipsis 0
+go mod tidy
+{{{end}}}
+
+When we run our Go code, it behaves the same as the `cue export` command above
+\- except that it *also* displays the interim `step1` and `step2` values:
+
+{{{with script "en" "go run"}}}
+go run .
+{{{end}}}
+
+{{{with _script_ "en" "https://github.com/cue-lang/cue/issues/3496"}}}
+go vet ./...
+#ellipsis 0
+go run honnef.co/go/tools/cmd/[email protected] ./...
+{{{end}}}
+
+## Related content
+
+- Tag: {{<tag "go api">}} -- Guides exploring CUE's Go API

content/docs/concept/working-with-incomplete-cue/gen_cache.cue

@@ -0,0 +1,131 @@
+package site
+{
+	content: {
+		docs: {
+			concept: {
+				"working-with-incomplete-cue": {
+					page: {
+						cache: {
+							upload: {
+								"example.cue": "lSc4mQOVt13ucRCN8PW9lvC2rWsKYxeLUDqYiH5Fyb0="
+								"data.yaml":   "Sm4bavm+gyoOANH83xv7b1yizv/QDycwh+P/ljI6mw8="
+								"main.go":     "2Vp1JchkoEcbNvo3MzItekAwLTG8qQt5mj+0kUsqtDc="
+							}
+							multi_step: {
+								hash:       "73LHVA359064DFLC4833GSVLSMPGITSI73FLH27N0RGUMM2UN3CG===="
+								scriptHash: "GDKIRP8KI7N6QV8BV02JOD5OJ9HSQ80NMNK77QJGH742TLTIFJAG===="
+								steps: [{
+									doc:      ""
+									cmd:      "export GOMODCACHE=/caches/gomodcache"
+									exitCode: 0
+									output:   ""
+								}, {
+									doc:      ""
+									cmd:      "export GOCACHE=/caches/gobuild"
+									exitCode: 0
+									output:   ""
+								}, {
+									doc:      ""
+									cmd:      "cue eval example.cue"
+									exitCode: 0
+									output: """
+											A: {
+											    b: 2
+											}
+											B: A.x + 10
+
+											"""
+								}, {
+									doc:      ""
+									cmd:      "cue export example.cue"
+									exitCode: 1
+									output: """
+											B: undefined field: x:
+											    ./example.cue:4:6
+
+											"""
+								}, {
+									doc:      ""
+									cmd:      "cue export example.cue data.yaml"
+									exitCode: 0
+									output: """
+											{
+											    "A": {
+											        "x": 5,
+											        "b": 2
+											    },
+											    "B": 15
+											}
+
+											"""
+								}, {
+									doc:      "#ellipsis 0"
+									cmd:      "go mod init go.example"
+									exitCode: 0
+									output: """
+											...
+
+											"""
+								}, {
+									doc:      "#ellipsis 0"
+									cmd:      "go get cuelang.org/[email protected]"
+									exitCode: 0
+									output: """
+											...
+
+											"""
+								}, {
+									doc:      "#ellipsis 0"
+									cmd:      "go mod tidy"
+									exitCode: 0
+									output: """
+											...
+
+											"""
+								}, {
+									doc:      ""
+									cmd:      "go run ."
+									exitCode: 0
+									output: """
+											step1: {
+											\tA: {
+											\t\tb: 2
+											\t}
+											\tB: A.x + 10
+											}
+											step2: {
+											\tA: {
+											\t\tx: 5
+											\t}
+											}
+											result: {
+											\tA: {
+											\t\tb: 2
+											\t\tx: 5
+											\t}
+											\tB: 15
+											}
+
+											"""
+								}, {
+									doc:      ""
+									cmd:      "go vet ./..."
+									exitCode: 0
+									output:   ""
+								}, {
+									doc:      "#ellipsis 0"
+									cmd:      "go run honnef.co/go/tools/cmd/[email protected] ./..."
+									exitCode: 0
+									output: """
+											...
+
+											"""
+								}]
+							}
+						}
+					}
+				}
+			}
+		}
+	}
+}

content/docs/concept/working-with-incomplete-cue/page.cue

@@ -0,0 +1,3 @@
+package site
+
+content: docs: concept: "working-with-incomplete-cue": page: _