Language Changes and Streamlined CLI semantics

This release makes several minor and bigger language changes. The changes are a result of analyzing CUE’s interoperability with other languages and what is needed for this. It is expected that these are the last major changes and that this fixes the look and feel of the language. In addition to the usual cue fmt-based rewrites, we now also provide a cue fix command to aid in the transition. Also, the old formats are still supported for now. Minor language changes and additions to the language are still planned on the path to stability.

The new language changes allow for a simpler JSON Schema mapping. This, in turn, allows for a more straightforward combination of schema and data on the command line.

New syntax for definitions

Before, definitions were indicated with a double colon ::. They lived in the same namespace as regular fields. This property complicated defining automated conversions of some other data formats to CUE. For instance, JSON Schema keeps separate sections for schema and definitions. As there is no enforced convention, as in Go, for naming the two kinds differently, one could not map these to the same struct. This forced schemas to be moved to another location, which turned out to be cumbersome and unnatural.

Another issue with the old syntax is that one could not determine from a reference a.b whether b would refer to a definition or regular field. Not a big problem per se, but it lacked clarity.

As of v0.2.0, definitions are denoted with special identifier starting with a #. As this was not a legal identifier before, such identifiers can safely be used exclusively for definitions. Effectively, definitions now have their own namespace. Other than that, definitions work as before. They still close a struct, they are not output during export, and you simply use their identifier to refer to them.

Before:

A :: b: int
b: c :: string

D: A
E: b.c

After

#A: b: int
b: #c: string
D: #A
E: b.#c

The #c notation is just an identifier: no special syntax is needed to handle them. As before, "#foo" still denotes a regular field with a name starting with #.

Using an initial # to distinguish between definitions and regular fields was inspired by Go's use of initial casing to distinguish between exported and non-exported identifiers. Indeed because CUE is a JSON superset and interop focused, there is little control over casing. As we explain later in the Hidden Fields section, we also resurrect _ as a means of excluding fields and definitions from output.

The new notation may take a bit of getting used to, but the increased readability and the flexibility from the separate namespaces are big wins. So far the feedback has been overwhelmingly positive.

The Go, Protobuf, JSON Schema, and OpenAPI mappings, as well as the tutorials, have been ported to use #-style definitions. Please beware that if one has other templates based on this they need to be rewritten as well.

Limitations

The new definition syntax is more restricted than the old syntax. In general, it is no longer allowed to have dynamic definition names. For instance, the syntax disallows using interpolations for creating names. Also bulk optional fields only apply to regular fields. We think this actually benefits static analysis and is overall a good change. If need be, though, the old semantics can be simulated by containing a struct within a definition used as a map. In case this proves to be too limiting, we have a possible language change up our sleeves that would allow dynamic definitions again.

Old style definitions will keep being supported up to the next minor release. Use cue fix to rewrite current files (see below).

API changes

The current API assumed definitions and fields lived in the same namespace. The current API is therefore inherently broken. The Struct.FieldByName has been deliberately broken to force users to disambiguate. To evaluate a reference, it is recommended to use Dereference instead of passing the result of Reference to Lookup.

There are some good ideas to make the API considerably simpler and more powerful. This is contemplated to be built on top of the new adt package, which is developed as part of the evaluator rewrite, which is planned for the next minor release.

cue fix

To aid in the conversion to #-style definitions, the cue fix command is introduced. This command does a module-wide update of packages by default, but allows specifying individual files and packages.

This is no longer piggybacked on cue fmt. Unlike previous rewrites, the definition rewrite requires evaluation. In addition, not all current definitions are legal in the new syntax. The converter will not remap those. Also, unfortunately, due to API limitations, the converter can also not handle all legal cases. We hope that such cases will be rare, though.

Cases that cannot be handled are clearly marked with locations that may reference the definition included. Definitions with issues have are marked with a deliberately obscure @tmpNoExportNewDef attribute, indicating they need manual fixing

There is also a possibility there will be undetected breakage. This may happen if a selection is made in a disjunction where the field may be either a definition or regular field. The fact that this is even possible indicates that the old model was probably too flexible and possibly more bug prone. With new-style definitions such shenanigans are no longer possible.

Unfortunately, cue fix currently does not handle updating multiple packages within the same directory at once unless manually specified. We intend to allow this at some point.

Tool semantics changes

Previously, the cue tool handled schemas and data differently. In practice, the distinction is not all that clear, making this distinction somewhat “forced”. The cue tool now treats data and schemas as the same thing and always unifies them without distinction by default. This greatly simplifies its use and makes it a very powerful tool for operating directly on non-CUE schema.

There are two exceptions to the new “always unify all files” rules: 1) with cue vet multiple data files are still individually verified against a single schema. 2) the -d option still separates schema from data files. One could argue that the latter isn’t really necessary anymore, as we will see next.

JSON Schema mapping

The following command

cue export schema.json data.yaml

converts the schema.json (which has a proper value for $schema to detect the format) to a schema and data.yaml to data, and unifies the results. This will naturally fail if the contents of data doesn’t correspond to the schema. There is no special logic needed to detect these cases, other than knowing when to interpret a data file as JSON Schema.

Note that #-style definitions were key to make this possible. Without them, there would be no obvious way to map JSON Schema to the root of a config as merging fields and schema to the same namespace could result in conflicts.

OpenAPI merging

Another example of the new capabilities is that one can now merge schema as one would merge data. For instance, the following command merges two OpenAPI files and then outputs it again as OpenAPI:

cue def openapi: file1.json file2.json --out openapi

The user won’t even see the intermediate CUE.

Hidden fields

This release also officially resurrects hidden fields (identifiers written as, for example, _foo). They were removed from the spec as with the introduction of definitions they were believed to be an incongruent feature. They fit, however, quite nicely with the new-style definitions. They were still in use and proved to be more useful than expected. As they were never really removed from the implementation nothing changes. However, their comeback is now official.

For clarity, both hidden fields and definitions are not shown in exported output. They serve different functions, though.

A definition defines a new composite type, the most general specification of something. They are at the other end of the spectrum to concrete instances. As definitions are supposed to fully define a schema, CUE can use them to catch typos in field names by detecting fields that are not supposed to be there.

A hidden field can be any value and is not subjected to such scrutiny. They are used to define fields that are not converted to output without making them a definition.

Although this is not yet enforced, hidden fields are local to a package. To make the matrix complete identifiers starting with _# denote hidden definitions, or definitions local to a package. These can be used today, although this restriction is also not yet enforced.

Streamlined syntax for list comprehension

List comprehensions are now written as

[ for x in src { x + 1 } ]

This brings them in line with field comprehensions, where the value also comes after the comprehension clauses. Overall, this greatly increases readability.

This also harmonized the syntax. It simplifies both spec and implementation, and prepares for some of the constructs mentioned in the query proposal.

The one thing that may require some getting used to is the curly braces alluding to the value being a struct. If the curly braces contain a single scalar value, however, it is promoted to be the result of an iteration. CUE users may already be familiar with this construct at the top-level scope, where “embedded scalars” are the final result of evaluation. This construct was originally invented to allow omitting curly braces, while still being a superset of JSON (JSonnet uses the same trick). It turns out, however, that this trick can be quite useful when generalized to apply to any struct, not just the top-level one. We continue to investigate this possibility. Con...

JSON Schema and OpenAPI fixes and improvements

This release contains several fixes and enhancements related to CUE conversions.

It fixes several issues with the related to JSON Schema conversion, including an incorrect implementation of the JSON Schema spec. Most notably, one may not infer from a constraint for a certain type that a value must be of that type. This means that if a schema, for instance, defines a properties section, but does not specify the type to an object, the resulting CUE value will be a disjunction from this object and all non-object top-level types. This is not a bug.

Another change is that the cue tool now may silently ignore imprecise conversions, as long as the produced result does not give false negatives. The --strict option will emit the original errors.

Changelog

0059b2b cmd/cue/cmd: allow lossy conversion by default and add --strict option
a38e811 cmd/cue/cmd: fix bug with help flag
b99d364 encoding/jsonschema: add several validators
bcc19ce encoding/jsonschema: encode minItems differently
78e21c1 encoding/jsonschema: fix type exclusion logic
ed4fc4d encoding/jsonschema: wrap comment lines
904f561 encoding/openapi: remove unused code
9b62b36 encoding/protobuf: fix required option parsing
efb89b8 github: fix release.on to push.tags

Stability and cosmetic improvements, can use CUE as OpenAPI encoding

The release fixes a critical regression that affected cmd eval as well as various other minor regressions and documentation and formatting issues.

Support for space-separated labels and old-style field comprehensions is now disabled by default. The intention is to remove these entirely soon.

This release now also allows OpenAPI inputs to be expressed in CUE itself, instead of just JSON or YAML.

Changelog

6f37a71 cmd/cue/cmd: fail earlier on faulty flags
83b0996 cmd/cue/cmd: fix newline discrepancies
cd2b3ef cmd/cue/cmd: fix regression that allowed space-separated fields again
d6fc6be cmd/cue/cmd: link commands help text
c5fc236 doc/tutorial/kubernetes: update to minimize diffs down the line
8dcec3c internal/encoding: allow CUE as OpenAPI input
d6f1eec internal: fix crash affecting eval

Streamlined and enhanced CLI capabilities

This release introduces various improvements to the cue command, including a streamlined CLI interface and improved interoperability between different file formats.

Many of the command-line flags were previously rather arbitrarily implemented for some commands and not others. This release fixes most of that. This release also makes it possible to convert between different formats, such as YAML, JSON, JSON Schema (input only), OpenAPI, and Protbuf (input only).

This release also starts a new versioning strategy. CUE exists of several fairly independent parts. Each next milestone focusses on a one of these parts. This round it was the cue command-line tool. Next round will be the evaluator. Each next minor version will correspond to one such milestone. This is not to say that upgrades and bug fixes cannot happen to other parts within a cycle—they can—but it helps focus.

The new strategy will also make it easier to do intermittent releases. We will use the minor releases to signify big additions and backwards-incompatible changes. Patch releases may include backwards-incompatible changes that include bug fixes or disabling previously deprecated features. We intent to follow this trategy until providing a general backwards-compatibility guarantee for v1.0.0. As usual, we intend to provide a smooth migration path for each incompatible language change.

As of this release, a Docker image is generated as part of the release process.

Define file types

A frequently asked feature was to be able to interpret files as anything. CUE now allows qualifiers to "cast" a file to any supported format. For instance,

$ cue eval json: foo.data

interprets foo.data as JSON and then prints it as CUE. This also shows that the input for eval no longer needs to be CUE but can be any format that is convertible to CUE.

This means that CUE can now also be used between non-CUE formats. For instance,

$ cue export foo.json -o foo.yaml

will convert JSON to YAML.

Packages can now be qualified by package name (for instance ./foo:bar), not just directory.

Run cue help inputs and cue help filetypes for more information.

Streamlined flags

The import command already had a set of flags that maps JSON and YAML files to a prescribed location within a CUE configuration. These flags are now also available for many other commands so that JSON and YAML files, as well as other formats, can be processed by the cue command without first having to convert them to CUE.

For instance,

$ cue eval -l 'strings.ToLower(kind)' -l name foo.yaml

puts any document in foo.yaml at a path based on its kind and name fields.

Also the output flags (file and format), among others, have been harmonized across commands. This sometimes led to backwards-incompatible changes.

Run cue help flags for more information.

CUE as a data format

The qualifier mechanism can also be used to restrict what language constructions can be used in .cue files. Most notably,

$ cue eval data: foo.cue

specifies that the file foo.cue may only contain data fields, but no references, definitions, builtin invocations, or any other CUE construct other than simple data. In other words, this allows CUE to be used as an alternative JSON or YAML format.

In the future we may add a file extension to indicate this CUE dialect.

New cue def command

The cue eval command was configured to generate cue that seemed pleasing to read to humans, but it would generate something that is neither useful as data nor as a schema. cue export could already be used to generate data, but there was nothing to conveniently generate a schema.

cue def is also the command of choice converting schema-based formats to CUE.

Value injection

Many users have requested a convenient way to defining environment-specific variants of configurations and ways to inject values based on environemt variables and command line flags.

We have added support for environment variables at various points in the tooling layer. Due to their non-hermetic nature, though, environment variables are not an appropriate mechanism for CUE configurations in general.

An important criterion for allowing such injection was retaining the ability to analyze configurations and make intent of how to use a configuration explicit. Simultaneously, we did not want to impose on the structure of a configuration.

Our solution is to use attributes to explicitly mark injection points. A field that with a "tag" attribute of the following form:

environment: "prod" | "staging" @tag(env)

can be set to "prod" on the command line using -t env=prod. The tag attribute may also explicitly define some shorthands. For instance, by specifying

environment: "prod" | "staging" @tag(env,short=prod|staging)

the same can be accomplished using -t prod.

This mechanism makes it very clear to both the user and the cue tooling which non-concrete values may be set by the user and which should be resolved by values within a configuration.

In the future we may consider to also allow whole-file injection analogous to Go build tags.

This release does not include support for command-line flags in the tooling layer. This is still being considered for a future release.

Run cue help injection for more information.

OpenAPI support

The cue tool now has basic support for converting OpenAPI definitions to CUE. This is a first stab, does not yet support external references, and is not as feature rich as the CUE-to-OpenAPI Go API that is used by Istio to generate its OpenAPI. But it should be a good start.

OpenAPI definitions can be imported using cue import or just used on the fly using cue eval or cue def. OpenAPI can be defined in JSON or YAML. For instance:

$ cue def openapi: foo.yaml

will interpret the foo.yaml file as OpenAPI and generate corresponding CUE.

It is also possible to convert CUE to OpenAPI. OpenAPI can be represented in JSON, YAML, or even CUE. For instance,

$ cue def ./pkg -out openapi+yaml

writes the evaluated CUE as OpenAPI in YAML format to stdout.

JSON Schema support

The cue tool has very basic support for mapping JSON Schema to CUE. This is still experimental and the converse (mapping CUE to JSON Schema) is not yet supported. The typical conventions followed in OpenAPI make it easy to find a mapping from OpenAPI to CUE. The same does not hold for JSON Schema. This is partly complicated by fields and definitions sharing the same namespace in CUE. Also, the top-level schema of JSON Schema is unnamed, making it hard to find an appropriate destination.

There is a proposal for CUE to have definitions and fields live in a separate namespace. This will make such a mapping considerably easier. For now, consider the mapping experimental.

As with OpenAPI, external references and anchors are not yet supported.

Auto detection of file types

The cue tool now interprets files with a .json, .yaml and .yml extension as either data, OpenAPI, or JSON Schema based on markers in the data. For JSON Schema this is a $schema field that refers to a schema on json-schema.org. OpenAPI is selected if a file contains a top-level openapi field with major version 3, an info.title, and an info.version field.

The use of qualifiers disables auto detection. So auto detection for a JSON file, for instance, can be disabled using a json: or data: qualifier.

cue import modes

The cue import tool now has import modes that specify which kinds of files to import. The default is the old default: JSON and YAML files, but with auto detection enabled.

Run cue help import for more details.

Protobuf support

The new cue import proto command can import Protobuf definitions.

A successful run imports all transitive dependencies of a .proto file into a CUE module: .proto files located within the module are converted to a CUE package at that location. The package name is derived from the go_package option, if present. Imported .proto files that can be found in a given include directory are put in the corresponding path in the cue.mod/gen directory.

The existing Protobuf API has been modified to generate CUE types hierarchically (instead of mapping all files to the top level) to make them closely correspond to the Protobuf definitions.

Protobuf definitions can be annotated with CUE-specific options to specify additional constraints for fields.

Command and Task changes

The task section in commands is now optional. A task can be included anywhere in a command. So commands with a single task can now be written as

import "tool/cli"

command: echo: cli.Print & {
   text: "Hello world!"
}

The cue tool will look for tasks anywhere nested within the run command where each task is considered a leaf node in this nesting.

Tasks can also be defined at the top of the file. These will only be run if they are referenced by a task defined in an executed command

Sometimes task must run after another task, even though it does not depend on any of its outputs. The $after field can be used in that case to explicitly mark a dependency.

See https://pkg.go.dev/cuelang.org/[email protected]/pkg/tool?tab=doc for more information.

Vet changes

In addition to the flag restructuring, it is now possible to check a data file against a CUE package.

A backwards-incompatible change is that one now needs to use the -d flag, instead of the -e flag, to select a schema against which to compare. The -e flag was used by other commands to mean something different and this was cleaned up for consistency sake.

Trim safeguard

The cue trim command now checks ...

Bug fixes and stability improvements

This release focusses on bug fixes and stability improvements.

There is one language change, and that is to allow the $ character in identifiers. The main reasons for this is that it is a commonly used character in identifiers in JSON. The single $ is currently reserved.

Changelog

6f9c049 Add YAML support to export command
503fecd Decode null as NULL token in yaml decoder
c4f74cc Revert "cue: fix lookup for cue API"
8a043dc Use baseValue from source in binOp to make gc working
4dc023b Use cache during walk
170697a cmd/cue/cmd: also make field optional if it is a pointer
d372ba8 cmd/cue/cmd: convert package block comments in cue get go
d100ae6 cmd/cue/cmd: small help command fixes
a9fdbe7 cmd/cue/cmd: track use of indexing op for dependencies
f16c065 cue/format: improve indentation of in comprehensions
8e72f0b cue/load: cleanup and made code more defensive
ff96e8d cue/parser: improve error message for misused keywords
8a2df96 cue/scanner: allow $ as a letter in identifiers
b3cde9d cue: Reference handles concrete values for index operations
3a2e63b cue: always check for errors before calling builtin functions
630935e cue: detect errors in struct when comparing to bottom
afdbec3 cue: disallow references to string labels (per spec)
5c2d937 cue: don't turn of manifesting in normalization
324cc0f cue: don't validate optional fields
279ca6a cue: fix bug related to non-structural direct self-references
13628f1 cue: fix compilation bug using $foo identifiers
3d0b204 cue: fix crash on alias use in lists
40d2667 cue: fix exporting of label aliases
c937e7a cue: fix lookup
353ede2 cue: fix lookup for cue API
8bd2426 cue: fix lookup for cue API
52b04d4 cue: fix optional handling in subsumption
6cd62ea cue: ignore optional fields for Equals
5a0a3f2 cue: no quotes needed for identifiers with $
d256a2c cue: prevent regression to bug in older version
030b9b7 cue: remove uses of log.Panicf
d7d2837 describe the go get step before cue get go in the tutorial
c8d6c39 doc/ref/spec.md: clarify newline behavior for multiline strings
c6e5d17 doc/ref/spec.md: fix typo
4d29dde doc/ref/spec.md: simplify attributes
0e038f6 doc/tutorial/kubernetes: update to latest
3960473 doc/tutorial/kubernetes: update to use new syntax consistently
8f9ef31 doc/tutorial/kubernetes: update to v1 versions
815dca2 doc/tutorial: fixing bad link in string literals
80ffe27 doc/tutorial: removing duplicate word in bounds
5b4fa8b doc: fix typos in language spec
085ce9c fix typos in docs
31cd2b7 internal/diff: first stab add diffing functionality
5cb6ffe internal/yaml: import null value properly
7fdff9c pkg/math/bits: complete operations
7334a13 pkg/math: use higher precision implementations
6ffe55d tools/trim: fix spurious removal of top-level declarations
cba0372 tools/trim: move trim logic to package

Major syntax update

This release is characterized by a major syntax change. We don't take such changes lightly. The new syntax makes the language more regular (and simpler to parse), prepares it for some key features, and, we believe, makes it easier to understand. This is also a major step towards stabilizing the grammar of the language to the extent we can start guaranteeing backwards compatibility.

The old syntax is still supported. The cue fmt will rewrite your files from the old syntax to the new one.

Colons instead of spaces as separators for shorthands

CUE allows single-field structs to be written without curly braces if they are declared at another field. In the old syntax

a: {
    b: {
        c: 4
    }
}

could be written as

a b c: 4

In the new syntax, this is written as

a: b: c: 4

Aside from enabling some new features and simplifying the parser, this syntax also allows the shorthand to be used for both definitions and regular fields. For instance,

a: b:: c: int

previous had to be written as

a: {
    b:: { c: int }
}

Bulk optional fields

The new syntax now allows optional fields to be specified in bulk:

// Define a map from string key to numbers
numMap: [string]: number

// Apply constraint T to all fields with a name ending in "Test"
[=~"Test$"]: T

Using [string]: number is very similar to using a "bind label" previously. More on that later.

The old syntax for optional fields

foo?: bar

is now just syntactic sugar for

["foo"]: bar

Label filters are the equivalent of patternProperties in JSON schema.

Note that using colons as separators, which often have a connotation of defining something, interacts well with this language construct. For instance, a map of maps of ints is written as:

[string]: [string]: int

More aliasing

Shadowing of fields is a common issue in CUE (or similar languages, for that matter). CUE already supported various mechanisms to deal with this. For instance, in this example an alias is used to reach the outer a:

A=a
a: c: a: A.b
b: 4

Other tricks up CUE sleeves were quoted identifiers and allowing to make field non-referable using double quotes. Together these tricks proved both cumbersome and inadequate. The existing techniques also proved to be hard to handle for code generators and rewriters.

CUE now extends the aliasing model to field and label aliases. A field alias,

X=foo: bar

binds X to bar just as foo does. This allows the above to be written as

X=a: c: a: X.b
b: 4

This is not a great win, bit in many cases it will be. Moreover, it allows defining references to fields that can otherwise not be referenced, such as

X="foo-bar": baz
Y="\(x)Test": y

CUE now also allows label aliases, which are placed inside the square brackets:

[X=string]: { name: X }

These are only visible within the field's value. This replaces the "bind" labels, such as <Name>: { name: Name }. These are essentially the same, but explained differently and with a different syntax. The old syntax makes CUE hard to parse and is less flexible and will be phased out.

Packaging

Another change is the way the cue tool manages packaging. The cue.mod file and pkg directory are replaced with a cue.mod directory marking the root of a CUE module and with contents managed by the cue tool, analogously to how the git tool manages the .git directory.

See https://cuelang.org/docs/concepts/packages for more details.

Deprecations

Fields names starting with double underscore __

This was defined in the language specification, but previously not enforced. It now is. Field aliases make it easier to work around this restriction.

<X> syntax

Bind labels, or templates, are being replaced by the new optional field construct.
cue fmt will rewrite the old format to the new one.

Back-quoted identifiers

Back-quoted identifiers (i.e. foo-bar) were introduced to improve reachability of fields in case of shadowing. But they only solve half the problem. Field proved to be a more general solution, making back-quoted identifiers a redundant part of the language. As they are not trivial, they will be removed. As for now, the cue tool will rewrite programs using back-quoted identifiers using aliases.

Changelog

fe8ab34 Fix typo (#164)
2cd4162 README.md: bump supported go version to 1.12+
391e7ae Readme.md: add GolangCI badge
9707302 cmd/cue/cmd: added cue mod init command
fd127fe cmd/cue/cmd: clarify location for commands in error message
f979aa6 cmd/cue/cmd: correct module output
329cd5b cmd/cue/cmd: get go: convert Go block comments
29f70f2 cmd/cue/cmd: report error for certain deprecated syntax
72b36fb cmd/cue/cmd: update more examples to the new format
26a1a0e cmd/cue/cmd: update trim documentation to new syntax
b763fde cmd/cue: rewrite quoted identifiers
9752844 cue/ast: add NewList helper
4759dd2 cue/format: formatting of nested single-line fields
0b4f016 cue/load: allow cue.mod to be directory
b34bce7 cue/load: allow cue.mod to be directory
b13155b cue/load: make cue.mod the default desitination
60b298b cue/load: revert: report when "tool" pkg is used in non-tool file
312fca9 cue/parser: disallow declarations with __foo identifiers as per spec
7057dde cue/parser: fix comment attachement issue
4177df9 cue/parser: fix comment attachement issues
98b8d2f cue/parser: record position of deprecated feature
fa59c10 cue: generate builtins to update task documentation
8192b54 cue: generate new-style maps when exporting
ff34272 cue: implement key filters using new syntax
300af3e cue: introduce colon-separated labels
6d8c95d cue: more conversions to new style templates
467d7bc cue: move to square brackets
39df6c9 doc/ref/spec.md: bug fix: add solidus to escaped_char
c7791ac doc/ref/spec.md: introduce colons as separators
9ffcbbc doc/ref/spec.md: write out spec for aliases and optional field sets
2437f9d doc/tutorial/kubernetes: update formatting
23623fa doc/tutorial/kubernetes: update to new syntax

Streamlining embedding semantics

This release introduces two changes that may break existing configurations. One is really a bug fix relative to the spec.
These changes were not included in v0.0.12 to allow some of the bug fixes of the latter release
to be available before introducing these changes.

Top-level "emit" structs are embedded

Previously, a top-level embedded struct had different semantics from embedded structs within structs itself.
For instance, this configuration

{ 
  a: 3
}
b: 3

would previously yield { a: 3 }, but will now yield { a: 3, b: 3 }.
This semantics was a leftover from before CUE supported embedding.
But keeping this around was just too confusing.

Other values, like lists or strings retain their original semantics.
However, for consistency sake, these may only be accompanied by definitions.

So previously this was allowed

"Hello \(who)!"

who: "World"

but must now be

"Hello \(who)!"

who :: "World"

Failing Definitions are an error

Previously a definition was, unlike a normal field, not an error.
This was not according to the spec and this has now been rectified.
To allow a definition to fail, make it optional (using a `?).

Changelog

a3c7bef cue: change emit value semantics
3022ae9 cue: fail if non-optional definitions are bottom
0d0b9ad doc/ref/spec.md: add newline before comments

Relaxed closed structs

This release contains a host of bug fixes and improved conformance to the spec.
The most notable change, however, is the relaxing of the closed struct constructs.

Relaxed closed structs

With the previous release, each struct literal in a definition was treated strictly as closed.
This turned out to be both tedious and counter-intuitive.
This led to a spec change and corresponding different implementation where
a definition is only closed once it is referenced.
This means that merging literal structs defined within a definition works exactly
like unification of regular structs up to the point they are referenced.

Features

There are several builtins added, including in the list and regexp packages.
The latter added builtins to extract fields from strings into a struct using named groups.

Changelog

30a6c2b README.md: add Github build badge
75b9c7f all: remove large dependency
9e294ac cmd/cue/cmd: add MainTest
22e054b cmd/cue/cmd: always print newline at end of eval
0b043c9 cmd/cue/cmd: auto-detect test mode
e606e0d cmd/cue/cmd: don't use os.Exit and fix error output
91c1684 cmd/cue/cmd: fix import paths for go get
48b8f9a cmd/cue/cmd: fix race
374d395 cmd/cue/cmd: fix trim help
78043ab cmd/cue/cmd: fix: disable elimination of slices
bd64ebc cmd/cue/cmd: import: use astutil
a31e1ac cmd/cue/cmd: only allow dot files when specified explicitly
18dab4b cmd/cue/cmd: report instance-level errors
0489caa cmd/cue/cmd: use testscript for top level command testing
22a42d4 cmd/cue: fix tests under different platforms
c7dc930 cue/ast/astutil: allow recursive application
1a33709 cue/ast: reorganization of node types
929e71f cue/format: indent single line instead of 2 after embedding
de86f0f cue/load: be more permissive with file loading
ca66319 cue/load: report when "tool" pkg is used in non-tool file
a38f5bc cue/parser: add support for failing on legacy grammar
6eefcd0 cue/scanner: fix JSON compliance
e1656a1 cue: add Definitions option and deprecate Hidden
7d1cb13 cue: add Kind.String method
ffff7fa cue: add test for Issue #94
8776f21 cue: allow valid reference cycles in export
75d2c37 cue: bind reference as lookup in parent node
8927b63 cue: bug fix: retain references with interleaved embedding
f2654de cue: close struct only after referencing
081251e cue: document that zero value of Runtime is OK to use.
8d81be4 cue: enable recursive opening when embedding
d282553 cue: exclude definitions from value lookup
fbb38bc cue: fix comparing against bottom
d499c46 cue: improve dependency analysis for References
405a191 cue: improved error message
6deb0df cue: make users aware of the --list flag when encountering a list
4524da6 cue: move up processing of comprehension
c85bebe cue: pass on compile-time errors
fa7e3ce cue: relax rules for closed structs
827ebfb cue: remove IsValid
7068dea cue: workaround for empty "or" lists
21f6c44 doc/ref/spec.md: bring scoping in line with reality
b2703c6 doc/ref/spec: reflect reality w.r.t. octal numbers
ded0e1d doc/ref: fix typo in spec
9d53106 doc/tutorial/basic: number chapter directories
27b4ea5 doc/tutorial/basic: use definitions with "emit" examples
a9f25f8 doc/tutorial/basics/intro: disable test
00eb9d2 doc/tutorial/basics: add pieces on structs and definitions
934fec9 doc/tutorial/basics: correct name of section
f3cd848 doc/tutorial/basics: generate testscript cases
d4c1734 doc/tutorial/basics: order files
b0d1a07 doc/tutorial/basics: remove old docs
df08087 doc/tutorial/basics: reorganize and simplify intro
aeaf006 doc/tutorial/basics: update some sections
1ce0c51 doc/tutorial/kubernetes: tutorial fixes
d437bd5 doc/tutorial/kubernetes: update definitions
b0bba12 doc/tutorial/kubernetes: update definitions
78783f2 encoding/json: add JSON exporter
dd75ef6 encoding/json: belated fixes of review
9b70cf3 encoding/openapi: add paths:{}
7e4dc22 encoding/openapi: support Definitions
7213fa9 encoding/protobuf: don't encode default values for enums
e7abb20 encoding/protobuf: don't use disallowed "__" prefix
074cf5d pkg/list: adding flatten function as a builtin
a83af29 pkg/list: adding flattenN function as a builtin
5ac1fbf pkg/list: adding range function as a builtin
e783813 pkg/list: fix MinItems
20b0a0a pkg/list: remove Flatten and simpilfy FlattenN
68a054b pkg/regexp: add more regexp builtins
8dd325b pkg/tool/exec: fix null bug for stderr and stdout
67169f4 pkg: exclude some builtins from generation
c40e450 pkg: update documentation and run gofmt
8529d77 spec: fix some backtic-quotes for markdown rendering
a76cbed start using GitHub Actions as CI

v0.0.11

A few bug fixes

There were a few small bug fixes that, even though old, were serious enough to warrant a new release.

This also finalizes the implementation of allowing full expressions for embeddings.

Changelog

e53305e cue/parser: allow full expressions for embeddings
0ac550c cue: eliminate generation of block quotes
7c9b88c cue: fix bug in type inference
fdd9b71 cue: fixes number groundness inference

v0.0.10

Language touch ups

New-style Comprehensions

This release is characterized by the redesign of field comprehensions, now struct comprehensions. The old format is still supported, but cue fmt will rewrite it to the new format.

The new format looks like:

for k, v in a {
   "\(k)": v
}

instead of

"\(k)": v for k, v in a

This may seem like a gratuitous change, but it has many advantages:

• generated structs are syntactically and semantically like embeddings
• large comprehensions are clearer
• a single iteration can add multiple fields
• makes scoping clearer

Removal of block quotes

Because a comprehension may start with if, we can use if false { ... } to disable a large block. This, and various other considerations have led to the decision to remove block quotes from the language.

Block quotes are still supported, but they are no longer generated and cue fmt will translate them.

Quoted identifiers

CUE now allows identifiers to be quoted with back ticks:

`foo-bar`: 2

baz: `foo-bar`

This allows fields with odd names to still be referenced using identifiers. More importantly, the allowed space of identifiers should ideally be limited. For instance, it is a good idea to require them to be in Unicode normal form NFC. Allowing arbitrary strings as identifiers seemed like a bad idea. CUE will soon disallow this. For now, cue fmt rewrites strings used labels is they are referenced by some nodes.

Quotes also have another benefit. A disadvantage of the new style comprehensions is that it is harder and less clear to allow the keywords of comprehensions as field names. Consider for instance:

for x in src: 2

CUE will now only allow one such keyword as a field name, but requires quoting otherwise. A future cue fmt may rewrite this as:

`for` x `in` src: 2

Backwards incompatible changes

The language changes introduced do not break anything, as the old formats are still supported. However, at some point they won't be. So be sure to run cue fmt at some point.

• The ast.LabelName has been changed to have proper semantics in light of the upcoming changes. The signature has been change to reflect the changes properly and to assure code is properly investigated in light of these changes.
• Comprehensions now iterate only over non-optional regular fields, as would perhaps be expected.
• Top-level emit values are printed in eval output
• Top-level struct embeddings are merged in the API

Changelog

b50aecd cmd/cue: make fmt rewrite deprecated cue
21fa535 cue/ast/astutil: export Resolve functionality
0dcf17b cue/ast/astutil: implement Apply
e4523e2 cue/ast/astutil: pick name based on import path
530467a cue/ast/astutil: support adding imports
24d14c5 cue/ast/astutil: use unlikely name for import
40c76b7 cue/ast: add API for setting a position
f0bcfb9 cue/ast: add API to modify comments
92408e3 cue/ast: add back LabelName
749a667 cue/ast: convenience constructors for common types
fe5f886 cue/ast: first step to new comprehension format.
57d6751 cue/format: expressions.input
9af9a90 cue/format: output new comprehension format
982b194 cue/format: tweaks to support embeddings
71eafc4 cue/parser: parse new comprehension syntax
3c38bfc cue: add test for package name override
4245fb4 cue: for structs, iterate only on non-optional regular fields
835ed58 cue: implement quoted identifiers
da00c8b cue: implementation of new comprehensions
7fc421b cue: remove block comments and slices from the language
1fa0552 cue: tighten API of comprehensions
b36d81a cue: various cleanup
1f5a903 doc/ref: allow embeding to be Expression
d44b092 doc/tutorial/basics: some minor fixes
48f82f1 doc/tutorial/basics: update w.r.t. spec.
15aeace pkg/list: add Haskell-style Take and Drop
1ffb95f pkg/list: adding slice function