cmd/cue: JSONSchema import translates a definition with nested sub-definitions as being equal to top

[jpluscplusm]

Is your feature request related to a problem? Please describe.

I'm cue importing a specific JSONSchema document, and a "definition" that it contains is translated into an empty constraint, unifying it with top. (I'm filing this as a feature request as the emitted CUE isn't technically invalid or incorrect, but is merely unhelpfully unconstrained (modulo #2698, filed in parallel with this, and which /feels/ unrelated)).

The source document is wget'd in the txtar repro, instead of including it here, as it's over a thousand lines long. The emitted CUE is <25% of that, but feels too long to embed in this Issue. I'll assume you can fetch, import, and view the emitted CUE.

exec wget -q https://raw.githubusercontent.com/buildkite/pipeline-schema/6396f68d5e983e0d2acbf829c565027a4cfd69bc/schema.json
exec cue import schema.json
exec grep commonOptions: schema.cue
cmp stdout the-only-constraint.txt
-- the-only-constraint.txt --
#commonOptions: _

Describe the solution you'd like

I'd like cue import to delve deeper into the contents of commonOptions and include its constraints in the CUE that's emitted.

Describe alternatives you've considered

I've considered hand-cranking a schema.

Additional context

commonOptions contains several different fields which are used as isolated containers for constraints, referenced elsewhere in the source document. cue import appears to correctly figure this out, emitting references to contained sub-fields that should be expected to exist inside commonOptions.

IIRC @myitcv suggested the possibility that it's the "$ref" use of a field inside commonOptions (and not directly referencing a top level JSONSchema "definition" such as commonOptions itself) that's prompting cue import to emit the useless top level _ constraint and to omit the constraints contained within the definition's member fields.

[rogpeppe]

The reason for this is that definitions should contain a list of schemas, but commonOptions is not a schema: it's a map that contains schemas. In general, referring into non-schemas is (not necessarily supported](https://json-schema.org/draft/2020-12/json-schema-core#section-9.4.2), and if we see an arbitrary map entry that's not a keyword, it should be ignored.

That said, the new refactored reference implementation does generate schemas for internal structure that's been referred to directly within the schema itself, and this does indeed in this case result in the definitions in commonOptions being defined, albeit in a private field. Initially it won't be possible to change that on the command line, but it certainly could/will be in the future.

[myitcv]

@rogpeppe - could/should we be suggesting an upstream fix in this case?

[rogpeppe]

@rogpeppe - could/should we be suggesting an upstream fix in this case?

Yup. As it turns out, I've actually done that already - I just forgot that that fix was for this particular schema :)

[myitcv]

@jpluscplusm @rogpeppe - please can you confirm if there is anything outstanding here therefore?

[rogpeppe]

The only outstanding thing here is that there's currently no way on the command-line to specify that all subschemas are made available as part of the API. It should be possible to work around that shortcoming in this particular case by adding some code in the same package, but I've created #3628 to track the more general request.

I'll close this as fixed for now.