Make bundle file available to reviewers for CSAPI Part 1

[Sam-Bolling]

This public comment is respectfully submitted by the Web Service Technical Panel (WSTP) of the Defence Geospatial Information Working Group (DGIWG). This comment addresses an identified issue where there is no completely self-contained OAS definition yaml file, also known as the “bundle file,” available for the OGC API – Connected Systems – Part 1: Feature Resources. Most code generation / automation tools expect a single schema file. The lack of this resource during the review period introduces a significant challenge for implementers and reviewers to evaluate the standard.

Request: For the purposes of reviewing, please provide and declare a completely self-contained OAS definition yaml file for OGC API – Connected Systems – Part 1: Feature Resources, in the SWG-decided version of OAS. It is recommended that the tool, Swagger CLI, used for creating OGC API – Maps and OGC API – Tiles bundle files is reused for this effort. A reference of this process is described at https://github.com/opengeospatial/ogcapi-maps/tree/master/openapi

[alexrobin]

We'll work on making the bundle file available but note that the API documentation is also available online for review purposes.

[alexrobin]

Discussed during 10/31 telecon:

@SpeckiJ will try to generate an OpenAPI 3.1 bundle with a tool he developped.

Or try bundling with https://redocly.com/redocly-cli.

[SpeckiJ]

Sorry for the delay - i built the bundle after the last telco but apparently forgot to communicate that i did and the github-workflow is apparently not public ..

The files are available here (as GitHub does not allow yaml attachments nor publication of workflow results): https://52north.org/delivery/ogc/openapi-connectedsystems-1.bundle.yaml

I have not tested them thoroughly.

Bundling of Part 2 is blocked by #81

[SpeckiJ]

Note: The yaml is quite large, most of the online-editors/viewers i found do not really cope well with the spec and warn of an "resource exhaustion attack" before consuming all the RAM and freezing.

[alexrobin]

@SpeckiJ I'm not surprised. That's why I had to modify redoc so it loads the schema pieces lazily. The change was minor since redoc already had a mechanism for doing this but it was not used by default.

[ghobona]

@SpeckiJ We usually validate API definition documents using SwaggerHub because it is one of the platforms we use for publishing API definition documents.

You could also try validating the file using spectral.

[ghobona]

Here are the steps when using spectral through docker.

First create a ruleset file in the folder that has the OAS 3.1 bundle file.

echo 'extends: ["spectral:oas"]' > .spectral.yaml

Then, from the folder that contains the ruleset file, run the following command.

docker run --rm -it -v $(pwd):/tmp stoplight/spectral lint --ruleset "/tmp/.spectral.yaml"  "/tmp/openapi-connectedsystems-1.bundle.yaml"

After a few seconds, I get the following error.

<--- Last few GCs --->

[1:0x7ffffcb34330]    48557 ms: Scavenge 2032.5 (2074.8) -> 2029.6 (2075.5) MB, 2.4 / 0.0 ms  (average mu = 0.297, current mu = 0.261) allocation failure; 
[1:0x7ffffcb34330]    48564 ms: Scavenge 2033.0 (2075.5) -> 2030.0 (2075.8) MB, 2.5 / 0.0 ms  (average mu = 0.297, current mu = 0.261) allocation failure; 
[1:0x7ffffcb34330]    48571 ms: Scavenge 2033.4 (2075.8) -> 2030.4 (2084.3) MB, 2.8 / 0.0 ms  (average mu = 0.297, current mu = 0.261) allocation failure; 


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

[SpeckiJ]

@SpeckiJ I'm not surprised. That's why I had to modify redoc so it loads the schema pieces lazily. The change was minor since redoc already had a mechanism for doing this but it was not used by default.

@alexrobin : I have debugged this a bit and will probably add a PR to redoc fixing their circular-dependency-detection to not allocate Gigabytes of Memory: Progress can be tracked here: Redocly/redoc#2613

For Swagger-UI i was not so lucky, while their domapi library correctly parses our bundled schema, into their internal format and displays it at first, after clicking any button the UI triggers a landslide of deepCopy calls that then freeze the system ...

[SpeckiJ]

You could also try validating the file using spectral.
After a few seconds, I get the following error.

@ghobona spectral probably also has a flawed/broken circular-dependency detection and then tries to dereference stuff until all the RAM is gone ...

Unfortunately these circular dependencies are conceptually rooted in SensorML and not in the CSA spec itself, so we probably will have to get creative to make all the tools/linters happy somehow ...