Create Async API schema from our own broker's topic hierarchy

[joepavitt]

Tasks

Preview Give feedback

1. Add initial schema generation for team-broker topics #4997
   +0
2. Add Open Schema button to Topic Hierarchy view #5008
   +0

We will auto-generate an AsyncAPI v3 schema based on the detected topic hierarchy for the team broker.

For the first iteration, this is auto-generated and uneditable. Being able to edit the schema (to provide additional descriptions/context to the entries is a future item) is out of scope but will be something we'll want to do in the future, so some consideration should be made for that.

Currently, the auto-detected topics are stored as a simple array of topics. That is likely to change in the future, but for now, we'll stick with that.

We will also be adding the ability to generate schemas for 3rd party brokers in the near future; so choices made here need to accommodate that.

I'm working on the basis that we will generate a separate schema for each broker (for now). A combined schema will be doable, but one step at a time.

Here's an first iteration of an example schema we can generate:

asyncapi: 3.0.0
info:
  version: 1.0.0
  title: dev Team Broker
  description: An auto-generated schema of the topics being used on the team broker
servers:
  team-broker:
    host: broker.example.com
    protocol: mqtt
channels:
  foo/bar:
    address: foo/bar
  top/a/b:
    address: top/a/b
  /top/c/d:
    address: /top/c/d
  top/a/b/c:
    address: top/a/b/c

Notes

1. info.title includes the Team Name + Broker name. For this iteration 'Team Broker' is the only broker. In the future, that will be the name of the relevant 3rd party broker.
2. version is hardcoded to 1.0.0 - its a required field for the schema. We will need to think about lifecycle of schema versions and how the value changes over time.
3. servers.team-broker - not including any username/password details here for first iteration. Will need to think about how to do that securely (if its a requirement) - and also what credentials to include if the Team has multiple clients defined.
4. channels - these are the individual topics. The top-level property is normally a more human-readable name for the channel and the address is the full topic string. Without any further information, I'm reusing topic for both. In a future where we allow users to edit the schema, this will be one of those editable things.

URL & Security

In my initial prototyping, I'm serving the schema from /api/v1/teams/:teamId/broker/:brokerAlias/schema.yml

The :brokerAlias property will allow us to serve schemas for the individual 3rd party brokers. For now team-broker is the hard-coded alias we'll use.

The security on this URL requires a valid session - as usual for our API.

This poses a challenge when we think about how the schema will actually be used. Typically, the URL will be fed into some AsyncAPI consuming tool; however that normally requires a the url to be publicly accessible to the tool. A workaround is the user can export the schema as a local file and then upload it to their tool of choice.

We need to spend some more time working up use-cases for the schema to identify how and where it will be used.

One obvious use case is within Node-RED itself. A custom FF Plugin could deal with the access to the schema under the covers.

Exposing the UI

As a first pass to expose it, I propose we add a suitable button to the topic hierarchy view that links to the schema. Here's my professional mock-up of that proposal:

[knolleary]

Have updated the description with more details of what we can do for a first iteration. Will have a PR for much of this tomorrow.

[knolleary]

UI done in #5008 - this can be closed once that is merged.