Message Clinic & Oem

[commonism]

but this project does not have discussions, so …

Hi,

I'm with Redfish as well, targeting things from the python side.
I use commonism/aiopenapi3 to create a dynamic OpenAPI client from the description documents and just started creating a thin layer on top to make things pretty.
I just published aiopenapi3_redfish to accompany this issue - it requires some changes to aiopenapi3 to be merged
aiopenapi3 uses pydantic for data validation, so I'm aware of the differences in specification and protocol implemented, as well as the inconsistencies in the description documents themselves.
But aiopenapi3 allows to interfere with the description document and message processing to align the description document to be valid OpenAPI and the Messages to comply to the description documents specification, for redfish this section is called the clinic.
The clinic is the alternative to waiting for a vendor to come up with firmware updates.

Some examples of what the clinic does does for Dell (d9-6.10.80.00-A00) …

Task as returned by an Action (here "#OemManager.ExportSystemConfiguration) is invalid, it does not have a JSON body and all you get is the Location header.
The clinic creates the minimum required to make it a valid message to comply to the description document.

class ExportSystemConfiguration(aiopenapi3.plugin.Message):
    def received(self, ctx: "Message.Context") -> "Message.Context":
        import json

        if ctx.request.path not in (
            "/redfish/v1/Managers/{ManagerId}/Actions/Oem/EID_674_Manager.ExportSystemConfiguration",
            "/redfish/v1/Managers/{ManagerId}/Actions/Oem/EID_674_Manager.ImportSystemConfiguration",
        ):
            return ctx

        if ctx.request.method != "post":
            return ctx

        l = ctx.headers["Location"]
        _, _, jobid = l.rpartition("/")
        ctx.received = json.dumps({"@odata.id": "", "@odata.type": "#x.x", "Id": jobid, "Name": ""})
        return ctx

TaskServer/{TaskId} … got issues as well - for tasks in progress it returns status_code 202, which is undefined by the spec, finished tasks (status_code == 200) just return the tasks data without json envelope, though there is no alternate content type defined in the spec.

The clinic mangles the Message …

class Task(aiopenapi3.plugin.Message):
    def received(self, ctx: "Message.Context") -> "Message.Context":
        if ctx.request.path != "/redfish/v1/TaskService/Tasks/{TaskId}":
            return ctx

        if ctx.request.method != "get":
            return ctx

        if ctx.status_code != "200":
            ctx.status_code = "200"
        else:
            ctx.received = json.dumps(
                {
                    "@odata.id": "",
                    "@odata.type": "#Task._.Task",
                    "Id": ctx.request.vars.parameters["TaskId"],
                    "Name": "",
                    "Messages": [{"MessageId": "", "Message": ctx.received.decode("utf-8")}],
                }
            )
            ctx.content_type = "application/json"
        return ctx

As aiopenapi3 creates the client dynamically, it would be possible to modify the description document to accept status_code 202 or alternating response content_types, but I prefer to limit the changes to description document to the minimum required.

e.g. …

class Document(aiopenapi3.plugin.Document):
    def __init__(self, url):
        self._url = url
        super().__init__()

    def parsed(self, ctx):
        if str(ctx.url) == self._url:
            # mangle the Task refs in the loaded openapi.yaml
            for k, v in ctx.document["paths"].items():
                for o, op in v.items():
                    for code, content in op["responses"].items():
                        if "content" not in content:
                            continue
                        try:
                            s = content["content"]["application/json"]["schema"]
                        except KeyError:
                            continue
                        if "$ref" in s and s["$ref"] == "/redfish/v1/Schemas/Task.v1_6_0.yaml#/components/schemas/Task":
                            s["$ref"] = "/redfish/v1/Schemas/Task.v1_6_0.yaml#/components/schemas/Task_v1_6_0_Task"

            """
            set the PathItems security to X-Auth OR basicAuth instead of X-Auth AND basicAuth
            """
            for k, v in ctx.document["paths"].items():
                for o, op in v.items():
                    if op["security"] == [{"basicAuth": [], "X-Auth": []}]:
                        op["security"] = [{"basicAuth": []}, {"X-Auth": []}]

That said my experience with the clinic are great and I propose to adapt this concept to gofish.
Allowing third parties to plugin doctors to the clinic will improve the compatibility with real world implementations, as discussed in #45 (comment).

As the comment picks up Oem as well … I allow detouring by path and @odata.type to redirect class creation for Oem (but not limited to).
First character indicates - # a @odata.type is used, "/" is a path.
I normalize all pathes in use to match the path from the description document.
e.g.
/redfish/v1/Managers/iDRAC.Embedded.1/Oem/Dell/Jobs/JID_959816896261
is
/redfish/v1/Managers/{ManagerId}/Oem/Dell/Jobs/{DellJobId}
with parameters ManagerId=iDRAC.Embedded.1, DellJobId=JID_959816896261

In the example - _v is the parsed value of the message as provided by aiopenapi3, ResourceRoot is the usability layer on top.

This example is paving the road to use DellAttributes …

First -
At /redfish/v1/Managers/iDRAC.Embedded.1

…
  Links:
…
    Oem:
      Dell:
        '@odata.type': '#DellOem.v1_3_0.DellOemLinks'
        DellAttributes:
        - '@odata.id': /redfish/v1/Managers/iDRAC.Embedded.1/Oem/Dell/DellAttributes/iDRAC.Embedded.1
        - '@odata.id': /redfish/v1/Managers/iDRAC.Embedded.1/Oem/Dell/DellAttributes/System.Embedded.1
        - '@odata.id': /redfish/v1/Managers/iDRAC.Embedded.1/Oem/Dell/DellAttributes/LifecycleController.Embedded.1
        [email protected]: 3
…

#DellOem.v1_3_0.DellOemLinks has to be converted to a Collection

@Detour("#DellOem..DellOemLinks")
class DellOemLinks(ResourceItem):
    @property
    def DellAttributes(self):
        cls = (
            self._root._client.api._documents[yarl.URL("/redfish/v1/Schemas/odata-v4.yaml")]
            .components.schemas["odata-v4_idRef"]
            .get_type()
        )
        data = [cls.model_validate(i) for i in self._v["DellAttributes"]]

        c = Collection[DellAttributes](client=self._root._client, data=data)
        return c

Give DellAttributes some helper to access the data:

Attributes:
  Users.1.UserName: nobody
  Users.1.Password: …
  Users.1.Privilege: …
…
  LocalSecurity.1.…: …
…

@Detour(
    "#DellAttributes.v1_0_0.DellAttributes",
    "/redfish/v1/Managers/{ManagerId}/Oem/Dell/DellAttributes/{DellAttributesId}",
)
class DellAttributes(ResourceRoot):
    class Permissions(enum.IntFlag):
        """
        Source: Chassis Management Controller Version 1.25 for Dell PowerEdge VRTX RACADM Command Line Reference Guide
        """

        LogintoiDRAC = 0x00000001
        ConfigureiDRAC = 0x00000002
        ConfigureUsers = 0x00000004
        ClearLogs = 0x00000008
        ExecuteServerControlCommands = 0x00000010
        AccessVirtualConsole = 0x00000020
        AccessVirtualMedia = 0x00000040
        TestAlerts = 0x00000080
        ExecuteDebugCommands = 0x00000100

    def list(self):
        r = collections.defaultdict(lambda: collections.defaultdict(dict))

        def compare(kv):
            cls, idx, attr = kv[0]
            return (cls, int(idx), attr)

        for (cls, idx, attr), value in sorted(
            map(lambda kv: (kv[0].split("."), kv[1]), self._v.Attributes.model_extra.items()), key=compare
        ):
            r[cls][idx][attr] = value
        return r

    def filter(self, jq_):
        return jq.compile(jq_).input(self.list())

can be used with:

da = await oem.Dell.DellAttributes.index('iDRAC.Embedded.1')
root = da.filter('.Users.[] | select(.UserName == "root")').first()
assert root["Privilege"] == DellAttributes.Permissions(511).value

That said, I really enjoyed looking at gofish, hope you can get some inspiration as well.

[stmcginnis]

Thanks for pointing this out! I'll have to take a closer look once I get caught up from the holidays.

Any idea if there is a Go equivalent for the python functionality?

[commonism]

I'm not aware of a OpenAPI client library using reflect to create the Schemas from the description document dynamically at runtime, but I'm not close with the go OpenAPI ecosystem.