Switch to JSON Object vs JSON Array based encoding of Attributes.

[DamianReeves]

I'll start off by saying I understand there is work on "decorators" happening, but I want to talk about an alternative.

Currently attributes are encoded in a very custom way.

Attributes for type nodes in morphir-ir.json are always encoded as Unit ({}), while attribute nodes for value nodes are encoded as a JSON array.

I feel like the reason we even need decorators is because of choosing this custom encoding.

I think users would be able to much more easily add their custom attributes to morphir-ir.json if attributes were always encoded as JSON objects.

In order to have a custom annotation we add a JSON Object into the attributes slot when encoding a node (i.e. value node).
That JSON object as all JSON Objects contains key value pairs.

The key would could one of the following (by this I mean we choose one of these not that we support all):

• A simple string i.e. "Morphir.Attributes.TypeAnnotation"
• An FQName encoded in a canonical string format, i.e.: "Morphir.SDK:Attributes:TypeAnnotation"
• A URL - This would allow us to move to attributes being similar to JSON-LD type of encoding

The most straightforward given what exists in the Morphir ecosystem today is a canonically encoded FQName, though a URL is extremely compelling.

The value portion would be encoded as a JSON Value (meaning it could be any of the standard JSON values).
From the Morphir standpoint Morphir could/would provide some standard things, such as Type Annotations which are currently encoded in the attributes slot of Morphir value nodes today.

This means the following:

Encoding Nulls

Nulls continue to encode as:

{}

Encoding Type Annotations

{
   "Morphir.SDK:Attributes:TypeAnnotation" : [
     "Reference",
     {},
     [[["com"],["example"]],[["my","module"]],["foo"]]
   ]
}

Encoding with Custom Attributes/Annotations

{
   "Morphir.SDK:Attributes:TypeAnnotation" : [
     "Reference",
     {},
     [[["morphir"],["sdk"]],[["basics"]],["int"]]
   ],
  "Org.Schemas:Naming:Name":"John Smith",
  "Morphir.SDK:Numeric:Scale": 5,
  "Morphir.SDK:Numeric:Precision": 15,
  "Com.FooCorp:TypeSystem:JvmType":[
     "Reference",
     {},
     [[["com"],["foo", "corp"]],[["basics"]],["amount"]]
   ],
}

Benefits of the above is that:

• It removes the added complexity of needing to use a sidecar file (work could continue there if desired, but needing to use sidecar files seems a bit intrusive IMHO).
• A more consistent encoding of attributes
• We can start to perhaps looking at Attributes in Morphir as maps from a FQName to a (JSON) Value

Downsides:

• This would require a change of the format (version 3).

I'm sure there are other upsides and downsides.

Thoughts?

[AttilaMihaly]

Thanks for the detailed proposal. I think the "decorations" feature that we are actively working on will achieve this with some tweaks. Before I get into that though let me clear up a misunderstanding:

Currently attributes are encoded in a very custom way.

Everything in Morphir including the IR, the attributes on the IR, test cases and decorations (in preview for now) use the same JSON serialization standard documented here: https://github.com/finos/morphir-elm/blob/main/docs/developers-guide/files/json-schema-mappings.md

The standard happens to map Unit to {} and custom types (tagged unions) as arrays. Type expressions which are the only attributes used in the IR right now are tagged unions. So in the JSON it might look like objects vs arrays, but it's really just about no attribute vs type as attribute.

In regards to your proposal, I think we are aiming to achieve the same thing with slightly different data structures. What you are proposing seems to look like this (I'll assume Attribute and Decoration means the same thing and will use Attribute to make it easier to follow):

type alias StorageFormat =
    IR (Dict AttributeID AttributeShape)

What we are proposing with Decorations is this:

type alias StorageFormat =
    ( IR, Dict AttributeID (Dict NodeID AttributeShape) )

As you can see, they are not all that different, it's just organized differently. What's common is that:

• Every Attribute/Decoration has a globally unique ID
• Every Attribute/Decoration has a well-defined shape (in Decorations we use a Morphir type to describe the shape with the standard JSON serialization)

There are also a few differences between Attributes and Decorations:

• Attributes cannot be added seamlessly after an IR has been created. We already have use-cases where the team adding decorations won't be the one that owns the business model.
• Attributes can currently only be added to type and value expression nodes in the IR, not modules or packages. Decorations support those too.
• Attributes are mandatory on every node by default. Decorations are optional by default (since they are in a dictionary).
• Attributes will need to be processed by every tool that reads/writes the IR. Decorations can be managed independently.

There's probably other things to consider but I think it will be easier to discuss all this with the Decoration spec in hand. We should have an initial version of that available by next week.