User metadata

[mfateev]

Is your feature request related to a problem? Please describe.
There is a need to attach additional information to commands for troubleshooting and diagraming purposes. For example thread-id of a thread that invoked an activity or scheduled a timer.

Describe the solution you'd like
The proposal is to add a metadata field to the Command protobuf. Then SDKs would be able to add use case specific information to commands.

[williambanfield]

This would be pretty helpful for us. At the moment, I am planning on creating a small activity called 'annotate' that takes a list of strings so that the list of strings will be stored and output alongside workflow information.

[cretz]

We have since added a SDK metadata command at temporalio/api#259 but it is task level meant for SDK internal use.

I am planning on creating a small activity called 'annotate'

Note, you'll probably want to make this a local activity that is a single-event marker instead of a full activity. But agreed, we should support arbitrary metadata on commands (and maybe an arbitrary log marker).

[Quinn-With-Two-Ns]

Another user ask that would be solved if we could attach metadata to commands

[williambanfield]

@cretz

single-event marker instead of a full activity

How does one go about doing this?

[cretz]

Use a local activity. It's automatically an inline single-command/event thing as opposed to activities which is multi-event and may execute on a remote worker.

[cretz]

To be more specific here, I think we take an approach similar to temporalio/api#259 but for commands/events. We should add temporal.api.sdk.v1.WorkflowEventMetadata to both temporal.api.command.v1.Command and temporal.api.history.v1.HistoryEvent (or just to specific attributes we know can handle them). And server should put the metadata on the events created from the commands. WorkflowEventMetadata can have common, well-typed fields like string friendly_name or in generic cases in cahoots with UI, there can be a map<string, string> ui_info or similar.

[cretz]

After internal discussion, it was decided that the approach here is an event metadata that, for now, is a map of enumerate keys and payload values. The enumerate keys are well-known metadata keys that UI can leverage, and the payload values are encryptable. I will link an API PR when available. Arbitrary user-defined keys is not going to be available at this time, but the well-known keys will have lots of opportunity for users to provide extra event information.

[cretz]

This has been changed to "user metadata" and after internal discussion, the initial API PR for public discussion is available at temporalio/api#371.

[cretz]

I am closing this in favor of the clearer #486.