-
Notifications
You must be signed in to change notification settings - Fork 61
Feedback endpoint for CDS Hooks 1.1
A CDS Hooks service is able to return three types of guidance to a user: info cards, suggestion cards and app link cards.
Once a CDS Hooks service responds to a hook by returning an info or suggestion card, the service has no further interaction with the CDS client. The acceptance or rejection of a suggestion is valuable information to enable a service to improve its behavior towards the goal of the end-user having a positive and meaningful experience with the CDS. A feedback endpoint enables suggestion tracking & analytics.
There are four distinct, possible outcomes for an end user's single interaction with a CDS Hooks suggestion card:
- Suggestion accepted. The end user viewed and acted on the guidance.
- Card ignored. The end user did not interact with the card.
- Card dismissed without reason. The end user viewed the card and dismissed it without providing a reason why.
- Card overridden with reason. The end user rejected the guidance and helpfully explained why by selecting an override.
Note that while a single suggestion can be accepted, an entire card is ignored, dismissed or overridden. To uniquely identify a card, this proposal adds an optional (unless the service wants feedback) card.uuid, which mirrors the existing hookInstance and suggestion.uuid.
Typically, an end user may only accept, dismiss or override a card once; however, a card once ignored could later be acted upon. CDS Hooks does not specify the UI behavior of CDS clients, including the persistence of cards. CDS clients should faithfully report each of these distinct end-user interactions as feedback.
The CDS client can inform the service when one or more suggestions were accepted by POSTing a simple json object.
Upon the user accepting a suggestion (perhaps when she clicks a displayed label (e.g., button) from a "suggestion" card), the CDS client informs the service by posting the card and suggestion uuids to the CDS Service's feedback endpoint with an outcome of accepted.
To enable a positive clinical experience, the analytics endpoint may be called for multiple hook instances or multiple cards at the same time or even multiple times for a card or suggestion. Depending upon the UI and workflow of the CDS client, a CDS Service may receive feedback for the same card instance multiple times.
POST {baseUrl}/cds-services/{serviceId}/feedback
{
"feedback":[
{
"card":"`card.uuid` from CDS Hooks response",
"outcome":"accepted",
"acceptedSuggestions":["`card.suggestion.uuid` from CDS Hooks response"],
"outcomeTimestamp": "iso timestamp in UTC when action was taken on card"
}
]
}
If either the card or the suggestion has no uuid, the CDS client does not send a notification.
The above HTTP post to a service endpoint for the suggestion informs the service when its suggestion(s) were accepted, but what happens when the user doesn't take the suggestion? A card can be ignored, dismissed or overridden.
If the end-user doesn't interact with the CDS Service's card at all, the card is ignored. Why would this happen? Perhaps the priority indicator of the card deprioritized it, or the user simply ignored the guidance. In this case, the CDS Client does not inform the CDS Service of the rejected guidance. Even with a card.uuid, a `suggestion.uuid' and an available feedback service, the service is not informed.
A CDS client may enable the end user to dismiss guidance without providing an explicit reason for doing so. The CDS client can inform the service when a suggestion was dismissed by specifying an outcome of dismissedWithoutReason.
POST {baseUrl}/cds-services/{serviceId}/feedback
{
"feedback":[
{
"card":"f6b95768-b1c8-40dc-8385-bf3504b82ffb", // uuid from `card.uuid`
"outcome":"dismissedWithoutReason",
"outcomeTimestamp": "iso timestamp in UTC when action was taken on card"
}
]
}
A CDS service may recommend a list of reasons, to be presented to the end user, to explain why she rejected the service's guidance. Each card may be accompanied by a service-provided array of override labels and identifiers, like so:
{
"cards":[
{
"summary":"Update med.",
"indicator":"warning",
"selectionBehavior":"at-most-one",
"suggestions":[
],
"overrides":[
{
"key":"reason-id-provided-by-service",
"label":"Patient refused"
},
{
"key":"cntrndctd",
"label":"Contraindicated"
}
]
}
]
}
The CDS client may extend this list.
TODO
The CDS client can inform the service when a suggestion was rejected by POSTing an outcome of overriddenWithReason to the service's feedback endpoint -
POST {baseUrl}/cds-services/{serviceId}/feedback
{
"feedback":[
{
"card":"9368d37b-283f-44a0-93ea-547cebab93ed",
"outcome":"overriddenWithReason",
"overrideReason":"reason-id-provided-by-service",
"outcomeTimestamp": "iso timestamp in UTC when action was taken on card"
}
]
}
POST {baseUrl}/cds-services/{serviceId}/feedback
{
"feedback":[
{
"card":"card uuid",
"outcome":"dismissedWithoutReason| accepted | overriddenWithReason",
"overrideReason":"reason provided by service in CDS Hooks response",
"outcomeTimestamp": "iso timestamp in UTC when action was taken on card",
"acceptedSuggestions":[
"suggestion uuid 1"
] /**MAY contain multiple uuids,
based on value of `card.selectionBehavior`**/
},
{
"card":"f6b95768-b1c8-40dc-8385-bf3504b82ffb",
"outcome":"dismissedWithoutReason",
"outcomeTimestamp": "iso timestamp in UTC when action was taken on card"
},
{
"card":"9368d37b-283f-44a0-93ea-547cebab93ed",
"outcome":"overriddenWithoutReason",
"overrideReason":"reason-id-provided-by-service",
"outcomeTimestamp": "iso timestamp in UTC when action was taken on card"
},
{
"card":"b2cdcf50-62c8-4be3-b574-6cc97c0a5617",
"outcome":"accepted",
"acceptedSuggestions":[
"c6a9fc61-5621-413f-8879-8679a830bddb"
],
"outcomeTimestamp": "iso timestamp in UTC when action was taken on card"
}
]
}
- Should this HTTP Post identify the user who took the action, or is the assumption that it was the same user who triggered the hook acceptable?
A: We should assume that the user who triggered the card (and therefore received the guidance) is the user who took action on the card. This is by far the 80%+ use-case. If/when we tackle asynchronous cards, we can always add a userId to the feedback message.
- Should the CDS client attempt to share urls to any FHIR resources created as part of the suggestion being accepted? That would significantly increase the effort to implement, would enable much deeper analysis capabilities by the service, but would be less valuable than widespread implementation of the simple suggestion accepted notification.
A: Potentially really neat functionality, but adds significant scope. Not for now.
- How should overrideReasons interact with a potential ability for the user to specify their own reason?