Cumulocity IoT operations created whilst the bridge is down are not processed upon reconnection

[reubenmiller]

Describe the bug

Cumulocity IoT operations which are created whilst the Cumulocity IoT MQTT bridge is disconnected do not get processed by the tedge-mapper-c8y when the network outage has passed (e.g. the bridge can communicate with Cumulocity IoT again).

The operations are left in the PENDING state until the mapper is either restarted, or a manual SmartREST 2.0 500 static SmartREST 2.0 message is published (once the mapper health status returns).

Workaround

The following MQTT message can be published locally to request the PENDING operations from Cumulocity IoT once the cloud connectivity has been restored:

tedge mqtt pub c8y/s/us 500

To Reproduce

1. Install and configuration thin-edge.io and verify the bridge connection to Cumulocity IoT is up/healthy

2. Disconnect the network

3. Wait until the bridge is down

4. Create a new operation in Cumulocity IoT. For example request a the tedge-configuration-plugin

5. Connect the network

6. Wait until the bridge has been re-established

7. Expectation: The pending operation should be processed by thin-edge.io, and set to SUCCESSFUL

Expected behavior

Any operations which are created whilst the Cumulocity IoT bridge is down, should be processed one the connectivity has been reestablished.

The way this should be done, is that the static SmartREST 2.0 template, 500 should be sent when the connection has been restored.

Screenshots

Environment (please complete the following information):

• OS [incl. version]: any
• Hardware [incl. revision]: any
• System-Architecture [e.g. result of "uname -a"]: any
• thin-edge.io version [e.g. 0.1.0]: 1.3.1

Additional context

[reubenmiller]

A system test case has been added to cover the bug found here. #3170

[reubenmiller]

I've tested this fix (included part of #3278), and it works as intended, but there is a following up issue where the devicecontrol/operations message payload can included multiple operations (newline delimited), which causes the parsing to fail. So technically this ticket can be closed, and the limitation on parsing the multiple operations in one message from the cloud is tracked in this ticket: #3297