Function notifies losing LNS with a direct method

[spygi]

PR for issue #1497

What is being addressed

When the winning LNS changes on the function side, we have to inform the losing LNS proactively that it lost the race. The reason why we do this is in order for its Edge Hub to not retry to create the connection which would result in a connection ping-pong.

How is this addressed

• A new direct method is called from the DeduplicationExecutionItem when a connection switch happens.
• When LNS receives it (in ModuleConnectionHost), it sets itself as the losing LNS and closes the connection.
• Unit tests added for LNS (ModuleConnectionHostTest) and the function side (MessageDeduplicationTests), integration tests for Function (DeduplicationTestWithRedis) and a simulation test (SimulatedLoadTest) for the whole flow.

Verification

Simulation test in the CI

[codecov-commenter]

Codecov Report

Merging #1577 (3bb6604) into dev (5a86399) will increase coverage by 0.07%.
The diff coverage is 95.45%.

@@            Coverage Diff             @@
##              dev    #1577      +/-   ##
==========================================
+ Coverage   86.46%   86.54%   +0.07%     
==========================================
  Files         235      235              
  Lines        9201     9258      +57     
==========================================
+ Hits         7956     8012      +56     
- Misses       1245     1246       +1     

Impacted Files	Coverage Δ
LoRaEngine/LoraKeysManagerFacade/FacadeStartup.cs	0.00% <0.00%> (ø)
...ade/FunctionsBundler/DeduplicationExecutionItem.cs	93.93% <96.55%> (+1.63%)	⬆️
LoRaEngine/LoraKeysManagerFacade/HttpUtilities.cs	80.00% <100.00%> (+2.22%)	⬆️
...ndCloudToDeviceMessage/SendCloudToDeviceMessage.cs	81.37% <100.00%> (-0.19%)	⬇️
...csStation/ModuleConnection/ModuleConnectionHost.cs	94.30% <100.00%> (+1.60%)	⬆️
...kSrvModule/LoRaWan.NetworkServer/MetricRegistry.cs	100.00% <100.00%> (ø)
...oRaWan.NetworkServer/ManagedConnectionException.cs	0.00% <0.00%> (-25.00%)	⬇️
...kPreferredGateway/PreferredGatewayExecutionItem.cs	80.70% <0.00%> (-3.51%)	⬇️
...LoraKeysManagerFacade/LoRaDeviceCacheRedisStore.cs	82.00% <0.00%> (+10.00%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a86399...3bb6604. Read the comment docs.

[p-schuler]

The logging is too verbose and I think we should add a metric tracking the ownership changes. That would allow an operator to track aggressive connection changes and alert on them.

Generally: When Mark strategy is configured, I think we could end up in a situation, where every message (worst case) does switch the connection. We could end up in a scenario, where we have a lot of direct methods going to the edge to request the closing of the connection. I'm not sure this is a concern really, as we make it clear that Mark should not be used in general, but on only for specific devices.