SubmitBlock Checking for Backup Daemons

[HashUnlimited]

We have found some very strange behaviour after the stratum running for a few hours.

System setup:

• several instances of stratum, everyone connected to one separate instance of coind, process name and mempool key prefix unique in the config
• all coind instances are serving the same chain, all supporting submitblock

Error occurs after a few hours of running, all instances, once there it will appear on every block found:

2017-07-08 18:55:26,264 ERROR bitcoin_rpc # getblocktemplate submit failed. Problem Submitting block 500 Internal Server Error
Traceback (most recent call last):
File "/root/EMC2-Stratum-10k/lib/bitcoin_rpc.py", line 91, in submitblock
resp = (yield self._call('getblocktemplate', [{'mode': 'submit', 'data': block_hex}]))
Error: 500 Internal Server Error
2017-07-08 18:55:26,264 ERROR bitcoin_rpc # Try Enabling TX Messages in config.py!
Traceback (most recent call last):
File "/root/EMC2-Stratum-10k/lib/bitcoin_rpc.py", line 91, in submitblock
resp = (yield self._call('getblocktemplate', [{'mode': 'submit', 'data': block_hex}]))
Error: 500 Internal Server Error

So far so good, what is it doing in line 91? We should never use getblocktemplate here...

2017-07-08 17:25:21,117 DEBUG bitcoin_rpc # Got to Bitcoin RPC
2017-07-08 17:25:21,117 DEBUG bitcoin_rpc # Got to Bitcoin RPC
2017-07-08 17:25:21,118 INFO mining # Connecting to litecoind...
2017-07-08 17:25:21,118 INFO bitcoin_rpc # Checking for submitblock
2017-07-08 17:25:21,135 DEBUG bitcoin_rpc # submitblock detected.
2017-07-08 17:25:21,135 INFO mining # Found submitblock

The first blocks found are submitted as expected:

2017-07-08 17:36:01,469 DEBUG bitcoin_rpc # Submitting Block with submitblock: attempt #1

Then, suddenly it will change and there for throw the error:

2017-07-08 18:42:07,808 DEBUG bitcoin_rpc # Submitting Block with getblocktemplate submit: attempt #1

actually there is no reason to change the method during a running session...

[HashUnlimited]

UPDATE:

It appears that the stratum is switching between main and backup coind based on their current block height. For some reason the BACKUP node is not checked for submitblock, there for using getblocktemplate.

2017-07-08 18:14:42,015 DEBUG bitcoin_rpc_manager # Check Height -- Current Pool 0 : 1492806
2017-07-08 18:14:42,058 DEBUG bitcoin_rpc_manager # Check Height -- Pool 1 : 1492807
2017-07-08 18:15:36,096 DEBUG bitcoin_rpc_manager # Check Height -- Current Pool 1 : 1492807
2017-07-08 18:15:36,152 DEBUG bitcoin_rpc_manager # Check Height -- Pool 0 : 1492807