Test concurrent feecurrency tx handling in tx pool

[piersy]

Adds a test to verify that the tx pool is executing concurrent fee currency checks safely.

I created this to see if I could reproduce #2318 but I was unable to reproduce that issue. It did however uncover a different race condition, which was that core.current was being set in one goroutine and accessed from others without synchronisation. This was easy to fix by just locking around the point where it is set.

[github-actions]

Coverage from tests in ./e2e_test/... for ./consensus/istanbul/... at commit 2dca825

coverage: 55.1% of statements across all listed packages

coverage:  68.4% of statements in consensus/istanbul
coverage:  63.3% of statements in consensus/istanbul/announce
coverage:  57.0% of statements in consensus/istanbul/backend
coverage:   0.0% of statements in consensus/istanbul/backend/backendtest
coverage:  24.3% of statements in consensus/istanbul/backend/internal/replica
coverage:  66.4% of statements in consensus/istanbul/core
coverage:  50.0% of statements in consensus/istanbul/db
coverage:   0.0% of statements in consensus/istanbul/proxy
coverage:  64.2% of statements in consensus/istanbul/uptime
coverage:  52.4% of statements in consensus/istanbul/validator
coverage:  79.2% of statements in consensus/istanbul/validator/random

[github-actions]

Test failures:
TestPriorityClient: geth Failed les_test.go:121: Initializing geth: [--networkid=42 init ./testdata/clique.json] test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.242] Maximum peer count ETH=175 LES=0 total=175 test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.244] Set global gas inflation rate rate=1.300 test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.244] Set global gas cap cap=25,000,000 test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.244] Allocated cache and file handles database=/tmp/geth-test1105644710/celo/chaindata cache=16.00MiB handles=16 test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.303] Writing custom genesis block test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.304] Persisted trie from memory database nodes=4 size=566.00B time=&quot;102.441µs&quot; gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=-82.00B test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.305] Successfully wrote genesis state database=chaindata hash=9195f2..122dcd test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.305] Allocated cache and file handles database=/tmp/geth-test1105644710/celo/lightchaindata cache=16.00MiB handles=16 test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.314] Writing custom genesis block test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.314] Persisted trie from memory database nodes=4 size=566.00B time=&quot;50.593µs&quot; gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=-82.00B test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.315] Successfully wrote genesis state database=lightchaindata hash=9195f2..122dcd test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.315] Allocated cache and file handles database=/tmp/geth-test1105644710/celo/lightestchaindata cache=16.00MiB handles=16 test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.347] Writing custom genesis block test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.348] Persisted trie from memory database nodes=4 size=566.00B time=&quot;37.433µs&quot; gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=-82.00B test_cmd.go:262: (stderr:31) INFO [11-26|15:31:52.348] Successfully wrote genesis state database=lightestchaindata hash=9195f2..122dcd les_test.go:130: Importing keys to geth test_cmd.go:262: (stderr:32) INFO [11-26|15:31:52.393] Maximum peer count ETH=175 LES=0 total=175 test_cmd.go:262: (stderr:32) INFO [11-26|15:31:52.394] Set global gas inflation rate rate=1.300 test_cmd.go:262: (stderr:32) INFO [11-26|15:31:52.394] Set global gas cap cap=25,000,000 les_test.go:99: Starting lightserver with rpc: [--networkid=42 --port=0 --ipcpath geth-1.ipc --allow-insecure-unlock --datadir /tmp/geth-test1105644710 --password ./testdata/password.txt --unlock 0x02f0d131f1f97aef08aec6e3291b957d9efe7105 --mine --miner.validator 0x02f0d131f1f97aef08aec6e3291b957d9efe7105 --tx-fee-recipient 0x02f0d131f1f97aef08aec6e3291b957d9efe7105 --light.serve=100 --light.maxpeers=1 --nodiscover --nat=extip:127.0.0.1 --verbosity=4] test_cmd.go:262: (stderr:33) INFO [11-26|15:31:52.668] Maximum peer count ETH=175 LES=1 total=176 test_cmd.go:262: (stderr:33) DEBUG[11-26|15:31:52.669] FS scan times list=&quot;264.054µs&quot; set=&quot;7.371µs&quot; diff=&quot;2.921µs&quot; test_cmd.go:262: (stderr:33) WARN [11-26|15:31:52.669] LES server cannot serve old transaction status and cannot connect below les/4 protocol version if transaction lookup index is limited test_cmd.go:262: (stderr:33) DEBUG[11-26|15:31:52.670] Sanitizing Go&apos;s GC trigger percent=100 test_cmd.go:262: (stderr:33) INFO [11-26|15:31:52.670] Set global gas inflation rate rate=1.300 test_cmd.go:262: (stderr:33) INFO [11-26|15:31:52.670] Set global gas cap cap=25,000,000 test_cmd.go:262: (stderr:33) INFO [11-26|15:31:52.670] Allocated trie memory caches clean=154.00MiB dirty=256.00MiB test_cmd.go:262: (stderr:33) INFO [11-26|15:31:52.670] Allocated cache and file handles database=/tmp/geth-test1105644710/celo/chaindata cache=512.00MiB handles=524,288 test_cmd.go:262: (stderr:33) DEBUG[11-26|15:31:54.269] Chain freezer table opened database=/tmp/geth-test1105644710/celo/chaindata/ancient table=bodies items=0 size=0.00B test_cmd.go:262: (stderr:33) DEBUG[11-26|15:31:54.492] Chain freezer table opened database=/tmp/geth-test1105644710/celo/chaindata/ancient table=receipts items=0 size=0.00B test_cmd.go:262: (stderr:33) DEBUG[11-26|15:31:54.740] Chain freezer table opened database=/tmp/geth-test1105644710/celo/chaindata/ancient table=diffs items=0 size=0.00B test_cmd.go:262: (stderr:33) DEBUG[11-26|15:31:55.120] Chain freezer table opened database=/tmp/geth-test1105644710/celo/chaindata/ancient table=headers items=0 size=0.00B test_cmd.go:262: (stderr:33) DEBUG[11-26|15:31:55.364] Chain freezer table opened database=/tmp/geth-test1105644710/celo/chaindata/ancient table=hashes items=0 size=0.00B test_cmd.go:262: (stderr:33) INFO [11-26|15:31:55.364] Opened ancient database database=/tmp/geth-test1105644710/celo/chaindata/ancient readonly=false test_cmd.go:262: (stderr:33) DEBUG[11-26|15:31:55.386] Current full block not old enough number=0 hash=9195f2..122dcd delay=90000 test_cmd.go:262: (stderr:33) INFO [11-26|15:31:55.387] Initialised chain configuration config=&quot;{ChainID: 15 Homestead: 0 DAO: &lt;nil&gt; DAOSupport: false EIP150: 0 EIP155: 0 EIP158: 0 Byzantium: 0 Constantinople: 0 Petersburg: 0 Istanbul: &lt;nil&gt; Churrito: &lt;nil&gt;, Donut: &lt;nil&gt;, Espresso: &lt;nil&gt;, Gingerbread: &lt;nil&gt;, Gingerbread P2: &lt;nil&gt;, HForkBlock: &lt;nil&gt;, Engine: istanbul}&quot; test_cmd.go:262: (stderr:33) DEBUG[11-26|15:31:55.387] Setting up Istanbul consensus engine les_test.go:115: lightserver rpc connect to /tmp/geth-test1105644710/geth-1.ipc: dial unix /tmp/geth-test1105644710/geth-1.ipc: connect: no such file or directory</code></pre></td></tr>
This test report was produced by the test-summary action.  Made with ❤️ in Cambridge.

[gastonponti]

LGTM
The only doubt I have, is that you are sending the txs from the same N accounts, and I thought that every txpool had a maximum amount of txs per account in a pending state, so I would have guess that it was going to fail due to that limitation, but it seems that works, so maybe was configured to not have a cap before?

[piersy]

@gastonponti Good point, that is strange there is a limit (see below), and the default which is used in the test is 64, so I'd really expect this test to at least be flaky if not fail all the time, but even running with a block time of 20 seconds, which means all txs will have been sent and in the tx pool before a block is constructed (300 txs per account) it still passes!

So I guess there is some understanding we are missing, anyway I've updated the tx sending to try and not overload the account queue.

celo-blockchain/core/tx_pool.go

Line 193 in ff274ca

AccountSlots uint64 // Number of executable transaction slots guaranteed per account

[palango]

@piersy Do we still want to merge this?