A notable variety of CacheKVStore improvements

[ValarDragon]

Description

closes: #XXXX

---

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

• Targeted PR against correct branch (see CONTRIBUTING.md)
• Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
• Code follows the module structure standards.
• Wrote unit and integration tests
• Updated relevant documentation (docs/) or specification (x/<module>/spec/)
• Added relevant godoc comments.
• Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
• Re-reviewed Files changed in the Github PR explorer
• Review Codecov Report in the comment section below once CI passes

[ValarDragon]

Merging this. I know it fails the race test, but I've looked into this in depth with others on the team. This isn't really a race condition. This seems to more accurately be an issue with the gorace checks being too time-sensitive in its detection of read/writes from similar processes indicating a race.

This is caused by switching the CacheKVIterator to NoMtx variants. However there is no race condition, as shown by the following:

• A race condition with no mutexes would imply a deadlock when the mutexes are there, which is not present.
• If you print out when mutexes are locked / unlocked, the race detector stops detecting it, and you see very clearly that the unlocks are always interleaved to be before the next lock. (Even applies if you comment out the actual mutex lines and just replace with printlns)
• We tried running the tests 1000 times -count=1000, and they still passed.
• Whats happening is that the iterator is in a second process, that blocks on the primary process starting it. However the primary process also does sets on the CacheKVStore. This second process will get removed once memdb: Make iterators not channel based tendermint/tm-db#188 is fixed