Remove or expand buffer limits on RequestManager and ResponseManager queues

[hannahhoward]

What

The basic architecture of the request and response manager is as follows: all public methods put messages into a buffered channel that is processed sequentially by an internal thread. (see https://github.com/ipfs/go-graphsync/blob/main/docs/architecture.md#actor-pattern-in-requestmanager-and-responsemanager for explanation)

We've now seen multiple deadlocks from these channels reaching their buffer limits. Moreover, the problem is usually that actual message processing of the internal thread indirectly gets stuck sending more messages to the queue.

I think we should remove limits on these queues for now. We can use use well tread architectures for a single consumer thread-safe queues using sync.Mutex and sync.Cond.Wait -- see https://www.justsoftwaresolutions.co.uk/threading/implementing-a-thread-safe-queue-using-condition-variables.html, https://dev.to/narasimha1997/crafting-a-concurrent-queue-in-golang-2n, and the notifications system in this repo: https://github.com/ipfs/go-graphsync/blob/main/notifications/publisher.go#L214-L235

If we need to set an upper limit, it's not hard to add.

[rvagg]

we move from a global queue to per request message queues, with a single go-routine per request processing messages

To be clear on this - you're suggesting a single go routine for a request (same as it is now), but a queue per message type (the structs in requestmanager/messages.go & responsemanager/messages.go)? Is this going to be a problem for prioritisation of messages or are we going to need to resort to timestamps or some kind of monotonic counter to determine what to do next? This is normally where the multiple-producer + single-consumer pattern becomes a problem and you need to resort to various prioritisation strategies to pick what should be done next.

[hannahhoward]

@rvagg wow that got added to this issue accidentally -- more clearly explained as a seperate more longterm approach in #251