Consumer ack floor sequence exceeds current stream sequence [v2.10.20]

[funkye]

Observed behavior

At the moment we are observing a situation that the consumer ack floor sequence is much higher than the stream sequence. As a result, the consumer does not read new messages.

Consumer info:

Stream info:

Expected behavior

Consumer ack floor sequence does not exceed stream sequence and new messages are read by the consumer.

Server and client version

NATS server version: 2.10.20, not a cluster
.NET client version: 2.2.3

Host environment

OS: Ubuntu 22.04.1

Steps to reproduce

No stable reproduce. Observed on WorkQueue type of retention.

[saurabhojha]

We are also facing the same issue for a workqueue retention type of stream. So far we have not been able to reproduce this consistently.
Server: 2.10.11

[neilalexander]

We've got a number of fixes lined up for 2.10.23 which should help significantly with this, would you be willing to try out 2.10.23-RC.7?

[wallyqs]

I think this would also have been solved by https://github.com/nats-io/nats-server/pull/6109/files on the v2.10.23 RC.2