What is the msg_store_persistent directory used for?

[icolombi]

Describe the bug

Hello,

today I noticed that after about 15 days the upgrade from 3.13.x to 4.0.5 of single nodes, the directory:

/var/lib/rabbitmq/mnesia/rabbit\@cu-rmq-17/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent/

is full of files .rdg, the oldest has the date of the upgrade.

For example:

-rw-r----- 1 rabbitmq rabbitmq 526829 Feb 10 18:47 16850229.rdq

RabbitMQ stopped to work since there was no more space left on /var/lib/rabbitmq/.

I manually deleted *.rdq files older than 1 day, restarted RabbitMQ and now it seems ok but again it seems these files are accumulating.

All the queues in these nodes are configured as classic v2, and are reported as empty from the UI.

My clusters instes, configured as quorum queues, seems to not suffer from this behaviour.

No other changes other than the upgrade of RabbitMQ have been made to these nodes.

Any idea why these files are accumulating? Thanks

Reproduction steps

1.Upgrade from 3.13.x to .4.0.5 of single nodes, files are accumilating in msg_store_persistent
2.
3.
...

Expected behavior

Not consuming disk space in msg_store_persistent

Additional context

No response

[michaelklishin]

@icolombi please never against use issues for what is clearly a question.

RabbitMQ 3.13.x is out of community support.

msg_store_persistent is a data directory where a durable classic queue message store (messages, indices) for a specific virtual host resides. Perhaps you have some classic queues with a backlog in that virtual host, or had
them and the queue were deleted without a chance for them to delete their data.

[michaelklishin]

Under the directory in question, there will be a .vhost file with the name of the virtual host,
so

cat /var/lib/rabbitmq/mnesia/rabbit\@cu-rmq-17/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent/.vhost

will tell you what virtual host has an accumulated backlog.

[michaelklishin]

This feels like total RabbitMQ 101:

• rabbitmq-diagnostics list_queues will list queues in a specific virtual host
• rabbitmq-diagnostics report will list all queues, streams, exchanges, bindings, users, and many others things
• rabbitmqadmin list queues will list queues in a specific virtual host using the HTTP API (in case you do not have access to rabbitmq-diagnostics)