Max Message Size in RabbitMQ .NET client 6.4.0 up to 7.0.0

[lukebakken]

Discussed in #1379

^{Originally posted by dank100 August 30, 2023}
From 6.4.0 and forward There is now a limit on the message size, as you rightfully explain in your changelogs:

"This release adds the ability to specify a maximum message size when receiving data. The default values are:

RabbitMQ .NET client 7.0.0 and beyond: 128MiB

RabbitMQ .NET client 6.4.0 up to 7.0.0: no limit by default"

However, when I specify the cf.MaxMessageSize = 536870912 as in your example. I'm still limited by the max size of the message, i.e. 134217728.

If I downgrade to 6.3.1, I'm not limited by the MessageSize, on the very same cluster and exchange.

I'm not sure this is any bug, but I cannot seem to figure it out.

An observation I've made in ConnectionFactory.cs is that on your main branch we have.

public const uint DefaultMaxMessageSize = 134217728;

However, on, tag 6.5.0 we have.

public const uint DefaultMaxMessageSize = 0;

In your docs you specify that 0 means "unlimited".

With me using 6.5.0, I would expect the MaxMessageSize being 0, whether I specify it or not. However, as explained I'm limited by the MaxMessageSize of 134217728.

Is this anything you've heard of before, and are you able to pinpoint me in the right direction?

• Dan

[lukebakken]

I think the intent is to disallow messages that are larger than a specific upper limit.

@Gsantomaggio @michaelklishin - should there be a way to disable the maximum message size limit?

I just merged #1566, and realized that, as it stands, there is no way for the user to specify a max message size greater than 128MiB. Frankly, I consider this a feature but others may think otherwise.

[michaelklishin]

@lukebakken in the Java client and RabbitMQ itself, you can bump the limit but not disable it entirely. Furthermore, in RabbitMQ itself there is a hard limit of 512 MiB which I find reasonable. Even 200-300 MiB large messages really belong to a blob store.

I think it makes sense for this client to simply follow suit.

[michaelklishin]

In addition, "unlimited message size" is not something that RabbitMQ supports, IIRC there is a hard cap at 512 MiB (it was last updated in January 2019 in https://github.com/rabbitmq/rabbitmq-common/pull/289/files).

Allowing "unlimited message size" also would go against the general direction of RabbitMQ server, see rabbitmq/rabbitmq-server#11187.

[Gsantomaggio]

agree with @michaelklishin

[amiradeli]

I am using RabbitMQ via MassTransit.
When I set MaxMessageSize, MassTransit send it to RabbitMQ client in GetConnectionFactory:

public static ConnectionFactory GetConnectionFactory(this RabbitMqHostSettings settings)
{
    ...
    if (settings.MaxMessageSize.HasValue)
        factory.MaxInboundMessageBodySize = settings.MaxMessageSize.Value;
   ...
   return factory;
}

Let's say MaxMessageSize is set to 1,800,000 bytes.

3 different scenarios happen here:

1. When I send a message between 1,800,000 and 134,217,728 (default 128MB), it does not throw any exception in either side (sender and receiver) and logs say the message is sent but receiver does not receive the message and then timeout happens. Then, next time, when I send a new message, it shows this error in the logs of sender:

RabbitMQ.Client.Exceptions.AlreadyClosedException | Already closed: The AMQP operation was interrupted: AMQP close-reason, initiated by Library, code=501, text='Frame body size '1959266' exceeds maximum of '1800000' bytes', classId=0, methodId=0, exception=RabbitMQ.Client.Exceptions.MalformedFrameException: Frame body size '1959266' exceeds maximum of '1800000' bytes

1. When I send a message between 134,217,728 (default 128MB) and a threshold which I could not find, then it seems it disregards 1,800,000 and the log in sender side shows this error:

Concurrent Peak: 1, 406-PRECONDITION_FAILED - message size 134473138 is larger than configured max size 134217728

and consumer restarts and again leads to timeout.

1. When I send a message which is bigger than the threshold I mentioned previously, I receive another error in sender logs regarding serialization:

Failed to serialize message | System.ArgumentException | The JSON value of length 257220414 is too large and not supported.

So, it seems the behavior is inconsistent and in none of the cases it works as expected.

[michaelklishin]

@amiradeli we do use specific, well defined issues for discussions or troubleshooting. This belongs to a discussion (that you can leave a link to here).

Client-library enforced maximums should result in an immediate exception on the publisher side.

When a message is larger than the configured limit on the RabbitMQ node the client is connected to, the standard asynchronous channel error notification mechanism is used, and it is not immediate even if it takes a tiny fraction of a second.

The goal of this issue is to introduce a client-side limit which will be verified before publishing, so, like in the first scenario.