MQTTNet next major Release

[rido-min]

As discussed with @chkr1011 offline, we'd like to propose a new major release with the next goals:

• Update the .NET support platforms. Target .NET LTS versions (6, 8) and remove UWP and netstandard. Customers requiring support for the removed platforms can still use the 4.X version.
• Split Client and Server. This will help to evolve (and support) the client and server independently.
• Sign Packages and DLLs with a DNF certificate. MQTTnet.dll in nuget package not signed #1777

There might be additional changes in the API, even breaking changes (per SemVer rules wrt Major Releases):

• Swtich to MQTT5 as the default protocol version
• Revisit the MqttConnectingFailedException
• Remove obsolete APIs (such as MqttApplicationMessage.Payload)

Thanks,
Rido

[kskog]

What is the timeframe of .NET 8 support?

Why not add support for .NET 8 in a 4.x minor release? Is it a lot of work to validate compatibility ?

[rido-min]

.NET8 is already supported (via net7.0)

MQTTnet/Source/MQTTnet/MQTTnet.csproj

Line 13 in f82d382

<TargetFrameworks>netstandard1.3;netstandard2.0;netstandard2.1;netcoreapp3.1;net5.0;net6.0;net7.0</TargetFrameworks>

.NET 8 LTS dates are described here

This proposal, for a new major release, is to drop support for old targets, specifically uap10.0.

MQTTnet/Source/MQTTnet/MQTTnet.csproj

Line 15 in f82d382

<TargetFrameworks Condition=" '$(OS)' == 'Windows_NT' AND '$(MSBuildRuntimeType)' != 'Core' ">$(TargetFrameworks);uap10.0</TargetFrameworks>

There are multiple benefits, starting with the reduction of the supported platform matrix, and a simple build system, to avoid issues like #1770

[kskog]

Thanks for clearing that up.

[mregen]

I would also like to see support for large payloads using ReadOnlySequence buffers.
Please see discussion/prototype in #1917, #1918 for details.

Thanks, Martin

[chkr1011]

@rido-min Do you have any updates regarding the signing certificate?

[rido-min]

sorry, I thought the signing certiticate (and the signing infrastructure) were being tracked in dotnet-foundation/projects#337.

Let me know if you want to me to chime in.

[chkr1011]

I hoped that you got some insights because it is delayed since Dez 15 th. Is there something we can do to move it forward?

[chkr1011]

@rido-min Regarding splitting the code into several libraries I have some ideas I want to share with you. Please let me know what you think or if you have another proposal.

Strategy A

Library	Description
MQTTnet.Core	Contains the protocol level classes like packets, formatter, AsyncEvent etc.
MQTTnet.LowLevelClient	Contains only the low-level client implementation
MQTTnet.Client	Contains only the "normal" client implementation.
MQTTnet.Server	Contains the server implementation.
MQTTnet.Server.AspNet	Former "MQTTnet.AspNetCore" library. Contains the same features as right now.
MQTTnet.Extensions.Rpc	Same as right now.
MQTTnet.Extensions.ManagedClient	Same as right now. I would still call this an extension because users should build their own clients based on the "normal" client or low-level client. This is only a try to put in the "most wanted" behavior in a library.
MQTTnet.Extensions.WebSocket4Net	Same as right now.

Having so many libraries will give us more insights which components are used based on the download statistics from nuget.
Mostly all libraries will reference the library "MQTTnet.Core".

Strategy B

Library	Description
MQTTnet	Contains the protocol level classes like packets, formatter, AsyncEvent etc. and the client implementations "low-level" and "normal"
MQTTnet.Server	Contains the server implementation.
MQTTnet.Server.AspNet	Former "MQTTnet.AspNetCore" library. Contains the same features as right now.
MQTTnet.Extensions.Rpc	Same as right now.
MQTTnet.Extensions.ManagedClient	Same as right now. I would still call this an extension because users should build their own clients based on the "normal" client or low-level client. This is only a try to put in the "most wanted" behavior in a library.
MQTTnet.Extensions.WebSocket4Net	Same as right now.

Keeping the "MQTTnet" library will make sure that most users will simply upgrade to a new version and do not need to change the nuget package ID in all projects. It will also keep the nuget statistics.

[rido-min]

The main reason to Split client and server is to be able to release fixes for each.

I'd prefer StrategyB, since having multiple packages will introduce additional overhead to manage versioning, and also - as you mentioned - it will be easier to migrate existing projects.

[chkr1011]

@rido-min I finally have some time to start with the implementation. After the next release of version 4 (next few days) I will start with the new version.

Summary:

• Apply Strategy B
• Only support .NET 6.0 and .NET 8.0
• Remove WebSocket4Net extension (since it is no longer supported)
• Remove ManagedClient extension (since it has some conceptual issues, lots of bugs and it turns out that lots of people have special needs which are not covered in the extension or require lots of options. Maybe there will be a new version implemented from scratch in the future)
• Change to MQTTv5 by default

After this is done, we can test and discuss the next steps.

Do you have any information when the signing certificate is finally available? I would like to sign version 4 as well.

[rido-min]

just pinged the dotnet foundation folks, hope they will get back to us soon.

I'd love to collaborate with you in the next release, are you planning to work in a branch on this repo?

This might be a good opportunity to review some API updates that could introduce breaking changes.

Curious to know more about the ManagedClient conceptual issues.

[chkr1011]

I will create a new branch for the next version soon and will let you know.

Regarding the managed client: If you go throw the open issues you will find a lot of issues which are related to the managed client and not yet fixed. A lot of them often complain about a special situation like the need to set a user property while the client sends an ACK packet etc. This cannot be done because it requires exposing a new set of events for every small detail of the QoS flows. Also storing the not sent packages in order to properly support QoS 2 is hard to implement. I already startet a managed client V2 in a separate branch but never finished it. We may talk about this in detail if you want.

[chkr1011]

@rido-min the new branch is finally here. It is called "version5"...

[rido-min]

what's the best way to provide feedback? PRs on top of that branch? filing new issues or discussions?

[chkr1011]

I would prefer PRs on the top of that branch. I am afraid that new issues will be "lost" within the "regular" issues.

[chkr1011]

@rido-min Here a quick update:

• Client has now version 5.0.0 set as default
• Server is a dedicated project
• nugets are signed (but I have problems publishing them)
• Client will no longer throw an exception when it received a proper ACK from server
• Removed UWP dedicated code
• Removed .NET 4.5.2 and 4.6 dedicated code
• Removed all things which are marked as obsolete
• Removed WebSocket4Net extension completely
• Only .NET 6.0 (since it is LTS), .NET 7.0 and .NET 8.0 are set as target frameworks

From my notes the only thing which is missing is the rewrite of the payload type which was already done as a draft by @mregen .
If there is more please let me know.

[rido-min]

This looks great !! Thanks @chkr1011

Are you planning to keep the ManagedClient?

One proposal I have is to start separating the client and server tests. But that is another story.

nit about CleanSession: to be MQTT5 friendly I think we should rename to CleanStart.

If you need help publishing the signed packages, let me know, I can help offline.

[SeppPenner]

@chkr1011 Why Net7? It's not LTS and already deprecated as well since 14th of May 2024.

[chkr1011]

I am not sure 😄 If it is already out of support I will remove the target framework.
Please have a look at the branch and let me know if you have more ideas what can/needs to be changed.

[rido-min]

I think @SeppPenner is right, .NET7 is out of support since last week https://dotnet.microsoft.com/en-us/download/dotnet, note that .NET6 will go out of support in November 2024, only 6 months away.

How about targeting only 8? once the v5-preview is out, we can gather feedback from the community.

If there is not much demand for older frameworks we could remove a few #if NET7_0_OR_GREATER that are required for TLS.

[chkr1011]

OK then let's try it that way. I will remove .NET 7 stuff.

[mregen]

Hi @chkr1011, sorry I am just back from vacation.

From my notes the only thing which is missing is the rewrite of the payload type which was already done as a draft by @mregen .

are you planning to implement the payload type or do you need another PR on the v5 branch?

[mregen]

Hi @chkr1011, we have been using the last 5.0.0.1184-beta Nuget since a while and no regressions were found: I think it would be ready for an official release?

[chkr1011]

@mregen That sounds great to me. I will first prepare new branches required for version 4 support, including signing of version 4. As soon as this works, I would like to release one official RC first so that other users can also give feedback.

[mregen]

Hi @chkr1011 , I noticed a couple of new 5.0.0.0-RC builds on myget, is that already something that should be consumed and tested?
We are currently happy campers on the 1184-beta, which seems to be removed from myget...
Thanks, Martin

[chkr1011]

The rentention policy for myget automatically deletes older nugets. It can only keep around 50 packages in the free version.

[mregen]

ok, that explains it. I tested the RC and there are so far no issues.

[mregen]

Hi @chkr1011, the PR #2017 is now ready to gather feedback also with the receive code path.
There are more optimizations possible to eliminate a payload copy in the receive codepath, but I left this out for now. Also array pool allocation do not take advantage yet of the ReadOnlySequence, they are just single allocations from the pool. But you get the idea..
Please let me know if you have any questions or if you would like to discuss on Teams.
Thanks, Martin

[mregen]

Hi @chkr1011, the PR #2017 is now ready to gather feedback also with the receive code path. There are more optimizations possible to eliminate a payload copy in the receive codepath, but I left this out for now. Also array pool allocation do not take advantage yet of the ReadOnlySequence, they are just single allocations from the pool...

--> change of plan, I created a minimal version #2046 as the receive codepath becomes really complicated to improve without breaking changes, so focus is now to get at least the publish improvements in place and look at the rest later...

[chkr1011]

I migrated some of my projects to the new version and will let them run a couple of days/weeks. If there are no issues I will release the new version.

@SeppPenner I will also start updating the Wiki and delete Version 3 stuff etc.

[chkr1011]

So far nothing serious has happened. One thing I want to finish before the release is the support of Extended Authentication. The fix is almost completed. So, I expect that the next major version will be released this year.