[feature] Max Retry per msg feature added

[ngoyal16]

(If this PR fixes a github issue, please add Fixes #<xyz>.)

Fixes #890

Motivation

Add a feature to allow Max Retry per msg.

Modifications

Msg will have max reconsume times property. if not present use default property from consumer max deliveries

Verifying this change

• Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

• Added Integration tests for per msg retry login

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

• Dependencies (does it add or upgrade a dependency): no
• The public API: (yes / no)
• The schema: (yes / no / don't know)
• The default values of configurations: no
• The wire protocol: (yes / no)

Documentation

• Does this pull request introduce a new feature? yes
• If yes, how is the feature documented? (not applicable / docs / GoDocs / not documented)
• If a feature is not applicable for documentation, explain why?
• If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

[nodece]

It looks like you want to use the Properties to override the MaxDeliveries of dlq.

[ngoyal16]

It looks like you want to use the Properties to override the MaxDeliveries of dlq.

This is the simplest and best way I found for it.. because retry and dlq is managed at client side in all libs so we can use the same...

[nodece]

It looks like you want to use the Properties to override the MaxDeliveries of dlq.

This is the simplest and best way I found for it.. because retry and dlq is managed at client side in all libs so we can use the same...

I see. I suggest you make a discussion to mailing lists, see https://pulsar.apache.org/community/#section-discussions.

[wolfstudy]

Thanks @ngoyal16 work for this. Sorry for request change.

Currently, we prefer users to use the nack backoff Policy to customize their own retry strategy. Currently NackBackoff is an interface that provides a default backoff retry strategy. If the strategy does not meet the needs, we can try to implement the NackBackoff interface to meet the current needs.

The current implementation of ReconsumeLater relies on the function of delaying messages, and the Go SDK still does not support the backoff retry function. Of course, it is not planned to continue to support it in the future. The following problems are found in actual production use:

1. Too many delayed messages will increase the load on Broker
2. NackBackoff can completely replace the usage logic of ReconsumeLater

If there is something unsatisfactory, we prefer to support more customized features in NackBackoff

[nodece]

Close this PR, this idea has not been approved by PMC.

[jcass8695]

@nodece what is PMC, and why was it not approved? I created an issue today (#1167) which I think this PR would have addressed.

[RobertIndie]

@nodece what is PMC, and why was it not approved? I created an issue today (#1167) which I think this PR would have addressed.

The discussion happened here: https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
For the term of PMC, please see: https://pulsar.apache.org/community/#section-governance