client: fix pd client metrics registration

[Tema]

What problem does this PR solve?

Issue Number: ref #8678

What is changed and how does it work?

@JmPotato if there is a code referencing metrics from the global context like in tikv/client-go#1543 (comment) then they could consume metrics created in the metrics.init() method. However, when latter the global metrics are recreated, reinitialized and registered with metrics.InitAndRegisterMetrics() the metrics consumed before are abandoned and not reported to prometheus. This PR allows to register metrics consumers, so that if they are consumed before metrics.InitAndRegisterMetrics() in invoked, they are reinitilized and registered later.

cc: @rleungx

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Code changes

• Has the configuration change
• Has HTTP API interfaces changed (Don't forget to add the declarative for the new API)
• Has persistent data change

Side effects

• Possible performance regression
• Increased code complexity
• Breaking backward compatibility

Related changes

• PR to update pingcap/docs/pingcap/docs-cn:
• PR to update pingcap/tiup:
• Need to cherry-pick to the release branch

Release note

None.

[ti-chi-bot]

Hi @Tema. Thanks for your PR.

I'm waiting for a tikv member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ti-chi-bot]

Now you can start all CI jobs with /test all in comment or query the triggers with /test ?

[codecov]

Codecov Report

Attention: Patch coverage is 84.21053% with 3 lines in your changes missing coverage. Please review.

Project coverage is 76.28%. Comparing base (ad172c7) to head (f9a6540).
Report is 13 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8994      +/-   ##
==========================================
- Coverage   76.33%   76.28%   -0.05%     
==========================================
  Files         465      466       +1     
  Lines       70565    70750     +185     
==========================================
+ Hits        53867    53975     +108     
- Misses      13355    13421      +66     
- Partials     3343     3354      +11     

Flag	Coverage Δ
unittests	76.28% <84.21%> (-0.05%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

[JmPotato]

What about using an atomic.Value to store []func()?

[Tema]

Just storing reference to []func() won't be enough as array append is not atomic operation in golang (it might trigger races in the resize flow). Or am I missing something?

[JmPotato]

Your consideration is thorough. My initial intention was to merge the lock on line 33 with line 34 into one structure, making it look more elegant.

[Tema]

My initial intention was to merge the lock on line 33 with line 34 into one structure, making it look more elegant.

Do you still insist on the change? If so, could you please show me how you propose to make it work with atomic.Value which will be atomic and race free?

[JmPotato]

I still prefer using a single structure, you could use a customed structure rather than atomic.Value.

[Tema]

@JmPotato the lock also covers init consumer to make sure there is no races. It is hard to reason whether we can give up on it or not. This not a hot code path, so should be fine to cover with lock to simplify reasoning.

[Tema]

@rleungx @niubell could you please take a look as well. I appreciate if you could elaborate a bit more what I'm missing here and how to achieve that.

[rleungx]

I think what @JmPotato means is:

type consumersInitializers struct {
	sync.Mutex
	initializers []func()
}

var ci consumersInitializers

func RegisterConsumer(initConsumer func()) {
	ci.Lock()
	defer ci.Unlock()
	ci.initializers = append(ci.initializers, initConsumer)
	initConsumer()
}

func initRegisteredConsumers() {
	ci.Lock()
	defer ci.Unlock()
	for _, initConsumer := range ci.initializers {
		initConsumer()
	}
}

[Tema]

Got it, thx! @JmPotato PTAL one more time.

[JmPotato]

Overall LGTM. My final question is whether a CircuitBreaker has a lifecycle? If we have a scenario where the old one is deprecated and a new CircuitBreaker is created, do we need to remove the corresponding consumer?

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JmPotato

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [JmPotato]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2025-01-23 07:34:23.4816742 +0000 UTC m=+338990.812593603: ☑️ agreed by JmPotato.

[Tema]

@rleungx could you please approve as well?