*: upgrade etcd to v3.5.15 #8441

lhy1024 · 2024-07-25T14:26:53Z

What problem does this PR solve?

Issue Number: Close #3877 Ref #5134 #5520

What is changed and how does it work?

Check List

Tests

Unit test

Release note

*: upgrade etcd to v3.5.15

ti-chi-bot · 2024-07-25T14:26:55Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

lhy1024 · 2024-08-07T10:13:22Z

pkg/utils/etcdutil/etcdutil.go

+	// If Linearizable is set to false, the member list will be returned with server's local data.
+	// If Linearizable is set to true, it is served with linearizable guarantee. If the server is disconnected from quorum, `MemberList` call will fail.
+	c := clientv3.RetryClusterClient(client)
+	resp, err := c.MemberList(newCtx, &etcdserverpb.MemberListRequest{Linearizable: false})


set it to false to keep it consistent with previous behavior.

Why is RetryClusterClient needed here and not elsewhere?

In order to pass &etcdserverpb.MemberListRequest{Linearizable: false}

If we use resp, err :=client.MemberList(ctx), it will always be true

https://github.com/etcd-io/etcd/blame/v3.5.15/client/v3/cluster.go#L125-L132

codecov · 2024-08-07T10:25:11Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.59%. Comparing base (1c1cd99) to head (35ae79d).
Report is 3 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8441      +/-   ##
==========================================
+ Coverage   77.40%   77.59%   +0.19%     
==========================================
  Files         473      474       +1     
  Lines       61934    61866      -68     
==========================================
+ Hits        47941    48006      +65     
+ Misses      10409    10326      -83     
+ Partials     3584     3534      -50

Flag	Coverage Δ
unittests	`77.59% <100.00%> (+0.19%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Signed-off-by: lhy1024 <[email protected]>

tests/integrations/go.mod

nolouch

lgtm

HuSharp · 2024-08-08T06:47:23Z

pkg/utils/etcdutil/etcdutil.go

@@ -88,7 +89,7 @@ func CheckClusterID(localClusterID types.ID, um types.URLsMap, tlsConfig *tls.Co
 		trp := &http.Transport{
 			TLSClientConfig: tlsConfig,
 		}
-		remoteCluster, gerr := etcdserver.GetClusterFromRemotePeers(nil, []string{u}, trp, true)


Is the default value of the new function GetClusterFromRemotePeers true?

It is introduced by #8017 and etcd-io/etcd#17330, it means Enable 3.4 to be compatible with next version 3.5, to allow 3.4 server to join 3.5 cluster and start on 3.5 schema.

lhy1024 · 2024-08-08T08:10:18Z

/hold
After discussion with @wuhuizuo, upgrade of etcd will be released in v8.4 rather than v8.3

go.mod

rleungx · 2024-08-08T08:33:57Z

server/config/config.go

@@ -739,6 +739,7 @@ func (c *Config) GenEmbedEtcdConfig() (*embed.Config, error) {
 	cfg.PeerTLSInfo.TrustedCAFile = c.Security.CAPath
 	cfg.PeerTLSInfo.CertFile = c.Security.CertPath
 	cfg.PeerTLSInfo.KeyFile = c.Security.KeyPath
+	//nolint:staticcheck


after etcd-io/etcd#18015, AllowedCN is Deprecated

It will be replaced by AllowedCNs in a later pr to support multi cn

tests/integrations/go.mod

pkg/utils/etcdutil/etcdutil.go

rleungx · 2024-08-08T08:38:10Z

pkg/utils/etcdutil/etcdutil.go

 	cancel()
 	if err != nil {
-		return listResp, errs.ErrEtcdMemberList.Wrap(err).GenWithStackByCause()
+		return (*clientv3.MemberListResponse)(resp), errs.ErrEtcdMemberList.Wrap(err).GenWithStackByCause()


why do we need (*clientv3.MemberListResponse)?

Because the return value of ListEtcdMembers is expected to be (*clientv3.MemberListResponse)

What is the current type without (*clientv3.MemberListResponse)?

Oh, I notice you are using retryClusterClient. Is it necessary?

Current type is etcdserverpb.MemberListResponse

In order to pass &etcdserverpb.MemberListRequest{Linearizable: false}

If we use resp, err :=client.MemberList(ctx), it will always be true

https://github.com/etcd-io/etcd/blame/v3.5.15/client/v3/cluster.go#L125-L132

client.MemberList call c.remote.MemberList. c.remote is built by RetryClusterClient
https://github.com/etcd-io/etcd/blob/v3.5.15/client/v3/cluster.go#L60-L67

Signed-off-by: lhy1024 <[email protected]>

ti-chi-bot · 2024-08-09T06:11:16Z

[LGTM Timeline notifier]

Timeline:

2024-08-08 03:07:47.977539285 +0000 UTC m=+494197.844638374: ☑️ agreed by nolouch.
2024-08-09 06:11:15.794041927 +0000 UTC m=+591605.661141015: ☑️ agreed by rleungx.

lhy1024 · 2024-08-14T03:03:40Z

/unhold
https://github.com/tikv/pd/tree/release-8.3 has been created cc @wuhuizuo

Signed-off-by: lhy1024 <[email protected]>

lhy1024 · 2024-08-15T05:20:27Z

/test pull-integration-realcluster-test

ti-chi-bot · 2024-08-15T05:56:25Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: niubell, nolouch, rleungx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [nolouch,rleungx]
~~server/config/OWNERS~~ [niubell]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

lhy1024 force-pushed the update-etcd branch from 3cf8786 to cc1b7f6 Compare July 30, 2024 09:36

lhy1024 mentioned this pull request Jul 30, 2024

Multiple CN support for TLS connections #5134

Closed

ti-chi-bot bot removed the do-not-merge/needs-linked-issue label Jul 30, 2024

lhy1024 changed the title ~~update etcd to 3.5.15~~ *: upgrade etcd to v3.5.15 Jul 30, 2024

ti-chi-bot bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Jul 30, 2024

lhy1024 mentioned this pull request Aug 2, 2024

update etcd to 3.5.15 pingcap/tidb-dashboard#1707

Merged

lhy1024 force-pushed the update-etcd branch from 2cabf06 to fe1dbf8 Compare August 7, 2024 10:12

lhy1024 marked this pull request as ready for review August 7, 2024 10:12

ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 7, 2024

lhy1024 commented Aug 7, 2024

View reviewed changes

lhy1024 added 4 commits August 7, 2024 18:59

*: upgrade etcd to v3.5.15

7ccd720

Signed-off-by: lhy1024 <[email protected]>

add nolint for allowedCN

ef81281

Signed-off-by: lhy1024 <[email protected]>

fix unstable test and avoid redundant close in test

b0a19eb

Signed-off-by: lhy1024 <[email protected]>

disable Linearizable

85ce2bb

Signed-off-by: lhy1024 <[email protected]>

lhy1024 force-pushed the update-etcd branch from fe1dbf8 to 85ce2bb Compare August 7, 2024 11:08

nolouch reviewed Aug 8, 2024

View reviewed changes

tests/integrations/go.mod Outdated Show resolved Hide resolved

nolouch approved these changes Aug 8, 2024

View reviewed changes

ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Aug 8, 2024

HuSharp reviewed Aug 8, 2024

View reviewed changes

ti-chi-bot bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 8, 2024

rleungx reviewed Aug 8, 2024

View reviewed changes

address comments: merge indirect go.mod and add comments

d099d54

Signed-off-by: lhy1024 <[email protected]>

rleungx approved these changes Aug 9, 2024

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Aug 9, 2024

lhy1024 mentioned this pull request Aug 12, 2024

security: support multiple CN for TLS connections #8518

Merged

ti-chi-bot bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 13, 2024

ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 14, 2024

lhy1024 added 2 commits August 15, 2024 13:06

update go.mod from dashboard

d0b7a2b

Signed-off-by: lhy1024 <[email protected]>

Merge branch 'master' of github.com:tikv/pd into update-etcd

c61057f

Signed-off-by: lhy1024 <[email protected]>

ti-chi-bot bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 15, 2024

fix conflict

35ae79d

Signed-off-by: lhy1024 <[email protected]>

niubell approved these changes Aug 15, 2024

View reviewed changes

ti-chi-bot bot added the approved label Aug 15, 2024

ti-chi-bot bot merged commit cc2900c into tikv:master Aug 15, 2024
25 checks passed

lhy1024 mentioned this pull request Aug 16, 2024

etcd: use Linearizable in memberList #8542

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: upgrade etcd to v3.5.15 #8441

*: upgrade etcd to v3.5.15 #8441

lhy1024 commented Jul 25, 2024 •

edited

Loading

ti-chi-bot bot commented Jul 25, 2024

lhy1024 Aug 7, 2024

HuSharp Aug 8, 2024

lhy1024 Aug 8, 2024 •

edited

Loading

codecov bot commented Aug 7, 2024 •

edited

Loading

nolouch left a comment

HuSharp Aug 8, 2024

lhy1024 Aug 8, 2024

lhy1024 commented Aug 8, 2024 •

edited

Loading

rleungx Aug 8, 2024

lhy1024 Aug 8, 2024

lhy1024 Aug 8, 2024

rleungx Aug 8, 2024

lhy1024 Aug 8, 2024

rleungx Aug 8, 2024

rleungx Aug 8, 2024

lhy1024 Aug 8, 2024

lhy1024 Aug 8, 2024 •

edited

Loading

lhy1024 Aug 9, 2024

ti-chi-bot bot commented Aug 9, 2024

lhy1024 commented Aug 14, 2024

lhy1024 commented Aug 15, 2024

ti-chi-bot bot commented Aug 15, 2024

*: upgrade etcd to v3.5.15 #8441

*: upgrade etcd to v3.5.15 #8441

Conversation

lhy1024 commented Jul 25, 2024 • edited Loading

What problem does this PR solve?

What is changed and how does it work?

Check List

Release note

ti-chi-bot bot commented Jul 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lhy1024 Aug 8, 2024 • edited Loading

Choose a reason for hiding this comment

codecov bot commented Aug 7, 2024 • edited Loading

Codecov Report

nolouch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lhy1024 commented Aug 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lhy1024 Aug 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ti-chi-bot bot commented Aug 9, 2024

[LGTM Timeline notifier]

lhy1024 commented Aug 14, 2024

lhy1024 commented Aug 15, 2024

ti-chi-bot bot commented Aug 15, 2024

lhy1024 commented Jul 25, 2024 •

edited

Loading

lhy1024 Aug 8, 2024 •

edited

Loading

codecov bot commented Aug 7, 2024 •

edited

Loading

lhy1024 commented Aug 8, 2024 •

edited

Loading

lhy1024 Aug 8, 2024 •

edited

Loading