Read local replica to reduce read latency in cross region scenario

[scsldb]

Description

In the cross-region deployment (#18273), let's say region A and B, read requests from region B may need to send messages to region A to make sure the read requests can see the latest update. These cross-region communications will result in high read latency.

• If the read requests don't need to read the latest update, they can tolerant reading stale data, the cross-region communication can be eliminated. In this case, reading local replica directly.
• If the read requests do need to read the latest update, we should keep the read latency is low enough, but the write latency may increase a little.

Tasks

• reading local replica directly, doesn't have the guarantee of "read after write"
• writing need to confirm all replicas have the latest update before return to the client.

Category

Performance

Value

Provide low latency reading in cross-region scenario

Workload Estimation

150

Time

GanttStart: 20xx-xx-xx
GanttDue: 20xx-xx-xx
GanttProgress: 0%

[zz-jason]

It's like the strong read and stale read in Cloud Spanner:

• Strong read requests can go to any read-write or read-only replica. If the request goes to a non-leader replica, that replica must communicate with the leader in order to execute the read.

• Stale read requests go to the closest available read-only or read-write replica that is caught up to the timestamp of the request. This can be the leader replica if the leader is the closest replica to the client that issued the read request.

See the document for more detail: Replication | Cloud Spanner

[zz-jason]

A use case on this feature:

• Applications config the preferred data staleness (30 seconds for example) in certain TiDB instances, or just on some certain queries. TiDB can offer both transaction-level and instance-level (via config or instance-level system variable) configuration for the staleness since both level have their use scenarios. We can use the start transaction/begin syntax described in support timestamp bounded read-only transactions on data replicas #19817 as the transaction-level hint.
• TiDB sends the coprocessor requests to the replica in the local AZ (Available Zone), the replica can be Leader, Follower, or Learner. If it's Follower or Learner and the staleness satisfies the application requirement, no read-index request (which may involve a cross-region network round trip) is sent to the Leader replica. Otherwise, a read-index request is needed to satisfy the requirement.

[nolouch]

finished.