From c11c9502decebdcfba697cd3a3deac0a5ea688aa Mon Sep 17 00:00:00 2001 From: houfaxin Date: Wed, 18 Sep 2024 22:39:39 +0800 Subject: [PATCH 1/6] global index ga --- partitioned-table.md | 5 +---- system-variables.md | 4 ---- 2 files changed, 1 insertion(+), 8 deletions(-) diff --git a/partitioned-table.md b/partitioned-table.md index e3140b633ba3a..7d54e5b27d612 100644 --- a/partitioned-table.md +++ b/partitioned-table.md @@ -1695,10 +1695,6 @@ Before the introduction of global indexes, TiDB created a local index for each p To address these issues, TiDB introduces the global indexes feature in v8.3.0. A global index covers the data of the entire table with a single index, allowing primary keys and unique keys to maintain global uniqueness without including all partition keys. Moreover, global indexes can access data across multiple partitions in a single operation, significantly improving query performance for non-partitioned keys. -> **Warning:** -> -> The global indexes feature is experimental. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. - To create a global index for a primary key or unique key that **does not include all the columns used in the partition expressions**, you can enable the [`tidb_enable_global_index`](/system-variables.md#tidb_enable_global_index-new-in-v760) system variable and add the `GLOBAL` keyword in the index definition. > **Note:** @@ -1789,6 +1785,7 @@ ALTER TABLE t1 PARTITION BY HASH (col1) PARTITIONS 3 UPDATE INDEXES (uidx12 LOCA - The `GLOBAL` and `LOCAL` keywords only apply to partitioned tables and do not affect non-partitioned tables. In other words, there is no difference between a global index and a local index in non-partitioned tables. - DDL operations such as `ADD PARTITION`, `DROP PARTITION`, `TRUNCATE PARTITION`, `REORGANIZE PARTITION`, `SPLIT PARTITION`, and `EXCHANGE PARTITION` also trigger updates to global indexes. The results of these DDL operations will only be returned after the global indexes of the corresponding tables are fully updated. This can delay operations that usually require quick DDL completion, such as data archiving operations (`EXCHANGE PARTITION`, `TRUNCATE PARTITION`, and `DROP PARTITION`). In contrast, when global indexes are not involved, these DDL operations can be completed immediately. - By default, the primary key of a partitioned table is a clustered index and must include the partition key. If you require the primary key to exclude the partition key, you can explicitly specify the primary key as a non-clustered global index when creating the table, for example, `PRIMARY KEY(col1, col2) NONCLUSTERED GLOBAL`. +- If a global index is added to an expression column, you need to collect statistics manually for that global index. ### Partitioning limitations relating to functions diff --git a/system-variables.md b/system-variables.md index 1bb95aa4baabb..63143953dad53 100644 --- a/system-variables.md +++ b/system-variables.md @@ -2147,10 +2147,6 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; ### tidb_enable_global_index New in v7.6.0 -> **Warning:** -> -> The feature controlled by this variable is experimental. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. - - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No From 3a05d0ed6c6fbaaedf0c9fb6d3b4de43380106a9 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 19 Sep 2024 17:01:27 +0800 Subject: [PATCH 2/6] Update partitioned-table.md --- partitioned-table.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/partitioned-table.md b/partitioned-table.md index 7d54e5b27d612..fb44672d08430 100644 --- a/partitioned-table.md +++ b/partitioned-table.md @@ -1785,7 +1785,7 @@ ALTER TABLE t1 PARTITION BY HASH (col1) PARTITIONS 3 UPDATE INDEXES (uidx12 LOCA - The `GLOBAL` and `LOCAL` keywords only apply to partitioned tables and do not affect non-partitioned tables. In other words, there is no difference between a global index and a local index in non-partitioned tables. - DDL operations such as `ADD PARTITION`, `DROP PARTITION`, `TRUNCATE PARTITION`, `REORGANIZE PARTITION`, `SPLIT PARTITION`, and `EXCHANGE PARTITION` also trigger updates to global indexes. The results of these DDL operations will only be returned after the global indexes of the corresponding tables are fully updated. This can delay operations that usually require quick DDL completion, such as data archiving operations (`EXCHANGE PARTITION`, `TRUNCATE PARTITION`, and `DROP PARTITION`). In contrast, when global indexes are not involved, these DDL operations can be completed immediately. - By default, the primary key of a partitioned table is a clustered index and must include the partition key. If you require the primary key to exclude the partition key, you can explicitly specify the primary key as a non-clustered global index when creating the table, for example, `PRIMARY KEY(col1, col2) NONCLUSTERED GLOBAL`. -- If a global index is added to an expression column, you need to collect statistics manually for that global index. +- If a global index is added to an expression column, or a global index is also a prefix index (for example `UNIQUE KEY idx_email_prefix (email(10)) GLOBAL)`), you need to collect statistics manually for that global index. ### Partitioning limitations relating to functions From bd4465f7a4bba0deeee3e5eff82949b03c627c3d Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 19 Sep 2024 17:17:23 +0800 Subject: [PATCH 3/6] Update partitioned-table.md --- partitioned-table.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/partitioned-table.md b/partitioned-table.md index fb44672d08430..3ddcc50145fe3 100644 --- a/partitioned-table.md +++ b/partitioned-table.md @@ -1785,7 +1785,7 @@ ALTER TABLE t1 PARTITION BY HASH (col1) PARTITIONS 3 UPDATE INDEXES (uidx12 LOCA - The `GLOBAL` and `LOCAL` keywords only apply to partitioned tables and do not affect non-partitioned tables. In other words, there is no difference between a global index and a local index in non-partitioned tables. - DDL operations such as `ADD PARTITION`, `DROP PARTITION`, `TRUNCATE PARTITION`, `REORGANIZE PARTITION`, `SPLIT PARTITION`, and `EXCHANGE PARTITION` also trigger updates to global indexes. The results of these DDL operations will only be returned after the global indexes of the corresponding tables are fully updated. This can delay operations that usually require quick DDL completion, such as data archiving operations (`EXCHANGE PARTITION`, `TRUNCATE PARTITION`, and `DROP PARTITION`). In contrast, when global indexes are not involved, these DDL operations can be completed immediately. - By default, the primary key of a partitioned table is a clustered index and must include the partition key. If you require the primary key to exclude the partition key, you can explicitly specify the primary key as a non-clustered global index when creating the table, for example, `PRIMARY KEY(col1, col2) NONCLUSTERED GLOBAL`. -- If a global index is added to an expression column, or a global index is also a prefix index (for example `UNIQUE KEY idx_email_prefix (email(10)) GLOBAL)`), you need to collect statistics manually for that global index. +- If a global index is added to an expression column, or a global index is also a prefix index (for example `UNIQUE KEY idx_id_prefix (id(10)) GLOBAL`), you need to collect statistics manually for that global index. ### Partitioning limitations relating to functions From e232c44720d3f65c8f5517f7aa678227e30fb96f Mon Sep 17 00:00:00 2001 From: Lilian Lee Date: Mon, 23 Sep 2024 17:33:04 +0800 Subject: [PATCH 4/6] Update wording --- partitioned-table.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/partitioned-table.md b/partitioned-table.md index 3ddcc50145fe3..e6b40b2ccc4a5 100644 --- a/partitioned-table.md +++ b/partitioned-table.md @@ -1785,7 +1785,7 @@ ALTER TABLE t1 PARTITION BY HASH (col1) PARTITIONS 3 UPDATE INDEXES (uidx12 LOCA - The `GLOBAL` and `LOCAL` keywords only apply to partitioned tables and do not affect non-partitioned tables. In other words, there is no difference between a global index and a local index in non-partitioned tables. - DDL operations such as `ADD PARTITION`, `DROP PARTITION`, `TRUNCATE PARTITION`, `REORGANIZE PARTITION`, `SPLIT PARTITION`, and `EXCHANGE PARTITION` also trigger updates to global indexes. The results of these DDL operations will only be returned after the global indexes of the corresponding tables are fully updated. This can delay operations that usually require quick DDL completion, such as data archiving operations (`EXCHANGE PARTITION`, `TRUNCATE PARTITION`, and `DROP PARTITION`). In contrast, when global indexes are not involved, these DDL operations can be completed immediately. - By default, the primary key of a partitioned table is a clustered index and must include the partition key. If you require the primary key to exclude the partition key, you can explicitly specify the primary key as a non-clustered global index when creating the table, for example, `PRIMARY KEY(col1, col2) NONCLUSTERED GLOBAL`. -- If a global index is added to an expression column, or a global index is also a prefix index (for example `UNIQUE KEY idx_id_prefix (id(10)) GLOBAL`), you need to collect statistics manually for that global index. +- If a global index is added to an expression column, or a global index is also a prefix index (for example `UNIQUE KEY idx_id_prefix (id(10)) GLOBAL`), you need to collect statistics manually for this global index. ### Partitioning limitations relating to functions From 5effba706aa9129849ecd88b226918f4ef494cad Mon Sep 17 00:00:00 2001 From: houfaxin Date: Fri, 27 Sep 2024 10:17:04 +0800 Subject: [PATCH 5/6] Update choose-index.md --- choose-index.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/choose-index.md b/choose-index.md index 19bf0f2f49d53..b6ffa7c9afc31 100644 --- a/choose-index.md +++ b/choose-index.md @@ -73,7 +73,7 @@ mysql> SHOW WARNINGS; ### Skyline-pruning -Skyline-pruning is a heuristic filtering rule for indexes, which can reduce the probability of wrong index selection caused by wrong estimation. To judge an index, the following three dimensions are needed: +Skyline-pruning is a heuristic filtering rule for indexes, which can reduce the probability of wrong index selection caused by wrong estimation. To judge an index, the following dimensions are needed: - How many access conditions are covered by the indexed columns. An "access condition" is a where condition that can be converted to a column range. And the more access conditions an indexed column set covers, the better it is in this dimension. @@ -81,7 +81,9 @@ Skyline-pruning is a heuristic filtering rule for indexes, which can reduce the - Select whether the index satisfies a certain order. Because index reading can guarantee the order of certain column sets, indexes that satisfy the query order are superior to indexes that do not satisfy on this dimension. -For these three dimensions above, if the index `idx_a` performs no worse than the index `idx_b` in all three dimensions and performs better than `idx_b` in one dimension, then `idx_a` is preferred. When executing the `EXPLAIN FORMAT = 'verbose' ...` statement, if skyline-pruning excludes some indexes, TiDB outputs a NOTE-level warning listing the remaining indexes after the skyline-pruning exclusion. +- Whether the index is a [global index](/partitioned-table.md#global-indexes). In partitioned tables, global indexes can effectively reduce the number of cop tasks for a SQL compared to normal indexes, thus reducing network overhead. + +For these preceding dimensions, if the index `idx_a` performs no worse than the index `idx_b` in all three dimensions and performs better than `idx_b` in one dimension, then `idx_a` is preferred. When executing the `EXPLAIN FORMAT = 'verbose' ...` statement, if skyline-pruning excludes some indexes, TiDB outputs a NOTE-level warning listing the remaining indexes after the skyline-pruning exclusion. In the following example, the indexes `idx_b` and `idx_e` are both inferior to `idx_b_c`, so they are excluded by skyline-pruning. The returned result of `SHOW WARNING` displays the remaining indexes after skyline-pruning. From 922373381ce1effdbd97907508a7118cf469e4d4 Mon Sep 17 00:00:00 2001 From: houfaxin Date: Fri, 27 Sep 2024 10:18:58 +0800 Subject: [PATCH 6/6] Update choose-index.md --- choose-index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/choose-index.md b/choose-index.md index b6ffa7c9afc31..5770c1e58544c 100644 --- a/choose-index.md +++ b/choose-index.md @@ -81,7 +81,7 @@ Skyline-pruning is a heuristic filtering rule for indexes, which can reduce the - Select whether the index satisfies a certain order. Because index reading can guarantee the order of certain column sets, indexes that satisfy the query order are superior to indexes that do not satisfy on this dimension. -- Whether the index is a [global index](/partitioned-table.md#global-indexes). In partitioned tables, global indexes can effectively reduce the number of cop tasks for a SQL compared to normal indexes, thus reducing network overhead. +- Whether the index is a [global index](/partitioned-table.md#global-indexes). In partitioned tables, global indexes can effectively reduce the number of cop tasks for a SQL compared to normal indexes, thus improving overall performance. For these preceding dimensions, if the index `idx_a` performs no worse than the index `idx_b` in all three dimensions and performs better than `idx_b` in one dimension, then `idx_a` is preferred. When executing the `EXPLAIN FORMAT = 'verbose' ...` statement, if skyline-pruning excludes some indexes, TiDB outputs a NOTE-level warning listing the remaining indexes after the skyline-pruning exclusion.