From 4a3ebabeb2159b917cd64cade3febf1d0c43fbc8 Mon Sep 17 00:00:00 2001 From: nhsmw Date: Mon, 11 Nov 2024 14:42:40 +0800 Subject: [PATCH 1/5] Update ticdc-faq.md --- ticdc/ticdc-faq.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/ticdc/ticdc-faq.md b/ticdc/ticdc-faq.md index 0e4b78e5ec58f..1b82293098ce1 100644 --- a/ticdc/ticdc-faq.md +++ b/ticdc/ticdc-faq.md @@ -332,3 +332,7 @@ This is because the default port number of the TiCDC cluster deployed by TiDB Op } ] ``` + +## What happens to generated columns in DML when TiCDC replicates data to Kafka or storage services? + +Generated columns include stored generated columns and virtual generated columns. TiCDC will ignore virtual generated columns and only replicate stored generated columns to downstream. When stored generated columns replicate to downstream, such as Kafka or storage services, and then write back to MySQL, an error will occur: Error 3105 (HY000): The value specified for generated column 'xx' in table 'xxx' is not allowed. From 9af6633eba6ba431a0edd0af22d3db502ff3cf84 Mon Sep 17 00:00:00 2001 From: nhsmw Date: Mon, 11 Nov 2024 15:03:27 +0800 Subject: [PATCH 2/5] Update ticdc-faq.md --- ticdc/ticdc-faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ticdc/ticdc-faq.md b/ticdc/ticdc-faq.md index 1b82293098ce1..3ce74729fc6ab 100644 --- a/ticdc/ticdc-faq.md +++ b/ticdc/ticdc-faq.md @@ -335,4 +335,4 @@ This is because the default port number of the TiCDC cluster deployed by TiDB Op ## What happens to generated columns in DML when TiCDC replicates data to Kafka or storage services? -Generated columns include stored generated columns and virtual generated columns. TiCDC will ignore virtual generated columns and only replicate stored generated columns to downstream. When stored generated columns replicate to downstream, such as Kafka or storage services, and then write back to MySQL, an error will occur: Error 3105 (HY000): The value specified for generated column 'xx' in table 'xxx' is not allowed. +Generated columns include stored generated columns and virtual generated columns. TiCDC will ignore virtual generated columns and only replicate stored generated columns to downstream. When stored generated columns replicate to downstream, such as Kafka or storage services, and then write back to MySQL, an error may occur `Error 3105 (HY000): The value specified for generated column 'xx' in table 'xxx' is not allowed`. you could use [Open Protocol](https://docs.pingcap.com/zh/tidb/stable/ticdc-open-protocol#ticdc-open-protocol) to avoid this error because this protocol carries column type that the generated could be distinguished. From f5b65a5755e6f857b7a0f0e611f9531ce0002468 Mon Sep 17 00:00:00 2001 From: nhsmw Date: Mon, 11 Nov 2024 15:07:26 +0800 Subject: [PATCH 3/5] Update ticdc-faq.md --- ticdc/ticdc-faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ticdc/ticdc-faq.md b/ticdc/ticdc-faq.md index 3ce74729fc6ab..ced446b173284 100644 --- a/ticdc/ticdc-faq.md +++ b/ticdc/ticdc-faq.md @@ -335,4 +335,4 @@ This is because the default port number of the TiCDC cluster deployed by TiDB Op ## What happens to generated columns in DML when TiCDC replicates data to Kafka or storage services? -Generated columns include stored generated columns and virtual generated columns. TiCDC will ignore virtual generated columns and only replicate stored generated columns to downstream. When stored generated columns replicate to downstream, such as Kafka or storage services, and then write back to MySQL, an error may occur `Error 3105 (HY000): The value specified for generated column 'xx' in table 'xxx' is not allowed`. you could use [Open Protocol](https://docs.pingcap.com/zh/tidb/stable/ticdc-open-protocol#ticdc-open-protocol) to avoid this error because this protocol carries column type that the generated could be distinguished. +Generated columns include stored generated columns and virtual generated columns. TiCDC will ignore virtual generated columns and only replicate stored generated columns to downstream. When stored generated columns replicate to downstream, such as Kafka or storage services, and then write back to MySQL, an error may occur `Error 3105 (HY000): The value specified for generated column 'xx' in table 'xxx' is not allowed`. you could use [Open Protocol](https://docs.pingcap.com/zh/tidb/stable/ticdc-open-protocol#ticdc-open-protocol) to avoid this error because this protocol carries column type that the generated column could be distinguished. From 17852893d328d98dc6e7195c5581edd32ae646dd Mon Sep 17 00:00:00 2001 From: nhsmw Date: Mon, 11 Nov 2024 15:23:25 +0800 Subject: [PATCH 4/5] Update ticdc-faq.md --- ticdc/ticdc-faq.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/ticdc/ticdc-faq.md b/ticdc/ticdc-faq.md index ced446b173284..93d3d29c86b3d 100644 --- a/ticdc/ticdc-faq.md +++ b/ticdc/ticdc-faq.md @@ -333,6 +333,10 @@ This is because the default port number of the TiCDC cluster deployed by TiDB Op ] ``` -## What happens to generated columns in DML when TiCDC replicates data to Kafka or storage services? +## What happens to generated columns in DML? -Generated columns include stored generated columns and virtual generated columns. TiCDC will ignore virtual generated columns and only replicate stored generated columns to downstream. When stored generated columns replicate to downstream, such as Kafka or storage services, and then write back to MySQL, an error may occur `Error 3105 (HY000): The value specified for generated column 'xx' in table 'xxx' is not allowed`. you could use [Open Protocol](https://docs.pingcap.com/zh/tidb/stable/ticdc-open-protocol#ticdc-open-protocol) to avoid this error because this protocol carries column type that the generated column could be distinguished. +Generated columns include stored generated columns and virtual generated columns. TiCDC will ignore virtual generated columns and only replicate stored generated columns to downstream. The stored generated columns are also ignored when the downstream is MySQL database or other MySQL-compatible databases, but not Kafka or other storage services. + +> **Note:** +> +> When replicating stored generated columns to Kafka or storage services, and then writing back to MySQL, an error may occur `Error 3105 (HY000): The value specified for generated column 'xx' in table 'xxx' is not allowed`. you could use [Open Protocol](https://docs.pingcap.com/zh/tidb/stable/ticdc-open-protocol#ticdc-open-protocol) to avoid this error because this protocol carries column type that the generated column could be distinguished. From 2d5deef9bdd9d7e8949862562a897f4299f6a5df Mon Sep 17 00:00:00 2001 From: nhsmw Date: Wed, 22 Jan 2025 14:22:22 +0800 Subject: [PATCH 5/5] Update ticdc/ticdc-faq.md Co-authored-by: Grace Cai --- ticdc/ticdc-faq.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ticdc/ticdc-faq.md b/ticdc/ticdc-faq.md index 93d3d29c86b3d..4aa723f5b2750 100644 --- a/ticdc/ticdc-faq.md +++ b/ticdc/ticdc-faq.md @@ -333,10 +333,10 @@ This is because the default port number of the TiCDC cluster deployed by TiDB Op ] ``` -## What happens to generated columns in DML? +## Does TiCDC replicate generated columns of DML operations? -Generated columns include stored generated columns and virtual generated columns. TiCDC will ignore virtual generated columns and only replicate stored generated columns to downstream. The stored generated columns are also ignored when the downstream is MySQL database or other MySQL-compatible databases, but not Kafka or other storage services. +Generated columns include virtual generated columns and stored generated columns. TiCDC ignores virtual generated columns and only replicates stored generated columns to the downstream. However, stored generated columns are also ignored when the downstream is MySQL or another MySQL-compatible database (rather than Kafka or other storage services). > **Note:** > -> When replicating stored generated columns to Kafka or storage services, and then writing back to MySQL, an error may occur `Error 3105 (HY000): The value specified for generated column 'xx' in table 'xxx' is not allowed`. you could use [Open Protocol](https://docs.pingcap.com/zh/tidb/stable/ticdc-open-protocol#ticdc-open-protocol) to avoid this error because this protocol carries column type that the generated column could be distinguished. +> When replicating stored generated columns to Kafka or a storage service and then writing them back to MySQL, `Error 3105 (HY000): The value specified for generated column 'xx' in table 'xxx' is not allowed` might occur. To avoid this error, you can use [Open Protocol](/ticdc/ticdc-open-protocol.md#ticdc-open-protocol) for replication. The output of this protocol includes [bit flags of columns](/ticdc/ticdc-open-protocol.md#bit-flags-of-columns), which can distinguish whether a column is a generated column.