Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM can not support MySQL8.0 utf8mb3 character #8109

Closed
chaplinthink opened this issue Jan 19, 2023 · 3 comments
Closed

DM can not support MySQL8.0 utf8mb3 character #8109

chaplinthink opened this issue Jan 19, 2023 · 3 comments
Labels
area/dm Issues or PRs related to DM.

Comments

@chaplinthink
Copy link

What did you do?

When i sync mysql 8.0.28 table by dm, the console report error:
"RawCause": "[parser:1115]Unknown character set: 'utf8mb3'", "Workaround": "Please confirm your DDL statement is correct and needed. For TiDB compatible DDL, see https://docs.pingcap.com/tidb/stable/mysql-compatibility#ddl. You can usehandle-error command to skip or replace the DDL or add a binlog filter rule to ignore it if the DDL is not needed."

The create table sql:

CREATE TABLE  test.`test1` (
  `id` bigint NOT NULL AUTO_INCREMENT,
   `name` varchar(100)  NOT NULL ,
    PRIMARY KEY (`id`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=99 DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_general_ci COMMENT='test';

What did you expect to see?

No response

What did you see instead?

No response

Versions of the cluster

DM version (run dmctl -V or dm-worker -V or dm-master -V):

DM: Release Version: v6.4.0

Upstream MySQL/MariaDB server version:

MySQL 8.0.28

Downstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

Release Version: v6.1.0

How did you deploy DM: tiup or manually?

TiUP 

Other interesting information (system version, hardware config, etc):

current status of DM cluster (execute query-status <task-name> in dmctl)

{
    "result": true,
    "msg": "",
    "sources": [
        {
            "result": true,
            "msg": "",
            "sourceStatus": {
                "source": "source-test",
                "worker": "dm-192.168.11.11-8271",
                "result": null,
                "relayStatus": {
                    "masterBinlog": "(binlog.000429, 364018782)",
                    "masterBinlogGtid": "",
                    "relaySubDir": "6c041e1b-7058-11ed-86d8-0242ac120002.000001",
                    "relayBinlog": "(binlog.000429, 364018782)",
                    "relayBinlogGtid": "",
                    "relayCatchUpMaster": true,
                    "stage": "Running",
                    "result": null
                }
            },
            "subTaskStatus": [
                {
                    "name": "sync-test",
                    "stage": "Paused",
                    "unit": "Sync",
                    "result": {
                        "isCanceled": false,
                        "errors": [
                            {
                                "ErrCode": 36067,
                                "ErrClass": "sync-unit",
                                "ErrScope": "internal",
                                "ErrLevel": "high",
                                "Message": "startLocation: [position: (binlog|000001.000429, 39528101), gtid-set: ], endLocation: [position: (binlog|000001.000429, 39529568), gtid-set: ], origin SQL: [/* ApplicationName=DBeaver 22.3.1 - SQLEditor \u003cScript-6.sql\u003e */CREATE TABLE  test.`test1` (
  `id` bigint NOT NULL AUTO_INCREMENT,
   `name` varchar(100)  NOT NULL ,
    PRIMARY KEY (`id`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=99 DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_general_ci COMMENT='test'; ",
                                "RawCause": "[parser:1115]Unknown character set: 'utf8mb3'",
                                "Workaround": "Please confirm your DDL statement is correct and needed. For TiDB compatible DDL, see https://docs.pingcap.com/tidb/stable/mysql-compatibility#ddl. You can use `handle-error` command to skip or replace the DDL or add a binlog filter rule to ignore it if the DDL is not needed."
                            }
                        ],
                        "detail": null
                    },
                    "unresolvedDDLLockID": "",
                    "sync": {
                        "totalEvents": "674840950",
                        "totalTps": "4114883",
                        "recentTps": "0",
                        "masterBinlog": "(binlog.000429, 364018782)",
                        "masterBinlogGtid": "",
                        "syncerBinlog": "(binlog|000001.000429, 39528022)",
                        "syncerBinlogGtid": "",
                        "blockingDDLs": [
                        ],
                        "unresolvedGroups": [
                        ],
                        "synced": false,
                        "binlogType": "local",
                        "secondsBehindMaster": "0",
                        "blockDDLOwner": "",
                        "conflictMsg": "",
                        "totalRows": "0",
                        "totalRps": "0",
                        "recentRps": "0"
                    },
                    "validation": null
                }
            ]
        }
    ]
}
@chaplinthink chaplinthink added area/dm Issues or PRs related to DM. type/bug The issue is confirmed as a bug. labels Jan 19, 2023
@lance6716
Copy link
Contributor

TiDB does not support utf8mb3 now pingcap/tidb#26226

@chaplinthink
Copy link
Author

Can we replace utf8mb3 to utf8 when parse data? DM can parse utf8 correctly in mysql8.0.

@fubinzh
Copy link

fubinzh commented Jan 30, 2023

/remove-type bug

@ti-chi-bot ti-chi-bot removed the type/bug The issue is confirmed as a bug. label Jan 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/dm Issues or PRs related to DM.
Projects
None yet
Development

No branches or pull requests

4 participants