Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sink(ticdc): Add Support for Multiple MySQL-Compatible Downstream Addresses in TiCDC for High Availability #11527

Open
wants to merge 52 commits into
base: master
Choose a base branch
from
Open

sink(ticdc): Add Support for Multiple MySQL-Compatible Downstream Addresses in TiCDC for High Availability #11527

wants to merge 52 commits into from

Commits on Aug 7, 2024

  1. sync to office

    @wlwilliamx
    wlwilliamx committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    0f06a11 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2024

  1. feat(retry): add WithPreExecutionWhenRetry option to execute action… 

    …s before each retry attempt

Introduced the `WithPreExecutionWhenRetry` feature to allow a PreExecution action to be specified,
which will be executed before each retry attempt, but only if the initial execution fails. This
ensures that the PreExecution function is triggered before every retry following the first failure.
If the initial execution is successful, no retry occurs, and the PreExecution function will not be
executed.
    @wlwilliamx
    wlwilliamx committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    7c8890f View commit details
    Browse the repository at this point in the history

  2. sync

    @wlwilliamx
    wlwilliamx committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    d856ea8 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. feat(DDLSink): support multi mysql downstream addresses for DDL Sink

    @wlwilliamx
    wlwilliamx committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    120561f View commit details
    Browse the repository at this point in the history

  2. feat(mysqlBackend): support multi mysql downstream addresses for DML … 

    …Sink
    @wlwilliamx
    wlwilliamx committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    6b1ca91 View commit details
    Browse the repository at this point in the history

  3. feat(mysqlSyncPointStore): support multi mysql downstream addresses f… 

    …or MySQL Syncpoint Store
    @wlwilliamx
    wlwilliamx committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    ccb58ff View commit details
    Browse the repository at this point in the history

  4. feat(Observer): support multi mysql downstream addresses for Observer

    @wlwilliamx
    wlwilliamx committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    735b089 View commit details
    Browse the repository at this point in the history

  5. refactor(Observer): remove a duplicate import

    @wlwilliamx
    wlwilliamx committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    4e826ce View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2024

  1. WIP

    @wlwilliamx
    wlwilliamx committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    b4d9191 View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2024

  1. Merge remote-tracking branch 'upstream/master' into feature/multi-mys… 

    …ql-addresses-in-sink-uri
    @wlwilliamx
    wlwilliamx committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    e4bacc0 View commit details
    Browse the repository at this point in the history

  2. test(MySQLDBConnector): add unit tests for MySQLDBConnector

    @wlwilliamx
    wlwilliamx committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    ec856f9 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2024

  1. fix(TestGenerateDSNByConfig): fix missed character_set_name query

    @wlwilliamx
    wlwilliamx committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    6a797aa View commit details
    Browse the repository at this point in the history

  2. fix(TestNewMySQLBackend): fix the idempotency of close mysql connector

    @wlwilliamx
    wlwilliamx committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    93aee29 View commit details
    Browse the repository at this point in the history

  3. test(mysql connector): add TestConfigureDBWhenSwitch_NilConfigureFunc… 

    …tion
    @wlwilliamx
    wlwilliamx committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    d0c844a View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. feat(start_tidb_cluster): support launching multiple downstream TiDB … 

    …instances within a single cluster
    @wlwilliamx
    wlwilliamx committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    54addd5 View commit details
    Browse the repository at this point in the history

  2. fix(start_tidb_cluster): fix the error in shell parsing of the port

    @wlwilliamx
    wlwilliamx committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    ea7a1a9 View commit details
    Browse the repository at this point in the history

  3. fix(test_prepare): fix the error of parsing DOWN_TIDB_STATUS by modif… 

    …ying all DOWN_TIDB_STATUS to DOWN_TIDB_STATUS_1
    @wlwilliamx
    wlwilliamx committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    9cc3c49 View commit details
    Browse the repository at this point in the history

  4. feat(start_tidb_cluster): add --downstream_db option to the `start_… 

    …tidb_cluster` command
    @wlwilliamx
    wlwilliamx committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    a772191 View commit details
    Browse the repository at this point in the history

  5. feat(start tidb): extract the code for starting the downstream tidb i… 

    …nstances into a separate script
    @wlwilliamx
    wlwilliamx committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    199b14c View commit details
    Browse the repository at this point in the history

  6. chore(start tidb): modify start_downstream_tidb_instances to execut… 

    …able
    @wlwilliamx
    wlwilliamx committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    2bf78b4 View commit details
    Browse the repository at this point in the history

  7. fix(start tidb): fix the typo "workdir"

    @wlwilliamx
    wlwilliamx committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    829c6f3 View commit details
    Browse the repository at this point in the history

  8. fix(start tidb): add source test_prepare to `start_downstream_tidb_… 

    …instances`
    @wlwilliamx
    wlwilliamx committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    e1967b0 View commit details
    Browse the repository at this point in the history

  9. feat(stop_tidb_cluster): add the operation to stop downstream tidb 2 … 

    …and 3
    @wlwilliamx
    wlwilliamx committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    36770a0 View commit details
    Browse the repository at this point in the history

  10. feat(start tidb): add a check for the value of the --db parameter

    @wlwilliamx
    wlwilliamx committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    b2c11e4 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. feat(start tidb): support start tidb instances with the specific port

    @wlwilliamx
    wlwilliamx committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    207f395 View commit details
    Browse the repository at this point in the history

  2. feat(start tidb): capture the PIDs of TiDB instances started by start… 

    …_downstream_tidb_instances script
    @wlwilliamx
    wlwilliamx committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    8867452 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. test(multi addresses): add a integration test for the multi downstrea… 

    …m addresses feature
    @wlwilliamx
    wlwilliamx committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    fb712a5 View commit details
    Browse the repository at this point in the history

  2. fix(connector): fix a data race in (*mysqlBackend).Close()

    @wlwilliamx
    wlwilliamx committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    e7808a6 View commit details
    Browse the repository at this point in the history

  3. fix(multi_down_addresses): fix the wrong SQL in integration test mult… 

    …i_down_addresses
    @wlwilliamx
    wlwilliamx committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    b8b6c49 View commit details
    Browse the repository at this point in the history

  4. fix(multi_down_addresses): fix the wrong config path for check_sync_diff

    @wlwilliamx
    wlwilliamx committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    3304d21 View commit details
    Browse the repository at this point in the history

  5. fix(multi_down_addresses): fix the wrong SQL of duplicate entry for a… 

    … primary key error
    @wlwilliamx
    wlwilliamx committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    fd7d71c View commit details
    Browse the repository at this point in the history

  6. fix(start_downstream_tidb_instances): fix the idempotence of `CREATE … 

    …USER`
    @wlwilliamx
    wlwilliamx committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    dd4f074 View commit details
    Browse the repository at this point in the history

  7. Merge remote-tracking branch 'upstream/master' into feature/multi-mys… 

    …ql-addresses-in-sink-uri
    @wlwilliamx
    wlwilliamx committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    970dc3d View commit details
    Browse the repository at this point in the history

  8. chore(connector): add the copyright information for the new files

    @wlwilliamx
    wlwilliamx committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    50573e9 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. chore(integration test): add the multi_down_addresses integration t… 

    …est to the `run_group.sh`
    @wlwilliamx
    wlwilliamx committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    b7c112a View commit details
    Browse the repository at this point in the history

  2. chore(integration test): format the new files by make check

    @wlwilliamx
    wlwilliamx committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    72e6e31 View commit details
    Browse the repository at this point in the history

  3. docs(connector): add and modify some comments by following the rule o… 

    …f `make check`
    @wlwilliamx
    wlwilliamx committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    4d8c0d4 View commit details
    Browse the repository at this point in the history

  4. refactor(connector): rename MySQLDBConnector to DBConnector by th… 

    …e suggestion of `make check`
    @wlwilliamx
    wlwilliamx committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    44a3c9b View commit details
    Browse the repository at this point in the history

  5. refactor(connector): rename the function SwitchToAvailableMySQLDB t… 

    …o `SwitchToAnAvailableDB`
    @wlwilliamx
    wlwilliamx committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    da73ffa View commit details
    Browse the repository at this point in the history

  6. test(multi_down_addresses): add cdc cli changefeed update integrati… 

    …on test
    @wlwilliamx
    wlwilliamx committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    78a80a4 View commit details
    Browse the repository at this point in the history

  7. fix(multi_down_addresses): modify the changefeed-id according to th… 

    …e naming convention for changefeed-id
    @wlwilliamx
    wlwilliamx committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    aec17b3 View commit details
    Browse the repository at this point in the history

  8. fix(multi_down_addresses): fix a naming error of tables

    @wlwilliamx
    wlwilliamx committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    97db8dd View commit details
    Browse the repository at this point in the history

  9. test(multi_down_addresses): add DDL SQL into multi_down_addresses i… 

    …ntegration test for testing DDL Sink when handling connection error
    @wlwilliamx
    wlwilliamx committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    3673ef8 View commit details
    Browse the repository at this point in the history

  10. fix(multi_down_addresses): fix a wrong SQL by duplicate entry for one… 

    … key
    @wlwilliamx
    wlwilliamx committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    e1c6a43 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. fix(DDL Sink): fix IsRetryableDDLError function to allow DDL to be … 

    …retryable when encounter network error
    @wlwilliamx
    wlwilliamx committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    13edec0 View commit details
    Browse the repository at this point in the history

  2. feat(connector): add log output

    @wlwilliamx
    wlwilliamx committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    bac745d View commit details
    Browse the repository at this point in the history

  3. refactor(connector): modify the log output of DBConnctor

    @wlwilliamx
    wlwilliamx committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    fc445c0 View commit details
    Browse the repository at this point in the history

  4. refactor(multi_down_addresses): modify multi_down_addresses integra… 

    …tion test
    @wlwilliamx
    wlwilliamx committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    9aeeeca View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. refactor(connector): modify the log output of DBConnctor

    @wlwilliamx
    wlwilliamx committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    757d8d0 View commit details
    Browse the repository at this point in the history

  2. fix(DDL Sink): fix the unit test failure of TestIsRetryableDDLError

    @wlwilliamx
    wlwilliamx committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    d5f4fa7 View commit details
    Browse the repository at this point in the history

  3. refactor(test_prepare): modify TLS_TIDB_PORT from 3307 to 3406

    @wlwilliamx
    wlwilliamx committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    3dbfef1 View commit details
    Browse the repository at this point in the history

  4. refactor(DDL Sink): remove leftover TODO tag

    @wlwilliamx
    wlwilliamx committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    d804633 View commit details
    Browse the repository at this point in the history