Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: remove TiDB Binlog after deprecation #18908

Merged
merged 8 commits into from
Sep 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 0 additions & 25 deletions TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,6 @@
- [PD Microservices Topology](/pd-microservices-deployment-topology.md)
- [TiProxy Topology](/tiproxy/tiproxy-deployment-topology.md)
- [TiCDC Topology](/ticdc-deployment-topology.md)
- [TiDB Binlog Topology](/tidb-binlog-deployment-topology.md)
- [TiSpark Topology](/tispark-deployment-topology.md)
- [Cross-DC Topology](/geo-distributed-deployment-topology.md)
- [Hybrid Topology](/hybrid-deployment-topology.md)
Expand Down Expand Up @@ -604,26 +603,6 @@
- [Troubleshoot](/ticdc/troubleshoot-ticdc.md)
- [FAQs](/ticdc/ticdc-faq.md)
- [Glossary](/ticdc/ticdc-glossary.md)
- TiDB Binlog (Deprecated)
- [Overview](/tidb-binlog/tidb-binlog-overview.md)
- [Quick Start](/tidb-binlog/get-started-with-tidb-binlog.md)
- [Deploy](/tidb-binlog/deploy-tidb-binlog.md)
- [Maintain](/tidb-binlog/maintain-tidb-binlog-cluster.md)
- [Configure](/tidb-binlog/tidb-binlog-configuration-file.md)
- [Pump](/tidb-binlog/tidb-binlog-configuration-file.md#pump)
- [Drainer](/tidb-binlog/tidb-binlog-configuration-file.md#drainer)
- [Upgrade](/tidb-binlog/upgrade-tidb-binlog.md)
- [Monitor](/tidb-binlog/monitor-tidb-binlog-cluster.md)
- [Reparo](/tidb-binlog/tidb-binlog-reparo.md)
- [binlogctl](/tidb-binlog/binlog-control.md)
- [Binlog Consumer Client](/tidb-binlog/binlog-consumer-client.md)
- [TiDB Binlog Relay Log](/tidb-binlog/tidb-binlog-relay-log.md)
- [Bidirectional Replication Between TiDB Clusters](/tidb-binlog/bidirectional-replication-between-tidb-clusters.md)
- [Glossary](/tidb-binlog/tidb-binlog-glossary.md)
- Troubleshoot
- [Troubleshoot](/tidb-binlog/troubleshoot-tidb-binlog.md)
- [Handle Errors](/tidb-binlog/handle-tidb-binlog-errors.md)
- [FAQ](/tidb-binlog/tidb-binlog-faq.md)
- PingCAP Clinic Diagnostic Service
- [Overview](/clinic/clinic-introduction.md)
- [Quick Start](/clinic/quick-start-with-clinic.md)
Expand Down Expand Up @@ -766,8 +745,6 @@
- [`CALIBRATE RESOURCE`](/sql-statements/sql-statement-calibrate-resource.md)
- [`CANCEL IMPORT JOB`](/sql-statements/sql-statement-cancel-import-job.md)
- [`COMMIT`](/sql-statements/sql-statement-commit.md)
- [`CHANGE DRAINER`](/sql-statements/sql-statement-change-drainer.md)
- [`CHANGE PUMP`](/sql-statements/sql-statement-change-pump.md)
- [`CREATE BINDING`](/sql-statements/sql-statement-create-binding.md)
- [`CREATE DATABASE`](/sql-statements/sql-statement-create-database.md)
- [`CREATE INDEX`](/sql-statements/sql-statement-create-index.md)
Expand Down Expand Up @@ -847,7 +824,6 @@
- [`SHOW CREATE TABLE`](/sql-statements/sql-statement-show-create-table.md)
- [`SHOW CREATE USER`](/sql-statements/sql-statement-show-create-user.md)
- [`SHOW DATABASES`](/sql-statements/sql-statement-show-databases.md)
- [`SHOW DRAINER STATUS`](/sql-statements/sql-statement-show-drainer-status.md)
- [`SHOW ENGINES`](/sql-statements/sql-statement-show-engines.md)
- [`SHOW ERRORS`](/sql-statements/sql-statement-show-errors.md)
- [`SHOW FIELDS FROM`](/sql-statements/sql-statement-show-fields-from.md)
Expand All @@ -862,7 +838,6 @@
- [`SHOW PRIVILEGES`](/sql-statements/sql-statement-show-privileges.md)
- [`SHOW PROCESSLIST`](/sql-statements/sql-statement-show-processlist.md)
- [`SHOW PROFILES`](/sql-statements/sql-statement-show-profiles.md)
- [`SHOW PUMP STATUS`](/sql-statements/sql-statement-show-pump-status.md)
- [`SHOW SCHEMAS`](/sql-statements/sql-statement-show-schemas.md)
- [`SHOW STATS_BUCKETS`](/sql-statements/sql-statement-show-stats-buckets.md)
- [`SHOW STATS_HEALTHY`](/sql-statements/sql-statement-show-stats-healthy.md)
Expand Down
43 changes: 3 additions & 40 deletions alert-rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ aliases: ['/docs/dev/alert-rules/','/docs/dev/reference/alert-rules/']

# TiDB Cluster Alert Rules

This document describes the alert rules for different components in a TiDB cluster, including the rule descriptions and solutions of the alert items in TiDB, TiKV, PD, TiFlash, TiDB Binlog, TiCDC, Node_exporter and Blackbox_exporter.
This document describes the alert rules for different components in a TiDB cluster, including the rule descriptions and solutions of the alert items in TiDB, TiKV, PD, TiFlash, TiCDC, Node_exporter and Blackbox_exporter.

According to the severity level, alert rules are divided into three categories (from high to low): emergency-level, critical-level, and warning-level. This division of severity levels applies to all alert items of each component below.

Expand Down Expand Up @@ -123,12 +123,11 @@ This section gives the alert rules for the TiDB component.
The number of events that happen in the TiDB service. An alert is triggered when the following events happen:

1. start: The TiDB service starts.
2. hang: When a critical-level event (currently there is only one scenario: TiDB cannot write binlog) happens, TiDB enters the `hang` mode and waits to be killed manually.
2. hang: When a critical-level event happens, TiDB enters the `hang` mode and waits to be killed manually.

* Solution:

* Restart TiDB to recover the service.
* Check whether the TiDB Binlog service is normal.
Restart TiDB to recover the service.

#### `TiDB_tikvclient_backoff_seconds_count`

Expand Down Expand Up @@ -774,10 +773,6 @@ This section gives the alert rules for the TiKV component.

For the detailed descriptions of TiFlash alert rules, see [TiFlash Alert Rules](/tiflash/tiflash-alert-rules.md).

## TiDB Binlog alert rules

For the detailed descriptions of TiDB Binlog alert rules, see [TiDB Binlog monitoring document](/tidb-binlog/monitor-tidb-binlog-cluster.md#alert-rules).

## TiCDC Alert rules

For the detailed descriptions of TiCDC alert rules, see [TiCDC Alert Rules](/ticdc/ticdc-alert-rules.md).
Expand Down Expand Up @@ -967,38 +962,6 @@ This section gives the alert rules for the Blackbox_exporter TCP, ICMP, and HTTP
* Check whether the TiFlash process exists.
* Check whether the network between the monitoring machine and the TiFlash machine is normal.

#### `Pump_server_is_down`

* Alert rule:

`probe_success{group="pump"} == 0`

* Description:

Failure to probe the pump service port.

* Solution:

* Check whether the machine that provides the pump service is down.
* Check whether the pump process exists.
* Check whether the network between the monitoring machine and the pump machine is normal.

#### `Drainer_server_is_down`

* Alert rule:

`probe_success{group="drainer"} == 0`

* Description:

Failure to probe the Drainer service port.

* Solution:

* Check whether the machine that provides the Drainer service is down.
* Check whether the Drainer process exists.
* Check whether the network between the monitoring machine and the Drainer machine is normal.

#### `TiKV_server_is_down`

* Alert rule:
Expand Down
4 changes: 2 additions & 2 deletions basic-features.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ You can try out TiDB features on [TiDB Playground](https://play.tidbcloud.com/?u
| [Dumpling logical dumper](/dumpling-overview.md) | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| [Transactional `LOAD DATA`](/sql-statements/sql-statement-load-data.md) [^5] | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| [Database migration toolkit (DM)](/migration-overview.md) | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| [TiDB Binlog](/tidb-binlog/tidb-binlog-overview.md) [^6] | Deprecated | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| [TiDB Binlog](https://docs.pingcap.com/tidb/v8.3/tidb-binlog-overview) [^6] | Deprecated | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| [Change data capture (CDC)](/ticdc/ticdc-overview.md) | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| [Stream data to Amazon S3, GCS, Azure Blob Storage, and NFS through TiCDC](/ticdc/ticdc-sink-to-cloud-storage.md) | Y | Y | Y | Y | Y | E | N | N | N | N | N |
| [TiCDC supports bidirectional replication between two TiDB clusters](/ticdc/ticdc-bidirectional-replication.md) | Y | Y | Y | Y | Y | Y | N | N | N | N | N |
Expand Down Expand Up @@ -274,4 +274,4 @@ You can try out TiDB features on [TiDB Playground](https://play.tidbcloud.com/?u

[^5]: Starting from [TiDB v7.0.0](/releases/release-7.0.0.md), the new parameter `FIELDS DEFINED NULL BY` and support for importing data from S3 and GCS are experimental features. Starting from [v7.6.0](/releases/release-7.6.0.md), TiDB processes `LOAD DATA` in transactions in the same way as MySQL. The `LOAD DATA` statement in a transaction no longer automatically commits the current transaction or starts a new transaction. Moreover, you can explicitly commit or roll back the `LOAD DATA` statement in a transaction. Additionally, the `LOAD DATA` statement is affected by the TiDB transaction mode setting (optimistic or pessimistic transaction).

[^6]: Starting from v7.5.0, [TiDB Binlog](/tidb-binlog/tidb-binlog-overview.md) replication is deprecated. Starting from v8.3.0, TiDB Binlog is fully deprecated, with removal planned for a future release. For incremental data replication, use [TiCDC](/ticdc/ticdc-overview.md) instead. For point-in-time recovery (PITR), use [PITR](/br/br-pitr-guide.md).
[^6]: Starting from v7.5.0, [TiDB Binlog](https://docs.pingcap.com/tidb/v8.3/tidb-binlog-overview) replication is deprecated. Starting from v8.3.0, TiDB Binlog is fully deprecated. Starting from v8.4.0, TiDB Binlog is removed. For incremental data replication, use [TiCDC](/ticdc/ticdc-overview.md) instead. For point-in-time recovery (PITR), use [PITR](/br/br-pitr-guide.md).
4 changes: 0 additions & 4 deletions binary-package.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,12 +59,8 @@ The `TiDB-community-toolkit` package contains the following contents.
| errdoc-{version}-linux-{arch}.tar.gz | |
| dba-{version}-linux-{arch}.tar.gz | |
| PCC-{version}-linux-{arch}.tar.gz | |
| pump-{version}-linux-{arch}.tar.gz | |
| drainer-{version}-linux-{arch}.tar.gz | |
| binlogctl | New in v6.0.0 |
| sync_diff_inspector | |
| reparo | |
| arbiter | |
| server-{version}-linux-{arch}.tar.gz | New in v6.2.0 |
| grafana-{version}-linux-{arch}.tar.gz | New in v6.2.0 |
| alertmanager-{version}-linux-{arch}.tar.gz | New in v6.2.0 |
Expand Down
9 changes: 0 additions & 9 deletions clustered-indexes.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,15 +147,6 @@ Currently, there are several different types of limitations for the clustered in
- Downgrading tables with clustered indexes is not supported. If you need to downgrade such tables, use logical backup tools to migrate data instead.
- Situations that are not supported yet but in the support plan:
- Adding, dropping, and altering clustered indexes using `ALTER TABLE` statements are not supported.
- Limitations for specific versions:
- In v5.0, using the clustered index feature together with TiDB Binlog is not supported. After TiDB Binlog is enabled, TiDB only allows creating a single integer column as the clustered index of a primary key. TiDB Binlog does not replicate data changes (such as insertion, deletion, and update) on existing tables with clustered indexes to the downstream. If you need to replicate tables with clustered indexes to the downstream, upgrade your cluster to v5.1 or use [TiCDC](https://docs.pingcap.com/tidb/stable/ticdc-overview) for replication instead.

After TiDB Binlog is enabled, if the clustered index you create is not a single integer primary key, TiDB returns the following error:

```sql
mysql> CREATE TABLE t (a VARCHAR(255) PRIMARY KEY CLUSTERED);
ERROR 8200 (HY000): Cannot create clustered index table when the binlog is ON
```

If you use clustered indexes together with the attribute `SHARD_ROW_ID_BITS`, TiDB reports the following error:

Expand Down
5 changes: 0 additions & 5 deletions command-line-flags-for-tidb-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,6 @@ When you start the TiDB cluster, you can use command-line options or environment
- Specifies the `Access-Control-Allow-Origin` value for Cross-Origin Request Sharing (CORS) request of the TiDB HTTP status service
- Default: `""`

## `--enable-binlog`

+ Enables or disables TiDB binlog generation
+ Default: `false`

## `--host`

- The host address that the TiDB server monitors
Expand Down
1 change: 0 additions & 1 deletion credits.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ TiDB developers contribute to new feature development, performance improvement,
- [pingcap/tiflash](https://github.com/pingcap/tiflash/graphs/contributors)
- [pingcap/tidb-operator](https://github.com/pingcap/tidb-operator/graphs/contributors)
- [pingcap/tiup](https://github.com/pingcap/tiup/graphs/contributors)
- [pingcap/tidb-binlog](https://github.com/pingcap/tidb-binlog/graphs/contributors)
- [pingcap/tidb-dashboard](https://github.com/pingcap/tidb-dashboard/graphs/contributors)
- [pingcap/tiflow](https://github.com/pingcap/tiflow/graphs/contributors)
- [pingcap/tidb-tools](https://github.com/pingcap/tidb-tools/graphs/contributors)
Expand Down
1 change: 0 additions & 1 deletion download-ecosystem-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ Depending on which tools you want to use, you can install the corresponding offl
| [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md) | `tidb-lightning-ctl` <br/>`tidb-lightning-{version}-linux-{arch}.tar.gz` |
| [TiDB Data Migration (DM)](/dm/dm-overview.md) | `dm-worker-{version}-linux-{arch}.tar.gz` <br/>`dm-master-{version}-linux-{arch}.tar.gz` <br/>`dmctl-{version}-linux-{arch}.tar.gz` |
| [TiCDC](/ticdc/ticdc-overview.md) | `cdc-{version}-linux-{arch}.tar.gz` |
| [TiDB Binlog](/tidb-binlog/tidb-binlog-overview.md) | `pump-{version}-linux-{arch}.tar.gz` <br/>`drainer-{version}-linux-{arch}.tar.gz` <br/>`binlogctl` <br/>`reparo` |
| [Backup & Restore (BR)](/br/backup-and-restore-overview.md) | `br-{version}-linux-{arch}.tar.gz` |
| [sync-diff-inspector](/sync-diff-inspector/sync-diff-inspector-overview.md) | `sync_diff_inspector` |
| [PD Recover](/pd-recover.md) | `pd-recover-{version}-linux-{arch}.tar` |
Expand Down
15 changes: 0 additions & 15 deletions ecosystem-tool-user-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,21 +123,6 @@ The following are the basics of TiCDC:
- Target: TiDB clusters, MySQL, Kafka, and Confluent
- Supported TiDB versions: v4.0.6 and later versions

### Incremental log replication - TiDB Binlog

[TiDB Binlog](/tidb-binlog/tidb-binlog-overview.md) is a tool that collects binlog for TiDB clusters and provides nearly real-time data replication and backup. You can use it for incremental data replication between TiDB clusters, such as making a TiDB cluster the secondary cluster of the primary TiDB cluster.

> **Warning:**
>
> Starting from v7.5.0, [TiDB Binlog](/tidb-binlog/tidb-binlog-overview.md) replication is deprecated. Starting from v8.3.0, TiDB Binlog is fully deprecated, with removal planned for a future release. For incremental data replication, use [TiCDC](/ticdc/ticdc-overview.md) instead. For point-in-time recovery (PITR), use [PITR](/br/br-pitr-guide.md).

The following are the basics of TiDB Binlog:

- Source: TiDB clusters
- Target: TiDB clusters, MySQL, Kafka, or incremental backup files
- Supported TiDB versions: v2.1 and later versions
- Kubernetes support: Yes. See [TiDB Binlog Cluster Operations](https://docs.pingcap.com/tidb-in-kubernetes/stable/deploy-tidb-binlog) and [TiDB Binlog Drainer Configurations on Kubernetes](https://docs.pingcap.com/tidb-in-kubernetes/stable/configure-tidb-binlog-drainer) for details.

### sync-diff-inspector

[sync-diff-inspector](/sync-diff-inspector/sync-diff-inspector-overview.md) is a tool that compares data stored in the MySQL or TiDB databases. In addition, you can also use sync-diff-inspector to repair data in the scenario where a small amount of data is inconsistent.
Expand Down
6 changes: 3 additions & 3 deletions faq/backup-and-restore-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,13 +110,13 @@ To address this problem, delete the current task using `br log stop`, and then c

## Feature compatibility issues

### Why does data restored using br command-line tool cannot be replicated to the upstream cluster of TiCDC or Drainer?
### Why does data restored using br command-line tool cannot be replicated to the upstream cluster of TiCDC?

+ **The data restored using BR cannot be replicated to the downstream**. This is because BR directly imports SST files but the downstream cluster currently cannot obtain these files from the upstream.

+ Before v4.0.3, DDL jobs generated during the restore might cause unexpected DDL executions in TiCDC/Drainer. Therefore, if you need to perform restore on the upstream cluster of TiCDC/Drainer, add all tables restored using br command-line tool to the TiCDC/Drainer block list.
+ Before v4.0.3, DDL jobs generated during the restore might cause unexpected DDL executions in TiCDC. Therefore, if you need to perform restore on the upstream cluster of TiCDC, add all tables restored using br command-line tool to the TiCDC block list.

You can use [`filter.rules`](https://github.com/pingcap/tiflow/blob/7c3c2336f98153326912f3cf6ea2fbb7bcc4a20c/cmd/changefeed.toml#L16) to configure the block list for TiCDC and use [`syncer.ignore-table`](/tidb-binlog/tidb-binlog-configuration-file.md#ignore-table) to configure the block list for Drainer.
You can use [`filter.rules`](https://github.com/pingcap/tiflow/blob/7c3c2336f98153326912f3cf6ea2fbb7bcc4a20c/cmd/changefeed.toml#L16) to configure the block list for TiCDC.

### Why is `new_collation_enabled` mismatch reported during restore?

Expand Down
4 changes: 1 addition & 3 deletions faq/deploy-and-maintain-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ If the resources are adequate, it is recommended to use RAID 10 for SSD. If the

### What's the recommended configuration of TiDB components?

- TiDB has a high requirement on CPU and memory. If you need to enable TiDB Binlog (deprecated), the local disk space should be increased based on the service volume estimation and the time requirement for the GC operation. But the SSD disk is not a must.
- TiDB has a high requirement on CPU and memory.
- PD stores the cluster metadata and has frequent Read and Write requests. It demands a high I/O disk. A disk of low performance will affect the performance of the whole cluster. It is recommended to use SSD disks. In addition, a larger number of Regions has a higher requirement on CPU and memory.
- TiKV has a high requirement on CPU, memory and disk. It is required to use SSD.

Expand Down Expand Up @@ -70,8 +70,6 @@ Check the time difference between the machine time of the monitor and the time w
| `enable_ntpd` | to monitor the NTP service of the managed node, True by default; do not close it |
| `machine_benchmark` | to monitor the disk IOPS of the managed node, True by default; do not close it |
| `set_hostname` | to edit the hostname of the managed node based on the IP, False by default |
| `enable_binlog` | whether to deploy Pump and enable the binlog, False by default, dependent on the Kafka cluster; see the `zookeeper_addrs` variable |
| `zookeeper_addrs` | the ZooKeeper address of the binlog Kafka cluster |
| `enable_slow_query_log` | to record the slow query log of TiDB into a single file: ({{ deploy_dir }}/log/tidb_slow_query.log). False by default, to record it into the TiDB log |
| `deploy_without_tidb` | the Key-Value mode, deploy only PD, TiKV and the monitoring service, not TiDB; set the IP of the tidb_servers host group to null in the `inventory.ini` file |

Expand Down
1 change: 0 additions & 1 deletion faq/faq-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ This document summarizes frequently asked questions (FAQs) about TiDB.
<li>Incremental data replication
<ul>
<li><a href="https://docs.pingcap.com/tidb/dev/ticdc-faq">TiCDC FAQs</a></li>
<li><a href="https://docs.pingcap.com/tidb/dev/tidb-binlog-faq">TiDB Binlog FAQs</a></li>
</ul>
</li>
</ul>
Expand Down
Loading
Loading