Skip to content

Commit d41c3b8

Browse files
authored
cloud: update doc for premium changefeed (#21964)
1 parent a8fbb60 commit d41c3b8

8 files changed

+339
-23
lines changed

TOC-tidb-cloud-premium.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -221,6 +221,13 @@
221221
- [CSV Configurations for Importing Data](/tidb-cloud/csv-config-for-import-data.md)
222222
- [Troubleshoot Access Denied Errors during Data Import from Amazon S3](/tidb-cloud/troubleshoot-import-access-denied-error.md)
223223
- [Connect AWS DMS to TiDB Cloud clusters](/tidb-cloud/tidb-cloud-connect-aws-dms.md)
224+
- Stream Data
225+
- [Changefeed Overview](/tidb-cloud/changefeed-overview.md)
226+
- [To MySQL Sink](/tidb-cloud/changefeed-sink-to-mysql.md)
227+
- [To Kafka Sink](/tidb-cloud/changefeed-sink-to-apache-kafka.md)
228+
- Reference
229+
- [Set Up Self-Hosted Kafka Private Link Service in AWS](/tidb-cloud/setup-aws-self-hosted-kafka-private-link-service.md)
230+
- [Set Up Private Endpoint for Changefeeds](/tidb-cloud/premium/set-up-sink-private-endpoint-premium.md)
224231
- Security
225232
- [Security Overview](/tidb-cloud/security-overview.md)
226233
- Identity Access Control
@@ -243,6 +250,7 @@
243250
- [Credits](/tidb-cloud/tidb-cloud-billing.md#credits)
244251
- [Payment Method Setting](/tidb-cloud/tidb-cloud-billing.md#payment-method)
245252
- [Billing from Cloud Provider Marketplace](/tidb-cloud/tidb-cloud-billing.md#billing-from-cloud-provider-marketplace)
253+
- [Billing for Changefeed](/tidb-cloud/premium/tidb-cloud-billing-ticdc-ccu.md)
246254
- [Manage Budgets](/tidb-cloud/tidb-cloud-budget.md)
247255
- Integrations
248256
- [Airbyte](/tidb-cloud/integrate-tidbcloud-with-airbyte.md)

tidb-cloud/changefeed-overview.md

Lines changed: 32 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,21 +9,21 @@ TiDB Cloud changefeed helps you stream data from TiDB Cloud to other data servic
99

1010
> **Note:**
1111
>
12-
> - Currently, TiDB Cloud only allows up to 100 changefeeds per cluster.
12+
> - Currently, TiDB Cloud only allows up to 100 changefeeds per <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>.
1313
> - Currently, TiDB Cloud only allows up to 100 table filter rules per changefeed.
1414
> - For [{{{ .starter }}}](/tidb-cloud/select-cluster-tier.md#starter) and [{{{ .essential }}}](/tidb-cloud/select-cluster-tier.md#essential) clusters, the changefeed feature is unavailable.
1515
1616
## View the Changefeed page
1717

1818
To access the changefeed feature, take the following steps:
1919

20-
1. In the [TiDB Cloud console](https://tidbcloud.com), navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project.
20+
1. In the [TiDB Cloud console](https://tidbcloud.com), <CustomContent plan="dedicated">navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project.</CustomContent><CustomContent plan="premium">navigate to the [**TiDB Instances**](https://tidbcloud.com/tidbs) page.</CustomContent>
2121

2222
> **Tip:**
2323
>
2424
> You can use the combo box in the upper-left corner to switch between organizations, projects, and clusters.
2525
26-
2. Click the name of your target cluster to go to its overview page, and then click **Data** > **Changefeed** in the left navigation pane. The changefeed page is displayed.
26+
2. Click the name of your target <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent> to go to its overview page, and then click **Data** > **Changefeed** in the left navigation pane. The changefeed page is displayed.
2727

2828
On the **Changefeed** page, you can create a changefeed, view a list of existing changefeeds, and operate the existing changefeeds (such as scaling, pausing, resuming, editing, and deleting a changefeed).
2929

@@ -36,22 +36,46 @@ To create a changefeed, refer to the tutorials:
3636
- [Sink to TiDB Cloud](/tidb-cloud/changefeed-sink-to-tidb-cloud.md)
3737
- [Sink to cloud storage](/tidb-cloud/changefeed-sink-to-cloud-storage.md)
3838

39-
## Query Changefeed RCUs
39+
## Query changefeed capacity
40+
41+
<CustomContent plan="dedicated">
42+
43+
For TiDB Cloud Dedicated, you can query the TiCDC Replication Capacity Units (RCUs) of a changefeed.
4044

4145
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
4246
2. Locate the corresponding changefeed you want to query, and click **...** > **View** in the **Action** column.
4347
3. You can see the current TiCDC Replication Capacity Units (RCUs) in the **Specification** area of the page.
4448

49+
</CustomContent>
50+
<CustomContent plan="premium">
51+
52+
For {{{ .premium }}}, you can query the TiCDC Changefeed Capacity Units (CCUs) of a changefeed.
53+
54+
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB instance.
55+
2. Locate the corresponding changefeed you want to query, and click **...** > **View** in the **Action** column.
56+
3. You can see the current TiCDC Changefeed Capacity Units (CCUs) in the **Specification** area of the page.
57+
58+
</CustomContent>
59+
4560
## Scale a changefeed
4661

62+
<CustomContent plan="dedicated">
63+
4764
You can change the TiCDC Replication Capacity Units (RCUs) of a changefeed by scaling up or down the changfeed.
4865

4966
> **Note:**
5067
>
5168
> - To scale a changefeed for a cluster, make sure that all changefeeds for this cluster are created after March 28, 2023.
5269
> - If a cluster has changefeeds created before March 28, 2023, neither the existing changefeeds nor newly created changefeeds for this cluster support scaling up or down.
5370
54-
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
71+
</CustomContent>
72+
<CustomContent plan="premium">
73+
74+
You can change the TiCDC Changefeed Capacity Units (CCUs) of a changefeed by scaling up or down the changfeed.
75+
76+
</CustomContent>
77+
78+
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>.
5579
2. Locate the corresponding changefeed you want to scale, and click **...** > **Scale Up/Down** in the **Action** column.
5680
3. Select a new specification.
5781
4. Click **Submit**.
@@ -60,7 +84,7 @@ It takes about 10 minutes to complete the scaling process (during which the chan
6084

6185
## Pause or resume a changefeed
6286

63-
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
87+
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>.
6488
2. Locate the corresponding changefeed you want to pause or resume, and click **...** > **Pause/Resume** in the **Action** column.
6589

6690
## Edit a changefeed
@@ -69,7 +93,7 @@ It takes about 10 minutes to complete the scaling process (during which the chan
6993
>
7094
> TiDB Cloud currently only allows editing changefeeds in the paused status.
7195
72-
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
96+
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>.
7397
2. Locate the changefeed you want to pause, and click **...** > **Pause** in the **Action** column.
7498
3. When the changefeed status changes to `Paused`, click **...** > **Edit** to edit the corresponding changefeed.
7599

@@ -84,7 +108,7 @@ It takes about 10 minutes to complete the scaling process (during which the chan
84108

85109
## Delete a changefeed
86110

87-
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
111+
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>.
88112
2. Locate the corresponding changefeed you want to delete, and click **...** > **Delete** in the **Action** column.
89113

90114
## Changefeed billing

tidb-cloud/changefeed-sink-to-apache-kafka.md

Lines changed: 58 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,17 +7,31 @@ summary: This document explains how to create a changefeed to stream data from T
77

88
This document describes how to create a changefeed to stream data from TiDB Cloud to Apache Kafka.
99

10+
<CustomContent plan="dedicated">
11+
1012
> **Note:**
1113
>
1214
> - To use the changefeed feature, make sure that your TiDB Cloud Dedicated cluster version is v6.1.3 or later.
1315
> - For [{{{ .starter }}}](/tidb-cloud/select-cluster-tier.md#starter) and [{{{ .essential }}}](/tidb-cloud/select-cluster-tier.md#essential) clusters, the changefeed feature is unavailable.
1416
17+
</CustomContent>
18+
<CustomContent plan="premium">
19+
20+
> **Note:**
21+
>
22+
> For [{{{ .starter }}}](/tidb-cloud/select-cluster-tier.md#starter) and [{{{ .essential }}}](/tidb-cloud/select-cluster-tier.md#essential) clusters, the changefeed feature is unavailable.
23+
24+
</CustomContent>
25+
1526
## Restrictions
1627

17-
- For each TiDB Cloud cluster, you can create up to 100 changefeeds.
28+
- For each TiDB Cloud <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>, you can create up to 100 changefeeds.
1829
- Currently, TiDB Cloud does not support uploading self-signed TLS certificates to connect to Kafka brokers.
1930
- Because TiDB Cloud uses TiCDC to establish changefeeds, it has the same [restrictions as TiCDC](https://docs.pingcap.com/tidb/stable/ticdc-overview#unsupported-scenarios).
2031
- If the table to be replicated does not have a primary key or a non-null unique index, the absence of a unique constraint during replication could result in duplicated data being inserted downstream in some retry scenarios.
32+
33+
<CustomContent plan="dedicated">
34+
2135
- If you choose Private Link or Private Service Connect as the network connectivity method, ensure that your TiDB cluster version meets the following requirements:
2236

2337
- For v6.5.x: version v6.5.9 or later
@@ -30,6 +44,8 @@ This document describes how to create a changefeed to stream data from TiDB Clou
3044
- If you want to distribute changelogs by primary key or index value to Kafka partition with a specified index name, make sure the version of your TiDB cluster is v7.5.0 or later.
3145
- If you want to distribute changelogs by column value to Kafka partition, make sure the version of your TiDB cluster is v7.5.0 or later.
3246

47+
</CustomContent>
48+
3349
## Prerequisites
3450

3551
Before creating a changefeed to stream data to Apache Kafka, you need to complete the following prerequisites:
@@ -39,12 +55,14 @@ Before creating a changefeed to stream data to Apache Kafka, you need to complet
3955

4056
### Network
4157

42-
Ensure that your TiDB cluster can connect to the Apache Kafka service. You can choose one of the following connection methods:
58+
Ensure that your TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent> can connect to the Apache Kafka service. You can choose one of the following connection methods:
4359

4460
- Private Connect: ideal for avoiding VPC CIDR conflicts and meeting security compliance, but incurs additional [Private Data Link Cost](/tidb-cloud/tidb-cloud-billing-ticdc-rcu.md#private-data-link-cost).
4561
- VPC Peering: suitable as a cost-effective option, but requires managing potential VPC CIDR conflicts and security considerations.
4662
- Public IP: suitable for a quick setup.
4763

64+
<CustomContent plan="dedicated">
65+
4866
<SimpleTab>
4967
<div label="Private Connect">
5068

@@ -87,6 +105,35 @@ It is **NOT** recommended to use Public IP in a production environment.
87105

88106
</div>
89107
</SimpleTab>
108+
</CustomContent>
109+
110+
<CustomContent plan="premium">
111+
112+
<SimpleTab>
113+
<div label="Private Connect">
114+
115+
Private Connect leverages **Private Link** or **Private Service Connect** technologies from cloud providers to enable resources in your VPC to connect to services in other VPCs using private IP addresses, as if those services were hosted directly within your VPC.
116+
117+
TiDB Cloud currently supports Private Connect only for self-hosted Kafka. It does not support direct integration with MSK, Confluent Kafka, or other Kafka SaaS services. To connect to these Kafka SaaS services via Private Connect, you can deploy a [kafka-proxy](https://github.com/grepplabs/kafka-proxy) as an intermediary, effectively exposing the Kafka service as self-hosted Kafka.
118+
119+
If your Apache Kafka service is hosted on AWS, follow [Set Up Self-Hosted Kafka Private Link Service in AWS](/tidb-cloud/setup-aws-self-hosted-kafka-private-link-service.md) to configure the network connection and obtain the **Bootstrap Ports** information, and then follow [Set Up Private Endpoint for Changefeeds](/tidb-cloud/premium/set-up-sink-private-endpoint-premium.md) to create a private endpoint.
120+
121+
</div>
122+
<div label="Public IP">
123+
124+
If you want to provide Public IP access to your Apache Kafka service, assign Public IP addresses to all your Kafka brokers.
125+
126+
It is **NOT** recommended to use Public IP in a production environment.
127+
128+
</div>
129+
130+
<div label="VPC Peering">
131+
132+
Currently, the VPC Peering feature for {{{ .premium }}} instances is only available upon request. To request this feature, click **?** in the lower-right corner of the [TiDB Cloud console](https://tidbcloud.com) and click **Request Support**. Then, fill in "Apply for VPC Peering for {{{ .premium }}} instance" in the **Description** field and click **Submit**.
133+
134+
</div>
135+
</SimpleTab>
136+
</CustomContent>
90137

91138
### Kafka ACL authorization
92139

@@ -100,7 +147,7 @@ For example, if your Kafka cluster is in Confluent Cloud, you can see [Resources
100147
## Step 1. Open the Changefeed page for Apache Kafka
101148

102149
1. Log in to the [TiDB Cloud console](https://tidbcloud.com).
103-
2. Navigate to the cluster overview page of the target TiDB cluster, and then click **Data** > **Changefeed** in the left navigation pane.
150+
2. Navigate to the overview page of the target TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>, and then click **Data** > **Changefeed** in the left navigation pane.
104151
3. Click **Create Changefeed**, and select **Kafka** as **Destination**.
105152

106153
## Step 2. Configure the changefeed target
@@ -140,6 +187,8 @@ The steps vary depending on the connectivity method you select.
140187
11. Return to the [TiDB Cloud console](https://tidbcloud.com) to confirm that you have accepted the connection request. TiDB Cloud will test the connection and proceed to the next page if the test succeeds.
141188

142189
</div>
190+
191+
<CustomContent plan="dedicated">
143192
<div label="Private Service Connect (Google Cloud)">
144193

145194
1. In **Connectivity Method**, select **Private Service Connect**.
@@ -158,6 +207,9 @@ The steps vary depending on the connectivity method you select.
158207
11. Return to the [TiDB Cloud console](https://tidbcloud.com) to confirm that you have accepted the connection request. TiDB Cloud will test the connection and proceed to the next page if the test succeeds.
159208

160209
</div>
210+
</CustomContent>
211+
212+
<CustomContent plan="dedicated">
161213
<div label="Private Link (Azure)">
162214

163215
1. In **Connectivity Method**, select **Private Link**.
@@ -176,6 +228,7 @@ The steps vary depending on the connectivity method you select.
176228
11. Return to the [TiDB Cloud console](https://tidbcloud.com) to confirm that you have accepted the connection request. TiDB Cloud will test the connection and proceed to the next page if the test succeeds.
177229

178230
</div>
231+
</CustomContent>
179232
</SimpleTab>
180233

181234
## Step 3. Set the changefeed
@@ -219,7 +272,7 @@ The steps vary depending on the connectivity method you select.
219272
6. If you select **Avro** as your data format, you will see some Avro-specific configurations on the page. You can fill in these configurations as follows:
220273

221274
- In the **Decimal** and **Unsigned BigInt** configurations, specify how TiDB Cloud handles the decimal and unsigned bigint data types in Kafka messages.
222-
- In the **Schema Registry** area, fill in your schema registry endpoint. If you enable **HTTP Authentication**, the fields for user name and password are displayed and automatically filled in with your TiDB cluster endpoint and password.
275+
- In the **Schema Registry** area, fill in your schema registry endpoint. If you enable **HTTP Authentication**, the fields for user name and password are displayed and automatically filled in with your TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent> endpoint and password.
223276

224277
7. In the **Topic Distribution** area, select a distribution mode, and then fill in the topic name configurations according to the mode.
225278

@@ -272,7 +325,7 @@ The steps vary depending on the connectivity method you select.
272325

273326
## Step 4. Configure your changefeed specification
274327

275-
1. In the **Changefeed Specification** area, specify the number of Replication Capacity Units (RCUs) to be used by the changefeed.
328+
1. In the **Changefeed Specification** area, specify the number of <CustomContent plan="dedicated">Replication Capacity Units (RCUs)</CustomContent><CustomContent plan="premium">Changefeed Capacity Units (CCUs)</CustomContent> to be used by the changefeed.
276329
2. In the **Changefeed Name** area, specify a name for the changefeed.
277330
3. Click **Next** to check the configurations you set and go to the next page.
278331

0 commit comments

Comments
 (0)