Skip to content

Latest commit

 

History

History
72 lines (46 loc) · 3.97 KB

log-redaction.md

File metadata and controls

72 lines (46 loc) · 3.97 KB
Raw
title summary
Log Redaction
Learn the log redaction in TiDB components.

Log Redaction

When TiDB provides detailed log information, it might print sensitive data (for example, user data) in the log, which causes data security risks. To avoid such risks, each component (TiDB, TiKV, and PD) provides a configuration item that enables log redaction to shield user data values.

Log redaction in TiDB side

To enable log redaction in the TiDB side, set the value of global.tidb_redact_log to ON or MARKER. This configuration value defaults to OFF, which means that log redaction is disabled.

You can use the set syntax to set the global variable tidb_redact_log:

set @@global.tidb_redact_log = ON;

After the setting, all logs generated in new sessions are redacted:

create table t (a int, unique key (a));
Query OK, 0 rows affected (0.00 sec)

insert into t values (1),(1);
ERROR 1062 (23000): Duplicate entry '1' for key 't.a'

The error log for the INSERT statement above is printed as follows:

[2024/07/02 11:35:32.686 +08:00] [INFO] [conn.go:1146] ["command dispatched failed"] [conn=1482686470] [session_alias=] [connInfo="id:1482686470, addr:127.0.0.1:52258 status:10, collation:utf8mb4_0900_ai_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="insert into `t` values ( ... )"] [txn_mode=PESSIMISTIC] [timestamp=450859193514065921] [err="[kv:1062]Duplicate entry '?' for key 't.a'"]

From the preceding error log, you can see that when the value of tidb_redact_log is set to ON, sensitive information is replaced by the ? mark in the TiDB log to avoid data security risks.

In addition, TiDB provides the MARKER option. When the value of tidb_redact_log is set to MARKER, TiDB marks sensitive information in the log with ‹› instead of replacing it directly, so you can customize the redaction rules.

set @@global.tidb_redact_log = MARKER;

After the preceding configuration, the sensitive information is marked rather than replaced in all logs generated by new sessions:

create table t (a int, unique key (a));
Query OK, 0 rows affected (0.07 sec)

insert into t values (1),(1);
ERROR 1062 (23000): Duplicate entry '‹1›' for key 't.a'

The error log is as follows:

[2024/07/02 11:35:01.426 +08:00] [INFO] [conn.go:1146] ["command dispatched failed"] [conn=1482686470] [session_alias=] [connInfo="id:1482686470, addr:127.0.0.1:52258 status:10, collation:utf8mb4_0900_ai_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="insert into `t` values ( ‹1› ) , ( ‹1› )"] [txn_mode=PESSIMISTIC] [timestamp=450859185309483010] [err="[kv:1062]Duplicate entry '‹1›' for key 't.a'"]

As you can see from the preceding error log, after you set tidb_redact_log to MARKER, TiDB marks sensitive information using ‹ › in the log. You can customize redaction rules to handle sensitive information in the log as needed.

Log redaction in TiKV side

To enable log redaction in the TiKV side, set the value of security.redact-info-log to true or "marker". This configuration value defaults to false, which means that log redaction is disabled.

Log redaction in PD side

To enable log redaction in the PD side, set the value of security.redact-info-log to true or "marker". This configuration value defaults to false, which means that log redaction is disabled.

Log redaction in TiFlash side

To enable log redaction in the TiFlash side, set both the security.redact_info_log value in tiflash-server and the security.redact-info-log value in tiflash-learner to true or "marker". Both configuration values default to false, which means that log redaction is disabled.