From 49265dd9b3821670bdfc32724f7c7fdb089bf48e Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Tue, 22 Jul 2025 16:27:56 +0300 Subject: [PATCH 1/3] remove WAL warnings of it being in beta - add icons to links in several places - update ToC name to remove tech preview mention --- .../documentation/docs/architecture/index.md | 15 ++++++--------- .../docs/command-line-tools/pg-waldump.md | 3 --- contrib/pg_tde/documentation/docs/faq.md | 5 +---- contrib/pg_tde/documentation/docs/functions.md | 2 -- .../docs/how-to/multi-tenant-setup.md | 4 ++-- .../documentation/docs/index/tde-limitations.md | 5 +---- .../documentation/docs/release-notes/beta2.md | 2 +- contrib/pg_tde/documentation/docs/test.md | 2 +- contrib/pg_tde/documentation/docs/variables.md | 3 --- .../pg_tde/documentation/docs/wal-encryption.md | 9 ++++----- contrib/pg_tde/documentation/mkdocs.yml | 2 +- 11 files changed, 17 insertions(+), 35 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/architecture/index.md b/contrib/pg_tde/documentation/docs/architecture/index.md index 76bc66e3c9ae4..ae42285564fbf 100644 --- a/contrib/pg_tde/documentation/docs/architecture/index.md +++ b/contrib/pg_tde/documentation/docs/architecture/index.md @@ -22,7 +22,7 @@ The following sections break down the key architectural components of this desig * Temporary tables * Write Ahead Log (WAL), still in beta. **Do not enable this feature in production environments**. -**Extension** means that `pg_tde` should be implemented only as an extension, possibly compatible with any PostgreSQL distribution, including the open source community version. This requires changes in the PostgreSQL core to make it more extensible. Therefore, `pg_tde` currently works only with the [Percona Server for PostgreSQL](https://docs.percona.com/postgresql/17/index.html) - a binary replacement of community PostgreSQL and included in Percona Distribution for PostgreSQL. +**Extension** means that `pg_tde` should be implemented only as an extension, possibly compatible with any PostgreSQL distribution, including the open source community version. This requires changes in the PostgreSQL core to make it more extensible. Therefore, `pg_tde` currently works only with the [Percona Server for PostgreSQL :octicons-link-external-16:](https://docs.percona.com/postgresql/17/index.html) - a binary replacement of community PostgreSQL and included in Percona Distribution for PostgreSQL. ## Main components @@ -30,7 +30,7 @@ The main components of `pg_tde` are the following: * **Core server changes** focus on making the server more extensible, allowing the main logic of `pg_tde` to remain separate, as an extension. Core changes also add encryption-awareness to some command line tools that have to work directly with encrypted tables or encrypted WAL files. - [Percona Server for PostgreSQL location](https://github.com/percona/postgres/tree/{{tdebranch}}) + [Percona Server for PostgreSQL location :octicons-link-external-16:](https://github.com/percona/postgres/tree/{{tdebranch}}) * The **`pg_tde` extension itself** implements the encryption code by hooking into the extension points introduced in the core changes, and the already existing extension points in the PostgreSQL server. @@ -50,7 +50,7 @@ In the future these could be extracted into separate shared libraries with an op `pg_tde` uses one principal key per database. Every internal key for the given database is encrypted using this principal key. -Internal keys are used for specific database files: each file with a different [Object Identifier (OID)](https://www.postgresql.org/docs/current/datatype-oid.html) has a different internal key. +Internal keys are used for specific database files: each file with a different [Object Identifier (OID) :octicons-link-external-16:](https://www.postgresql.org/docs/current/datatype-oid.html) has a different internal key. This means that, for example, a table with 4 indexes will have at least 5 internal keys - one for the table, and one for each index. @@ -82,12 +82,9 @@ Later decisions are made using a slightly modified Storage Manager (SMGR) API: w ### WAL encryption -!!! note - The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. - WAL encryption is controlled globally via a global GUC variable, `pg_tde.wal_encrypt`, that requires a server restart. -WAL keys also contain the [LSN](https://www.postgresql.org/docs/17/wal-internals.html) of the first WAL write after key creation. This allows `pg_tde` to know which WAL ranges are encrypted or not and with which key. +WAL keys also contain the [LSN :octicons-link-external-16:](https://www.postgresql.org/docs/17/wal-internals.html) of the first WAL write after key creation. This allows `pg_tde` to know which WAL ranges are encrypted or not and with which key. The setting only controls writes so that only WAL writes are encrypted when WAL encryption is enabled. This means that WAL files can contain both encrypted and unencrypted data, depending on what the status of this variable was when writing the data. @@ -147,8 +144,8 @@ Principal keys are stored externally in a Key Management Services (KMS). In `pg_ The following key providers are supported: -* [HashiCorp Vault](https://developer.hashicorp.com/vault/docs/what-is-vault) KV2 secrets engine -* [OpenBao](https://openbao.org/) implementation of Vault +* [HashiCorp Vault :octicons-link-external-16:](https://developer.hashicorp.com/vault/docs/what-is-vault) KV2 secrets engine +* [OpenBao :octicons-link-external-16:](https://openbao.org/) implementation of Vault * KMIP compatible servers * A local file storage. This storage is intended only for development and testing and is not recommended for production use. diff --git a/contrib/pg_tde/documentation/docs/command-line-tools/pg-waldump.md b/contrib/pg_tde/documentation/docs/command-line-tools/pg-waldump.md index 0362eadcef7f0..a3ac50df4df02 100644 --- a/contrib/pg_tde/documentation/docs/command-line-tools/pg-waldump.md +++ b/contrib/pg_tde/documentation/docs/command-line-tools/pg-waldump.md @@ -2,9 +2,6 @@ [`pg_waldump` :octicons-link-external-16:](https://www.postgresql.org/docs/current/pgwaldump.html) is a tool to display a human-readable rendering of the Write-Ahead Log (WAL) of a PostgreSQL database cluster. -!!! warning - The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. - To read encrypted WAL records, `pg_waldump` supports the following additional arguments: * `keyring_path` is the directory where the keyring configuration files for WAL are stored. The following files are included: diff --git a/contrib/pg_tde/documentation/docs/faq.md b/contrib/pg_tde/documentation/docs/faq.md index 36430b722a1c4..7b8843c0ce2a0 100644 --- a/contrib/pg_tde/documentation/docs/faq.md +++ b/contrib/pg_tde/documentation/docs/faq.md @@ -94,9 +94,6 @@ The principal key is used to encrypt the internal keys. The principal key is sto ### WAL encryption -!!! note - WAL encryption is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. - WAL encryption is done globally for the entire database cluster. All modifications to any database within a PostgreSQL cluster are written to the same WAL to maintain data consistency and integrity and ensure that PostgreSQL cluster can be restored to a consistent state. Therefore, WAL is encrypted globally. When you turn on WAL encryption, `pg_tde` encrypts entire WAL files starting from the first WAL write after the server was started with the encryption turned on. @@ -140,7 +137,7 @@ Since the `SET ACCESS METHOD` command drops hint bits and this may affect the pe You must restart the database in the following cases to apply the changes: * after you enabled the `pg_tde` extension -* when enabling WAL encryption, which is currently in beta. **Do not enable this feature in production environments**. +* when enabling WAL encryption After that, no database restart is required. When you create or alter the table using the `tde_heap` access method, the files are marked as those that require encryption. The encryption happens at the storage manager level, before a transaction is written to disk. Read more about [how tde_heap works](index/table-access-method.md#how-tde_heap-works). diff --git a/contrib/pg_tde/documentation/docs/functions.md b/contrib/pg_tde/documentation/docs/functions.md index b5c51ec07cdf0..c18a570a9eea7 100644 --- a/contrib/pg_tde/documentation/docs/functions.md +++ b/contrib/pg_tde/documentation/docs/functions.md @@ -280,8 +280,6 @@ SELECT pg_tde_set_server_key_using_global_key_provider( ); ``` -!!! warning - The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. ======= The `ensure_new_key` parameter instructs the function how to handle a principal key during key rotation: diff --git a/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md b/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md index c8eaac1afcfe8..703461c74b3d5 100644 --- a/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md +++ b/contrib/pg_tde/documentation/docs/how-to/multi-tenant-setup.md @@ -32,7 +32,7 @@ Load the `pg_tde` at startup time. The extension requires additional shared memo sudo systemctl restart postgresql-17 ``` -3. Create the extension using the [CREATE EXTENSION](https://www.postgresql.org/docs/current/sql-createextension.html) command. You must have the privileges of a superuser or a database owner to use this command. Connect to `psql` as a superuser for a database and run the following command: +3. Create the extension using the [CREATE EXTENSION :octicons-link-external-16:](https://www.postgresql.org/docs/current/sql-createextension.html) command. You must have the privileges of a superuser or a database owner to use this command. Connect to `psql` as a superuser for a database and run the following command: ``` CREATE EXTENSION pg_tde; @@ -58,7 +58,7 @@ You must do these steps for every database where you have created the extension. The KMIP server setup is out of scope of this document. - Make sure you have obtained the root certificate for the KMIP server and the keypair for the client. The client key needs permissions to create / read keys on the server. Find the [configuration guidelines for the HashiCorp Vault Enterprise KMIP Secrets Engine](https://developer.hashicorp.com/vault/tutorials/enterprise/kmip-engine). + Make sure you have obtained the root certificate for the KMIP server and the keypair for the client. The client key needs permissions to create / read keys on the server. Find the [configuration guidelines for the HashiCorp Vault Enterprise KMIP Secrets Engine :octicons-link-external-16:](https://developer.hashicorp.com/vault/tutorials/enterprise/kmip-engine). For testing purposes, you can use the PyKMIP server which enables you to set up required certificates. To use a real KMIP server, make sure to obtain the valid certificates issued by the key management appliance. diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index 271665a0f17be..f9a0b26bd229d 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -2,10 +2,7 @@ * Keys in the local keyfile are stored **unencrypted**. For better security we recommend using the [Key management storage](../global-key-provider-configuration/index.md). * System tables, which include statistics data and database statistics, are currently **not encrypted**. -* The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. +* The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. * `pg_tde` RC 2 is incompatible with `pg_tde` Beta2 due to significant changes in code. There is no direct upgrade flow from one version to another. You must [uninstall](../how-to/uninstall.md) `pg_tde` Beta2 first and then [install](../install.md) and configure the new Release Candidate version. -!!! important - This is the {{release}} version of the extension and **it is not meant for production use yet**. We encourage you to use it in testing environments and [provide your feedback](https://forums.percona.com/c/postgresql/pg-tde-transparent-data-encryption-tde/82). - [Versions and Supported PostgreSQL Deployments](supported-versions.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/release-notes/beta2.md b/contrib/pg_tde/documentation/docs/release-notes/beta2.md index f134f6874b772..1680c9f58a4ad 100644 --- a/contrib/pg_tde/documentation/docs/release-notes/beta2.md +++ b/contrib/pg_tde/documentation/docs/release-notes/beta2.md @@ -7,7 +7,7 @@ !!! important This version of Percona Transparent Data Encryption extension **is - not recommended for production environments yet**. We encourage you to test it and [give your feedback](https://forums.percona.com/c/postgresql/pg-tde-transparent-data-encryption-tde/82). + not recommended for production environments yet**. We encourage you to test it and [give your feedback :octicons-link-external-16:](https://forums.percona.com/c/postgresql/pg-tde-transparent-data-encryption-tde/82). This will help us improve the product and make it production-ready faster. diff --git a/contrib/pg_tde/documentation/docs/test.md b/contrib/pg_tde/documentation/docs/test.md index be73e73863e76..2da3b4b33f17a 100644 --- a/contrib/pg_tde/documentation/docs/test.md +++ b/contrib/pg_tde/documentation/docs/test.md @@ -59,4 +59,4 @@ ALTER TABLE table_name SET ACCESS METHOD tde_heap; ## Next steps -[Configure WAL Encryption (tech preview) :material-arrow-right:](wal-encryption.md){.md-button} +[Configure WAL Encryption :material-arrow-right:](wal-encryption.md){.md-button} diff --git a/contrib/pg_tde/documentation/docs/variables.md b/contrib/pg_tde/documentation/docs/variables.md index 74bdd62138193..e0acbf03131f0 100644 --- a/contrib/pg_tde/documentation/docs/variables.md +++ b/contrib/pg_tde/documentation/docs/variables.md @@ -19,9 +19,6 @@ This variable only controls new writes to the WAL, it doesn't affect existing WA Enabling WAL encryption requires a configured global principal key. Refer to the [WAL encryption configuration](wal-encryption.md) topic for more information. -!!! warning - The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. - ## pg_tde.enforce_encryption **Type** - boolean
diff --git a/contrib/pg_tde/documentation/docs/wal-encryption.md b/contrib/pg_tde/documentation/docs/wal-encryption.md index 68cfa9c65e864..730f3531c352c 100644 --- a/contrib/pg_tde/documentation/docs/wal-encryption.md +++ b/contrib/pg_tde/documentation/docs/wal-encryption.md @@ -1,7 +1,4 @@ -# Configure WAL Encryption (tech preview) - -!!! warning - The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. +# Configure WAL Encryption Before enabling WAL encryption, follow the steps below to create a principal key and configure it for WAL: @@ -15,7 +12,7 @@ Before enabling WAL encryption, follow the steps below to create a principal key === "With KMIP server" - Make sure you have obtained the root certificate for the KMIP server and the keypair for the client. The client key needs permissions to create / read keys on the server. Find the [configuration guidelines for the HashiCorp Vault Enterprise KMIP Secrets Engine](https://developer.hashicorp.com/vault/tutorials/enterprise/kmip-engine). + Make sure you have obtained the root certificate for the KMIP server and the keypair for the client. The client key needs permissions to create / read keys on the server. Find the [configuration guidelines for the HashiCorp Vault Enterprise KMIP Secrets Engine :octicons-link-external-16:](https://developer.hashicorp.com/vault/tutorials/enterprise/kmip-engine). For testing purposes, you can use the PyKMIP server which enables you to set up required certificates. To use a real KMIP server, make sure to obtain the valid certificates issued by the key management appliance. @@ -118,3 +115,5 @@ Now WAL files start to be encrypted for both encrypted and unencrypted tables. For more technical references related to architecture, variables or functions, see: [Technical Reference](advanced-topics/index.md){.md-button} + +💬 Need help customizing this for your infrastructure? [Contact Percona support :octicons-link-external-16:](get-help.md) diff --git a/contrib/pg_tde/documentation/mkdocs.yml b/contrib/pg_tde/documentation/mkdocs.yml index aa490d2dd0ada..c3f3d4f6841bd 100644 --- a/contrib/pg_tde/documentation/mkdocs.yml +++ b/contrib/pg_tde/documentation/mkdocs.yml @@ -185,7 +185,7 @@ nav: - "Keyring File Configuration": global-key-provider-configuration/keyring.md - "2.2 Global Principal Key Configuration": global-key-provider-configuration/set-principal-key.md - "3. Validate Encryption with pg_tde": test.md - - "4. Configure WAL Encryption (tech preview)": wal-encryption.md + - "4. Configure WAL Encryption": wal-encryption.md - "Technical Reference": - "Overview": advanced-topics/index.md - "Architecture": architecture/index.md From 708f226f404dbadb86bb2b9e882e3c185344b94e Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Tue, 5 Aug 2025 11:43:00 +0300 Subject: [PATCH 2/3] fix-conflict-issues --- contrib/pg_tde/documentation/docs/wal-encryption.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/pg_tde/documentation/docs/wal-encryption.md b/contrib/pg_tde/documentation/docs/wal-encryption.md index 730f3531c352c..5064085c4bd18 100644 --- a/contrib/pg_tde/documentation/docs/wal-encryption.md +++ b/contrib/pg_tde/documentation/docs/wal-encryption.md @@ -1,6 +1,6 @@ # Configure WAL Encryption -Before enabling WAL encryption, follow the steps below to create a principal key and configure it for WAL: +Before enabling WAL encryption, follow the steps below to create a principal key and configure it for WAL: 1. Create the `pg_tde` extension if it does not exist: From ddf2872bc8e15370291e6cfee1d98b47ed33b924 Mon Sep 17 00:00:00 2001 From: Dragos Andriciuc Date: Tue, 5 Aug 2025 11:57:59 +0300 Subject: [PATCH 3/3] merge origin to branch to fix conflicts 1/2 --- .../docs/index/tde-limitations.md | 5 +- contrib/pg_tde/documentation/mkdocs.yml | 59 ++++++++++--------- 2 files changed, 34 insertions(+), 30 deletions(-) diff --git a/contrib/pg_tde/documentation/docs/index/tde-limitations.md b/contrib/pg_tde/documentation/docs/index/tde-limitations.md index f9a0b26bd229d..c9f3c68ca59b8 100644 --- a/contrib/pg_tde/documentation/docs/index/tde-limitations.md +++ b/contrib/pg_tde/documentation/docs/index/tde-limitations.md @@ -1,8 +1,9 @@ # Limitations of pg_tde -* Keys in the local keyfile are stored **unencrypted**. For better security we recommend using the [Key management storage](../global-key-provider-configuration/index.md). +The following are current limitations of `pg_tde`: + * System tables, which include statistics data and database statistics, are currently **not encrypted**. -* The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. +* The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**. * `pg_tde` RC 2 is incompatible with `pg_tde` Beta2 due to significant changes in code. There is no direct upgrade flow from one version to another. You must [uninstall](../how-to/uninstall.md) `pg_tde` Beta2 first and then [install](../install.md) and configure the new Release Candidate version. [Versions and Supported PostgreSQL Deployments](supported-versions.md){.md-button} diff --git a/contrib/pg_tde/documentation/mkdocs.yml b/contrib/pg_tde/documentation/mkdocs.yml index c3f3d4f6841bd..43725bb5de5e6 100644 --- a/contrib/pg_tde/documentation/mkdocs.yml +++ b/contrib/pg_tde/documentation/mkdocs.yml @@ -162,13 +162,13 @@ nav: - get-help.md - "Features": features.md - "Overview": - - "What is Transparent Data Encryption (TDE)?": - - "TDE overview": index/index.md - - "TDE benefits": index/how-tde-helps.md - - "How TDE works": index/how-does-tde-work.md + - "About Transparent Data Encryption": index/about-tde.md + - "About pg_tde": + - "Benefits of pg_tde": index/how-tde-helps.md + - "How pg_tde works": index/how-does-tde-work.md - "Encrypted data scope": index/tde-encrypts.md - - "Table access methods and TDE": index/table-access-method.md - - "Limitations of TDE": index/tde-limitations.md + - "Table access methods and pg_tde": index/table-access-method.md + - "Limitations of pg_tde": index/tde-limitations.md - "Versions and supported PostgreSQL deployments": index/supported-versions.md - "Get started": - "1. Install pg_tde": install.md @@ -176,25 +176,25 @@ nav: - "1.2 Via yum": yum.md - "2. Configure pg_tde": setup.md - "2.1 Configure Key Management (KMS)": - - "2.1 Configure Key Management (KMS)": global-key-provider-configuration/index.md - - "KMIP Configuration": global-key-provider-configuration/kmip-server.md - - "Fortanix Configuration": global-key-provider-configuration/kmip-fortanix.md - - "Vault Configuration": global-key-provider-configuration/vault.md - - "Thales Configuration": global-key-provider-configuration/kmip-thales.md - - "Using OpenBao as a Key Provider": global-key-provider-configuration/kmip-openbao.md - - "Keyring File Configuration": global-key-provider-configuration/keyring.md - - "2.2 Global Principal Key Configuration": global-key-provider-configuration/set-principal-key.md - - "3. Validate Encryption with pg_tde": test.md - - "4. Configure WAL Encryption": wal-encryption.md - - "Technical Reference": - - "Overview": advanced-topics/index.md - - "Architecture": architecture/index.md + - "Key management overview": global-key-provider-configuration/overview.md + - "KMIP configuration": global-key-provider-configuration/kmip-server.md + - "Fortanix configuration": global-key-provider-configuration/kmip-fortanix.md + - "Vault configuration": global-key-provider-configuration/vault.md + - "Thales configuration": global-key-provider-configuration/kmip-thales.md + - "Using OpenBao as a key provider": global-key-provider-configuration/kmip-openbao.md + - "Keyring file configuration": global-key-provider-configuration/keyring.md + - "2.2 Global Principal Key configuration": global-key-provider-configuration/set-principal-key.md + - "3. Validate encryption with pg_tde": test.md + - "4. Configure WAL encryption (tech preview)": wal-encryption.md + - "Technical reference": + - "Overview": advanced-topics/tech-reference.md + - "Architecture": architecture/architecture.md - "GUC variables": variables.md - "Functions": functions.md - "Streaming Replication with tde_heap": replication.md - - "TDE Operations": + - "TDE operations": - "pg_tde CLI Tools": - - "CLI Overview": command-line-tools/index.md + - "Overview": command-line-tools/cli-tools.md - "pg_tde_change_key_provider": command-line-tools/pg-tde-change-key-provider.md - "pg_waldump": command-line-tools/pg-waldump.md - "pg_checksums": command-line-tools/pg-tde-checksums.md @@ -206,11 +206,14 @@ nav: - faq.md - "Release notes": - "Release notes index": release-notes/release-notes.md - - release-notes/release-notes-v1.0.md - - release-notes/rc2.md - - release-notes/rc.md - - release-notes/beta2.md - - release-notes/beta.md - - release-notes/alpha1.md - - release-notes/mvp.md + - "2025": + - "1.0": release-notes/release-notes-v1.0.md + - "Release Candidate 2": release-notes/rc2.md + - "Release Candidate 1": release-notes/rc.md + - "2024 (Alpha 1 - Beta 2)": + - "Beta 2": release-notes/beta2.md + - "Beta": release-notes/beta.md + - "Alpha 1": release-notes/alpha1.md + - "2023 (MVP)": + - "MVP": release-notes/mvp.md - contribute.md