Skip to content

Commit

Permalink
[KYUUBI #6119] [DOC] Update doc for HA/Zookeeper Configuration
Browse files Browse the repository at this point in the history 
# 🔍 Description
## Issue References 🔗

This pull request update doc for HA/Zookeeper Configuration

## Describe Your Solution 🔧

Replace conf name with the new ones and mark olds as deprecated

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6119 from kevinjmh/doc-update.

Closes #6119

49d0fff [Manhua] [DOC] Update doc for HA/Zookeeper Configuration

Authored-by: Manhua <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
  • Loading branch information
@kevinjmh @pan3793
kevinjmh authored and pan3793 committed Mar 4, 2024
1 parent 3bc28fd commit 97d02a4
Show file tree
Hide file tree
Showing 6 changed files with 13 additions and 13 deletions.
6 changes: 3 additions & 3 deletions docs/configuration/settings.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -301,7 +301,7 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
| kyuubi.ha.etcd.ssl.client.key.path | &lt;undefined&gt; | Where the etcd SSL key file is stored. | string | 1.6.0 |
| kyuubi.ha.etcd.ssl.enabled | false | When set to true, will build an SSL secured etcd client. | boolean | 1.6.0 |
| kyuubi.ha.namespace | kyuubi | The root directory for the service to deploy its instance uri | string | 1.6.0 |
| kyuubi.ha.zookeeper.acl.enabled | false | Set to true if the ZooKeeper ensemble is kerberized | boolean | 1.0.0 |
| kyuubi.ha.zookeeper.acl.enabled | false | (deprecated) Set to true if the ZooKeeper ensemble is kerberized | boolean | 1.0.0 |
| kyuubi.ha.zookeeper.auth.digest | &lt;undefined&gt; | The digest auth string is used for ZooKeeper authentication, like: username:password. | string | 1.3.2 |
| kyuubi.ha.zookeeper.auth.keytab | &lt;undefined&gt; | Location of the Kyuubi server's keytab that is used for ZooKeeper authentication. | string | 1.3.2 |
| kyuubi.ha.zookeeper.auth.principal | &lt;undefined&gt; | Kerberos principal name that is used for ZooKeeper authentication. | string | 1.3.2 |
Expand Down Expand Up @@ -512,11 +512,11 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
| kyuubi.zookeeper.embedded.client.use.hostname | false | When true, embedded Zookeeper prefer to bind hostname, otherwise, ip address. | boolean | 1.7.2 |
| kyuubi.zookeeper.embedded.data.dir | embedded_zookeeper | dataDir for the embedded zookeeper server where stores the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database. If it is a relative path, it is resolved relative to KYUUBI_HOME. | string | 1.2.0 |
| kyuubi.zookeeper.embedded.data.log.dir | embedded_zookeeper | dataLogDir for the embedded ZooKeeper server where writes the transaction log. If it is a relative path, it is resolved relative to KYUUBI_HOME. | string | 1.2.0 |
| kyuubi.zookeeper.embedded.directory | embedded_zookeeper | The temporary directory for the embedded ZooKeeper server. If it is a relative path, it is resolved relative to KYUUBI_HOME. | string | 1.0.0 |
| kyuubi.zookeeper.embedded.directory | embedded_zookeeper | (deprecated) The temporary directory for the embedded ZooKeeper server. If it is a relative path, it is resolved relative to KYUUBI_HOME. | string | 1.0.0 |
| kyuubi.zookeeper.embedded.max.client.connections | 120 | maxClientCnxns for the embedded ZooKeeper server to limit the number of concurrent connections of a single client identified by IP address | int | 1.2.0 |
| kyuubi.zookeeper.embedded.max.session.timeout | 60000 | maxSessionTimeout in milliseconds for the embedded ZooKeeper server will allow the client to negotiate. Defaults to 20 times the tickTime | int | 1.2.0 |
| kyuubi.zookeeper.embedded.min.session.timeout | 6000 | minSessionTimeout in milliseconds for the embedded ZooKeeper server will allow the client to negotiate. Defaults to 2 times the tickTime | int | 1.2.0 |
| kyuubi.zookeeper.embedded.port | 2181 | The port of the embedded ZooKeeper server | int | 1.0.0 |
| kyuubi.zookeeper.embedded.port | 2181 | (deprecated) The port of the embedded ZooKeeper server | int | 1.0.0 |
| kyuubi.zookeeper.embedded.tick.time | 3000 | tickTime in milliseconds for the embedded ZooKeeper server | int | 1.2.0 |

## Spark Configurations
Expand Down
2 changes: 1 addition & 1 deletion docs/connector/spark/delta_lake_with_azure_blob.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ Add the following content:
kyuubi.frontend.bind.host <YOUR_HOST>
kyuubi.frontend.bind.port 10009
# If you use your own zk cluster, you need to configure your zk host port.
# kyuubi.ha.zookeeper.quorum <YOUR_HOST>:2181
# kyuubi.ha.addresses <YOUR_HOST>:2181
Start Kyuubi
************
Expand Down
10 changes: 5 additions & 5 deletions docs/deployment/high_availability_guide.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,21 +48,21 @@ Using multiple Kyuubi service units with load balancing instead of a single unit

When applying HA to Kyuubi deployment, we need to be aware of the below two thing basically,

- `kyuubi.ha.zookeeper.quorum` - the external zookeeper cluster address for deploy a `k.i.`
- `kyuubi.ha.zookeeper.namespace` - the root directory, a.k.a. the ServerSpace for deploy a `k.i.`
- `kyuubi.ha.addresses` - the external zookeeper cluster address for deploy a `k.i.`
- `kyuubi.ha.namespace` - the root directory, a.k.a. the ServerSpace for deploy a `k.i.`

For more configurations, please see the HA section of [Introduction to the Kyuubi Configurations System](./settings.html#ha)

### Pseudo mode

When `kyuubi.ha.zookeeper.quorum` is not configured, a `k.i.` will start an embedded zookeeper service and expose the address of itself there.
When `kyuubi.ha.addresses` is not configured, a `k.i.` will start an embedded zookeeper service and expose the address of itself there.
In this pseduo mode, the `k.i.` can be connected by clients through both raw ip address and zk quorum + namespace.
But it doesn't have any availability to being highly available.

### Production mode

For production deployment purpose, an external zookeeper cluster is required for `kyuubi.ha.zookeeper.quorum`.
In this mode, multiple `k.i.`s can be registered to the same ServerSpace configured by `kyuubi.ha.zookeeper.namespace` and serve together.
For production deployment purpose, an external zookeeper cluster is required for `kyuubi.ha.addresses`.
In this mode, multiple `k.i.`s can be registered to the same ServerSpace configured by `kyuubi.ha.namespace` and serve together.

## Client-side Usage

Expand Down
2 changes: 1 addition & 1 deletion docs/tools/kyuubi-ctl.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ Create server
***********
Expose Kyuubi server instance to another domain.

First read ``kyuubi.ha.zookeeper.namespace`` in ``conf/kyuubi-defaults.conf``, if there are server instances under this namespace, register them in the new namespace specified by the ``--namespace`` parameter.
First read ``kyuubi.ha.namespace`` in ``conf/kyuubi-defaults.conf``, if there are server instances under this namespace, register them in the new namespace specified by the ``--namespace`` parameter.

.. code-block:: bash
Expand Down
2 changes: 1 addition & 1 deletion kyuubi-ha/src/main/scala/org/apache/kyuubi/ha/HighAvailabilityConf.scala
View file
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ object HighAvailabilityConf {
"1.3.2")
val HA_ZK_ACL_ENABLED: ConfigEntry[Boolean] =
buildConf("kyuubi.ha.zookeeper.acl.enabled")
.doc("Set to true if the ZooKeeper ensemble is kerberized")
.doc("(deprecated) Set to true if the ZooKeeper ensemble is kerberized")
.version("1.0.0")
.booleanConf
.createWithDefault(UserGroupInformation.isSecurityEnabled)
Expand Down
4 changes: 2 additions & 2 deletions kyuubi-zookeeper/src/main/scala/org/apache/kyuubi/zookeeper/ZookeeperConf.scala
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,14 @@ object ZookeeperConf {

@deprecated("using kyuubi.zookeeper.embedded.client.port instead", since = "1.2.0")
val EMBEDDED_ZK_PORT: ConfigEntry[Int] = buildConf("kyuubi.zookeeper.embedded.port")
.doc("The port of the embedded ZooKeeper server")
.doc("(deprecated) The port of the embedded ZooKeeper server")
.version("1.0.0")
.intConf
.createWithDefault(2181)

@deprecated("using kyuubi.zookeeper.embedded.data.dir instead", since = "1.2.0")
val EMBEDDED_ZK_TEMP_DIR: ConfigEntry[String] = buildConf("kyuubi.zookeeper.embedded.directory")
.doc("The temporary directory for the embedded ZooKeeper server. " +
.doc("(deprecated) The temporary directory for the embedded ZooKeeper server. " +
"If it is a relative path, it is resolved relative to KYUUBI_HOME. ")
.version("1.0.0")
.stringConf
Expand Down

0 comments on commit 97d02a4

Please sign in to comment.