Skip to content

Commit

Permalink
[DOC] Update doc for HA/Zookeeper Configuration
Browse files Browse the repository at this point in the history
  • Loading branch information
kevinjmh committed Mar 1, 2024
1 parent 9f53a09 commit 49d0fff
Show file tree
Hide file tree
Showing 6 changed files with 13 additions and 13 deletions.
6 changes: 3 additions & 3 deletions docs/configuration/settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -301,7 +301,7 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
| kyuubi.ha.etcd.ssl.client.key.path | <undefined> | Where the etcd SSL key file is stored. | string | 1.6.0 |
| kyuubi.ha.etcd.ssl.enabled | false | When set to true, will build an SSL secured etcd client. | boolean | 1.6.0 |
| kyuubi.ha.namespace | kyuubi | The root directory for the service to deploy its instance uri | string | 1.6.0 |
| kyuubi.ha.zookeeper.acl.enabled | false | Set to true if the ZooKeeper ensemble is kerberized | boolean | 1.0.0 |
| kyuubi.ha.zookeeper.acl.enabled | false | (deprecated) Set to true if the ZooKeeper ensemble is kerberized | boolean | 1.0.0 |
| kyuubi.ha.zookeeper.auth.digest | <undefined> | The digest auth string is used for ZooKeeper authentication, like: username:password. | string | 1.3.2 |
| kyuubi.ha.zookeeper.auth.keytab | <undefined> | Location of the Kyuubi server's keytab that is used for ZooKeeper authentication. | string | 1.3.2 |
| kyuubi.ha.zookeeper.auth.principal | <undefined> | Kerberos principal name that is used for ZooKeeper authentication. | string | 1.3.2 |
Expand Down Expand Up @@ -512,11 +512,11 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
| kyuubi.zookeeper.embedded.client.use.hostname | false | When true, embedded Zookeeper prefer to bind hostname, otherwise, ip address. | boolean | 1.7.2 |
| kyuubi.zookeeper.embedded.data.dir | embedded_zookeeper | dataDir for the embedded zookeeper server where stores the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database. If it is a relative path, it is resolved relative to KYUUBI_HOME. | string | 1.2.0 |
| kyuubi.zookeeper.embedded.data.log.dir | embedded_zookeeper | dataLogDir for the embedded ZooKeeper server where writes the transaction log. If it is a relative path, it is resolved relative to KYUUBI_HOME. | string | 1.2.0 |
| kyuubi.zookeeper.embedded.directory | embedded_zookeeper | The temporary directory for the embedded ZooKeeper server. If it is a relative path, it is resolved relative to KYUUBI_HOME. | string | 1.0.0 |
| kyuubi.zookeeper.embedded.directory | embedded_zookeeper | (deprecated) The temporary directory for the embedded ZooKeeper server. If it is a relative path, it is resolved relative to KYUUBI_HOME. | string | 1.0.0 |
| kyuubi.zookeeper.embedded.max.client.connections | 120 | maxClientCnxns for the embedded ZooKeeper server to limit the number of concurrent connections of a single client identified by IP address | int | 1.2.0 |
| kyuubi.zookeeper.embedded.max.session.timeout | 60000 | maxSessionTimeout in milliseconds for the embedded ZooKeeper server will allow the client to negotiate. Defaults to 20 times the tickTime | int | 1.2.0 |
| kyuubi.zookeeper.embedded.min.session.timeout | 6000 | minSessionTimeout in milliseconds for the embedded ZooKeeper server will allow the client to negotiate. Defaults to 2 times the tickTime | int | 1.2.0 |
| kyuubi.zookeeper.embedded.port | 2181 | The port of the embedded ZooKeeper server | int | 1.0.0 |
| kyuubi.zookeeper.embedded.port | 2181 | (deprecated) The port of the embedded ZooKeeper server | int | 1.0.0 |
| kyuubi.zookeeper.embedded.tick.time | 3000 | tickTime in milliseconds for the embedded ZooKeeper server | int | 1.2.0 |

## Spark Configurations
Expand Down
2 changes: 1 addition & 1 deletion docs/connector/spark/delta_lake_with_azure_blob.rst
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ Add the following content:
kyuubi.frontend.bind.host <YOUR_HOST>
kyuubi.frontend.bind.port 10009
# If you use your own zk cluster, you need to configure your zk host port.
# kyuubi.ha.zookeeper.quorum <YOUR_HOST>:2181
# kyuubi.ha.addresses <YOUR_HOST>:2181
Start Kyuubi
************
Expand Down
10 changes: 5 additions & 5 deletions docs/deployment/high_availability_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,21 +48,21 @@ Using multiple Kyuubi service units with load balancing instead of a single unit

When applying HA to Kyuubi deployment, we need to be aware of the below two thing basically,

- `kyuubi.ha.zookeeper.quorum` - the external zookeeper cluster address for deploy a `k.i.`
- `kyuubi.ha.zookeeper.namespace` - the root directory, a.k.a. the ServerSpace for deploy a `k.i.`
- `kyuubi.ha.addresses` - the external zookeeper cluster address for deploy a `k.i.`
- `kyuubi.ha.namespace` - the root directory, a.k.a. the ServerSpace for deploy a `k.i.`

For more configurations, please see the HA section of [Introduction to the Kyuubi Configurations System](./settings.html#ha)

### Pseudo mode

When `kyuubi.ha.zookeeper.quorum` is not configured, a `k.i.` will start an embedded zookeeper service and expose the address of itself there.
When `kyuubi.ha.addresses` is not configured, a `k.i.` will start an embedded zookeeper service and expose the address of itself there.
In this pseduo mode, the `k.i.` can be connected by clients through both raw ip address and zk quorum + namespace.
But it doesn't have any availability to being highly available.

### Production mode

For production deployment purpose, an external zookeeper cluster is required for `kyuubi.ha.zookeeper.quorum`.
In this mode, multiple `k.i.`s can be registered to the same ServerSpace configured by `kyuubi.ha.zookeeper.namespace` and serve together.
For production deployment purpose, an external zookeeper cluster is required for `kyuubi.ha.addresses`.
In this mode, multiple `k.i.`s can be registered to the same ServerSpace configured by `kyuubi.ha.namespace` and serve together.

## Client-side Usage

Expand Down
2 changes: 1 addition & 1 deletion docs/tools/kyuubi-ctl.rst
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ Create server
***********
Expose Kyuubi server instance to another domain.

First read ``kyuubi.ha.zookeeper.namespace`` in ``conf/kyuubi-defaults.conf``, if there are server instances under this namespace, register them in the new namespace specified by the ``--namespace`` parameter.
First read ``kyuubi.ha.namespace`` in ``conf/kyuubi-defaults.conf``, if there are server instances under this namespace, register them in the new namespace specified by the ``--namespace`` parameter.

.. code-block:: bash
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ object HighAvailabilityConf {
"1.3.2")
val HA_ZK_ACL_ENABLED: ConfigEntry[Boolean] =
buildConf("kyuubi.ha.zookeeper.acl.enabled")
.doc("Set to true if the ZooKeeper ensemble is kerberized")
.doc("(deprecated) Set to true if the ZooKeeper ensemble is kerberized")
.version("1.0.0")
.booleanConf
.createWithDefault(UserGroupInformation.isSecurityEnabled)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,14 @@ object ZookeeperConf {

@deprecated("using kyuubi.zookeeper.embedded.client.port instead", since = "1.2.0")
val EMBEDDED_ZK_PORT: ConfigEntry[Int] = buildConf("kyuubi.zookeeper.embedded.port")
.doc("The port of the embedded ZooKeeper server")
.doc("(deprecated) The port of the embedded ZooKeeper server")
.version("1.0.0")
.intConf
.createWithDefault(2181)

@deprecated("using kyuubi.zookeeper.embedded.data.dir instead", since = "1.2.0")
val EMBEDDED_ZK_TEMP_DIR: ConfigEntry[String] = buildConf("kyuubi.zookeeper.embedded.directory")
.doc("The temporary directory for the embedded ZooKeeper server. " +
.doc("(deprecated) The temporary directory for the embedded ZooKeeper server. " +
"If it is a relative path, it is resolved relative to KYUUBI_HOME. ")
.version("1.0.0")
.stringConf
Expand Down

0 comments on commit 49d0fff

Please sign in to comment.