zookeeper and kyuubi server

[vamshik2022]

Appreciate your response, We are using Kyuubi server with the zookeeper as service.
But we noticed Embedded zookeeper is taking < 10secs during the session while we try to retrieve the data ( using spark connecting to Cassandra)
Just wanted to check if there is way to reduce this time . Please find the attached information .

---

zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a
2021-12-24 06:08:28.514 DEBUG server.FinalRequestProcessor: sessionid:0x104f9c40d280000 type:ping cxid:0xfffffffffffffffe zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a
2021-12-24 06:08:28.514 DEBUG zookeeper.ClientCnxn: Got ping response for sessionid: 0x104f9c40d280000 after 1ms
2021-12-24 06:08:32.150 DEBUG server.KyuubiBackendService: OperationHandle [type=EXECUTE_STATEMENT, identifier: d97f57c7-43db-4f09-9043-70332a65ac0a]: Long polling timed out, null
2021-12-24 06:08:48.530 DEBUG zookeeper.ClientCnxn: Got ping response for sessionid: 0x104f9c40d280000 after 1ms
2021-12-24 06:08:52.817 DEBUG transport.TSaslTransport: CLIENT: reading data length: 79
2021-12-24 06:08:52.817 DEBUG transport.TSaslTransport: writing data length: 117
2021-12-24 06:08:52.818 DEBUG transport.TSaslTransport: CLIENT: reading data length: 95
2021-12-24 06:09:08.534 DEBUG zookeeper.ClientCnxn: Got ping response for sessionid: 0x104f9c40d280000 after 0ms

2021-12-24 06:09:23.515 DEBUG server.FinalRequestProcessor: Processing request:: sessionid:0x104f9c40d280009 type:ping cxid:0xfffffffffffffffe zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a

[pan3793]

Zookeeper is only for service discovery, it doesn't play any role in your retrieving data case. You can compare Kyuubi and spark-sql or spark-shell in same cases and see what happens

[vamshik2022]

Thanks for the response , If they are not related then, during the query execution it is trying to contact the zk and Kyuubi vice versa .
Do you mean to say the query execution is continued between ZK and Kyuubi ping process. ? if that is true then do you see in the log ( 28 sec to 32 sec ) how the data is written after the interaction is completed b/w ZK and Kyuubi.

2021-12-24 06:08:27.151 DEBUG transport.TSaslTransport: writing data length: 100
2021-12-24 06:08:28.513 DEBUG server.FinalRequestProcessor: Processing request:: sessionid:0x104f9c40d280000 type:ping cxid:0xfffffffffffffffe zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a
2021-12-24 06:08:28.514 DEBUG server.FinalRequestProcessor: sessionid:0x104f9c40d280000 type:ping cxid:0xfffffffffffffffe zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a
2021-12-24 06:08:28.514 DEBUG zookeeper.ClientCnxn: Got ping response for sessionid: 0x104f9c40d280000 after 1ms
2021-12-24 06:08:32.150 DEBUG server.KyuubiBackendService: OperationHandle [type=EXECUTE_STATEMENT, identifier: d97f57c7-43db-4f09-9043-70332a65ac0a]: Long polling timed out, null
2021-12-24 06:08:32.150 DEBUG transport.TSaslTransport: writing data length: 79

[yaooqinn]

zk logs seem to be not related, try decrease kyuubi.operation.status.polling.timeout(defaults to PT5S/5 sec)