[Bug] flink yarn-session cannot create catalog type='hive'

[hanson2021]

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before asking

• I have searched in the issues and found no similar issues.

Describe the bug

flink.version=1.17.1
kyuubi.version=1.8.0
hive.version=3.1.2
when create hive catalog on as followings:
CREATE CATALOG bidwhive WITH ('type' = 'hive', 'hive-version' = '3.1.2', 'default-database' = 'test');
(1) when set flink.execution.target=yarn-application, it created catalog successfully. and query hive tables is ok.
(2)when set flink.execution.target=yarn-session and prestart yarn-session applicationid on yarn, runtime errors as following
(3) when reuse the same yarn-session applicationid on sql-client.sh embedded,
./flink-1.17.1/bin/sql-client.sh embedded
It also was ok to create catalog successfully and query hive tables .

Can anybody help to how to solve this problem?

Affects Version(s)

master/1.8.0

Kyuubi Server Log Output

No response

Kyuubi Engine Log Output

Error: Error operating ExecuteStatement: org.apache.flink.table.api.ValidationException: Unable to create catalog 'bidwhive'.

Catalog options are:
'default-database'='test'
'hive-version'='3.1.2'
'type'='hive'
        at org.apache.flink.table.factories.FactoryUtil.createCatalog(FactoryUtil.java:439)
        at org.apache.flink.table.api.internal.TableEnvironmentImpl.createCatalog(TableEnvironmentImpl.java:1466)
        at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:1212)
        at org.apache.flink.table.gateway.service.operation.OperationExecutor.callOperation(OperationExecutor.java:541)
        at org.apache.flink.table.gateway.service.operation.OperationExecutor.executeOperation(OperationExecutor.java:440)
        at org.apache.flink.table.gateway.service.operation.OperationExecutor.executeStatement(OperationExecutor.java:195)
        at org.apache.kyuubi.engine.flink.operation.ExecuteStatement.executeStatement(ExecuteStatement.scala:64)
        at org.apache.kyuubi.engine.flink.operation.ExecuteStatement.runInternal(ExecuteStatement.scala:56)
        at org.apache.kyuubi.operation.AbstractOperation.run(AbstractOperation.scala:171)
        at org.apache.kyuubi.session.AbstractSession.runOperation(AbstractSession.scala:101)
        at org.apache.kyuubi.session.AbstractSession.$anonfun$executeStatement$1(AbstractSession.scala:131)
        at org.apache.kyuubi.session.AbstractSession.withAcquireRelease(AbstractSession.scala:82)
        at org.apache.kyuubi.session.AbstractSession.executeStatement(AbstractSession.scala:128)
        at org.apache.kyuubi.service.AbstractBackendService.executeStatement(AbstractBackendService.scala:67)
        at org.apache.kyuubi.service.TFrontendService.ExecuteStatement(TFrontendService.scala:252)
        at org.apache.kyuubi.shade.org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
        at org.apache.kyuubi.shade.org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
        at org.apache.kyuubi.shade.org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.kyuubi.shade.org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.kyuubi.service.authentication.TSetIpAddressProcessor.process(TSetIpAddressProcessor.scala:36)
        at org.apache.kyuubi.shade.org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.table.api.ValidationException: Could not find any factory for identifier 'hive' that implements 'org.apache.flink.table.factories.CatalogFactory' in the classpath.

Available factory identifiers are:

generic_in_memory
        at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:546)
        at org.apache.flink.table.factories.FactoryUtil.getCatalogFactory(FactoryUtil.java:724)
        at org.apache.flink.table.factories.FactoryUtil.createCatalog(FactoryUtil.java:421)
        ... 23 more (state=,code=0)

Kyuubi Server Configurations

No response

Kyuubi Engine Configurations

kyuubi.engine.type=FLINK_SQL 
flink.execution.target=yarn-session
flink.yarn.application.id=application_XXX_XX

Additional context

No response

Are you willing to submit PR?

• Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
• No. I cannot submit a PR at this time.