[Feature] Support AWS Glue & Lake Formation for Paimon metadata/metastore

[gmdfalk]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Apache Iceberg supports AWS Glue & AWS Lake Formation to manage tables & permissions centrally and more easily than on the S3 level with IAM policies.

Can we get AWS Glue & Lake Formation support for Paimon metadata? This would make it easier to use Paimon in AWS ecosystems, including integrating with Athena for querying Paimon tables in the future.

Solution

We already support AWS Glue in Apache Paimon for the Iceberg-compatibility layer.
Currently, we need to build and configure an external client: https://github.com/awslabs/aws-glue-data-catalog-client-for-apache-hive-metastore/tree/branch-3.4.0

Maybe we can port some of the AWS/Iceberg code to Paimon, including GlueCatalog and LakeFormationAwsClientFactory.

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[JingsongLi]

+1 I think it is good to support GlueCatalog in Paimon too.