[FEATURE REQUEST] Develop Apache Ranger Plugin for Polaris to Enhance Access Control for Apache Iceberg

[dbosco]

Is your feature request related to a problem? Please describe.

No response

Describe the solution you'd like

Apache Polaris provides metadata management for Apache Iceberg. From the authorization point of view, key features of Polaris include:

• RBAC (Role-Based Access Control): Polaris supports RBAC for table and view-level operations. See Documentation
• Role Management: Polaris allows the creation of Principals with roles like Data Engineer, Data Scientist, etc.
• Catalog Roles: Specialized roles like Catalog Administrators, Catalog Readers, and Catalog Contributors can be defined to manage access to different parts of the data catalog.
• Granular Privileges: Polaris provides fine-grained privileges for operations on Tables, Views, Namespaces, and Catalogs. Examples include TABLE_CREATE, TABLE_READ_DATA, TABLE_WRITE_DATA, VIEW_CREATE, NAMESPACE_CREATE, CATALOG_MANAGE_CONTENT, and more.
• Credential Vending: Polaris vends credentials based on the specific table the user is trying to access.
• API for Role Management: Polaris offers an API to manage grants for roles, allowing fine-tuned control over data access.

Objective:

To enhance the usability and security of Polaris for Apache Iceberg users, the request is to develop an Apache Ranger plugin that integrates Polaris' access control features with Apache Ranger. This integration will allow for centralized and consistent management of access policies, audit logging, and fine-grained access control across different tools used with Apache Iceberg.

Use Cases:

1. Centralized Access Policy Management:

• Implement centralized and consistent management of access policies for data stored using Apache Iceberg across multiple tools and environments.

2. Access Control for Data Engineering Workloads:

• Manage and control access to datasets used by Data Engineering workloads (e.g., Apache Spark) with a coarser-grained approach at the table level.

3. Fine-Grained Access Control for Data Analysts:

• Provide fine-grained access control for Data Analysts using compute engines like Trino. This control can be enforced by leveraging the native Ranger Plugin in Trino, allowing for more granular control over data access at the table, view, or even column level.

4. Centralized Access Auditing:

• Enable centralized collection and analysis of access audit logs across all tools used to access datasets in Iceberg, ensuring comprehensive auditing and compliance.

Expected Deliverables:

• A fully functional Apache Ranger plugin for Polaris that supports the outlined use cases.
• Documentation on how to configure and deploy the plugin.
• Integration tests to ensure the plugin works as expected with Apache Iceberg and other tools like Apache Spark and Trino.
• A detailed user guide explaining how to use the plugin for managing access control in various scenarios.

Describe alternatives you've considered

No response

Additional context

References

PolarisAuthorizer Class on GitHub: The PolarisAuthorizer class provides the core authorization logic in Polaris, which can be leveraged by the Apache Ranger plugin.

Most Apache projects and Open Source projects like Presto (https://prestodb.io/docs/current/connector/hive-security.html#ranger-based-authorization) , Trino (trinodb/trino#22674), Apache Hive (https://github.com/apache/ranger/tree/master/hive-agent), Apache Kafka (https://cwiki.apache.org/confluence/display/RANGER/Kafka+Plugin have native integration with Apache Ranger. Some of these might also benefit with this integration

A corresponding tracking JIRA is also created in the Apache Ranger project. https://issues.apache.org/jira/browse/RANGER-4910