Federation of Catalog-level entities to remote catalogs

[dennishuo]

Is your feature request related to a problem? Please describe.

As Iceberg users sometimes find different Iceberg-catalog sources of truth being created organically over time, it can be difficult to consolidate or centralize workloads onto a new target architecture or onboard new catalog implementations like Polaris to unlock its RBAC features without disrupting ongoing workloads.

Being able to have one Catalog instance serve as a combined "pane of glass" serving the contents of a legacy Catalog alongside new datasets enables an incremental migration or even simply enabling different workloads to leverage different catalog features without a stop-the-world migration.

Describe the solution you'd like

Read-through federation of Catalog entities (e.g. Namespaces, Tables, Views) would allow a Polaris instance to link in an existing remote Catalog as a source-of-truth while exposing a Polaris entry-point along with Polaris RBAC features without performing a disruptive migration of Catalog contents.

Ideally, this federation would require minimal setup of the initial catalog connection while still benefiting from all the fine-grained access control features expected from normal Polaris catalogs; it should intelligently pass through API operations but also allow defining privilege grants on entities without requiring prior setup/syncing of those entities into the Polaris federation layer.

Describe alternatives you've considered

No response

Additional context

No response

[dennishuo]

Proposal with MVP scope and some tentative design elements in this document: https://docs.google.com/document/d/1Q6eEytxb0btpOPcL8RtkULskOlYUCo_3FLvFRnHkzBY/edit?tab=t.0