Feat/migrate sql gen to macro

[aiss93]

resolves #457

Description

The following PR addresses the following points:

• Migrates the PySpark code for the Iceberg file format at a macro level, making the impl.py file more readable.
• Fixes the get_columns_in_relation function to work for both Iceberg and non-Iceberg tables without hard-coding the catalog name.
• Fixes the get_location table function to work for both Iceberg and non-Iceberg tables on macOS and Windows. The old os.path.join function didn’t work well on Windows OS, as it built a path using '' instead of '/', leading to S3 path location errors.
• Adds a helper function to retrieve the Iceberg catalog namespace from the profile.yaml file, allowing users to name their Iceberg catalog namespace as desired.
• Adds merge_exclude_columns and incremental_predicates features.

Checklist

• I have signed the CLA
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have updated the CHANGELOG.md and added information about my change to the "dbt-glue next" section.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[moomindani]

@aiss93 Apologizes I forgot to submit my comments last week. Submitted them now. Having test_iceberg is good idea, but I think we still need to make sure that we have enough coverage in the test cases. Plz refer to my comments.

[aiss93]

Hi @moomindani do you have any additional comments or can we launch the functional tests ?
Thank you for your time

[moomindani]

Hi @moomindani do you have any additional comments or can we launch the functional tests ? Thank you for your time

Thanks for making changes based on my comments. Apologizes for delay in review. We are still reviewing the changes. It may take some time as our bandwidth is very limited this month, but let us update you.

[aiss93]

Sure thank you !

[moomindani]

Thank you for your contribution! This patch brought huge value, we really appreciate your contribution.

[sanga8]

configuration

[moomindani]

Oops thanks I did not notice. Addressed it.