Skip to content

Commit

Permalink
Merge pull request #269 from aajisaka/fix-iceberg-doc
Browse files Browse the repository at this point in the history
[minor][doc] Fix Iceberg config in README
  • Loading branch information
menuetb authored Nov 10, 2023
2 parents f5c40fc + 203892f commit d98dc06
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -543,7 +543,8 @@ group by 1
- the latest connector for iceberg in AWS marketplace uses Ver 0.14.0 for Glue 3.0, and Ver 1.2.1 for Glue 4.0 where Kryo serialization fails when writing iceberg, use "org.apache.spark.serializer.JavaSerializer" for spark.serializer instead, more info [here](https://github.com/apache/iceberg/pull/546)
- For Athena version 2: The adapter is compatible with the Iceberg Connector from AWS Marketplace with Glue 3.0 as Fulfillment option and 0.12.0-2 (Feb 14, 2022) as Software version)
- For Glue 4.0, to add the following configurations in dbt-profile:
```--conf spark.sql.catalog.glue_catalog=org.apache.iceberg.spark.SparkCatalog
```
--conf spark.sql.catalog.glue_catalog=org.apache.iceberg.spark.SparkCatalog
--conf spark.sql.catalog.glue_catalog.warehouse=s3://<PATH_TO_YOUR_WAREHOUSE>
--conf spark.sql.catalog.glue_catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog
--conf spark.sql.catalog.glue_catalog.io-impl=org.apache.iceberg.aws.s3.S3FileIO
Expand Down

0 comments on commit d98dc06

Please sign in to comment.