Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue manifest file path use s3a/n scheme with scala 2.11 and hadoop aws 3.x #164

Open
jjgit95 opened this issue Sep 30, 2024 · 0 comments

Comments

@jjgit95
Copy link

jjgit95 commented Sep 30, 2024

Hi,
I need to run the connector spark-redshift-2.11 on a cluster that use hadoop aws 3.x.
The last version of connector that supports scala 2.11 is the 4.11 compiled with hadoop aws 2.7.7.
So, for my use case, i need a version that supports scala 2.11 compiled with hadoop 3.x.
In the version 4.1.1 the class Redshit writer create val fsDataOut = fs.create(new Path(manifestPath)) where manifestPath is the bucket s3 without s3a://.
So i need that the fix that was applied on version 5.1.1 being applied on a new version ( 4.1.2?) that works with scala 2.11.
It is possible? Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant