AWSSDK.S3 3.7.307.4 escapes colon in bucket name in GetPreSignedURL

[desdoades]

Describe the bug

We use ceph S3 where bucket names are not a subdomain but after the domain, e.g. https://example.com/11213ef55:mybucket/path/to/file.jpg.
When using AWSSDK.S3 3.7.307.4 it percent-encodes a colon in the bucket name (which is a valid character in an URL path) when calling GetPreSignedURL(request).
This breaks the upload to our S3 file host since it can not match to the correct bucket.

Can this be fixed on your end or would you say thats a problem of the ceph S3?

Expected Behavior

We are currently on AWSSDK 3.7.103.13 where it does not percent-encode a colon in the bucket name.
In this version the file upload to S3 works.

Current Behavior

Because of the colon percent-encode the s3 ceph file server returns a

Forbidden AccessDenied...
when trying to upload a file to the pre signed URL.

Reproduction Steps

That should not be neccessary

Possible Solution

No response

Additional Information/Context

No response

AWS .NET SDK and/or Package version used

AWSSDK.S3 3.7.307.4

Targeted .NET Platform

.NET 8

Operating System and version

Windows 10

[ashishdhingra]

@desdoades Good morning. Thanks for reporting the issue. Could you please share if you are using AWS S3 service or simulated service that mimics S3?

Thanks,
Ashish

EDIT: Looks like you are using CEPH S3 service.

[ashishdhingra]

Need reproduction with actual AWS S3 service.

[ashishdhingra]

@desdoades Good afternoon. Please confirm if your bucket is named 11213ef55:mybucket. AWS S3 console doesn't allow : (colon) character in bucket name. I'm unsure how CEPH S3 service is allowing it. So it appears be an issue with CEPH S3 side, not SDK side since : is not a valid character in bucket name.

Thanks,
Ashish

[desdoades]

Dear @ashishdhingra,

we are not using AWS S3 but netways S3 which uses ceph. As I said it does not have the bucketname in the subdomain but in the path.
Our actual bucket is not named 11213ef55:mybucket but very similar, so it does contain a colon.

[ashishdhingra]

Dear @ashishdhingra,

we are not using AWS S3 but netways S3 which uses ceph. As I said it does not have the bucketname in the subdomain but in the path. Our actual bucket is not named 11213ef55:mybucket but very similar, so it does contain a colon.

@desdoades Thanks for the reply. As mentioned in my previous comment, colon : character is not supported in AWS bucket names. The colon : is a reserved character in a URL and it has a special meaning in identifying the protocol used in the URL (http:, https:, ftp: etc.). However, you can still add a colon in other parts of a URL by using URL encoding. URL encoding is a process of converting reserved characters into their corresponding hexadecimal code so that they can be safely transmitted over the internet.

Please contact Ceph S3 support to handle this character at their end (this may be as simple as decoding the URL before processing various segments).

Thanks,
Ashish

[desdoades]

Dear @ashishdhingra,

While AWS might not support them many site allow colons in the URL.
Still thank you, I understand you wont do anything about it. I will try to reach our ceph S3 support.

[github-actions]

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.