Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Path style breaks fetching INDEX_PAGE #210

Open
HighOnMikey opened this issue Feb 7, 2024 · 5 comments · May be fixed by #230
Open

Path style breaks fetching INDEX_PAGE #210

HighOnMikey opened this issue Feb 7, 2024 · 5 comments · May be fixed by #230

Comments

@HighOnMikey
Copy link

loadContent will always attempt fetching the INDEX_PAGE by building the URI path using s3uri().

When using S3_STYLE = 'path', this will prepend the URI path with the S3 bucket. However, the fetch is made against nginx itself, meaning the URI path will always be incorrect for the requested index.

To Reproduce

Set the following variables:

ALLOW_DIRECTORY_LIST=true
PROVIDE_INDEX_PAGE=true
S3_STYLE=path

Expected behavior

Incorrect URI currently being built: http://127.0.0.1/<bucket_name>/<request_path>/<INDEX_PAGE>

Expected URI: http://127.0.0.1/<request_path>/<INDEX_PAGE>

Environments Tested

  • master @ c687663
  • ghcr.io/nginxinc/nginx-s3-gateway/nginx-oss-s3-gateway:latest
  • Backends: AWS, Ceph, OpenStack Swift

Workaround

I would submit a PR but I'm not sure which direction would be preferred by the project to go about building the URI path. However, here's a quick fix to demonstrate the issue:

From

const uri = s3uri(r);

To

let uri;
if (S3_STYLE === 'path') {
    uri = `${r.uri}${INDEX_PAGE}`;
} else {
    uri = s3uri(r);
}
@4141done
Copy link
Collaborator

4141done commented Feb 8, 2024

Thank you for the report @HighOnMikey , I'm a new maintainer on this project so let me take some time to understand that specific area more deeply and reply.

@4141done
Copy link
Collaborator

Hi @HighOnMikey thank you for your patience.
I took a look at this and want to be sure that I understand the issue. It's possible I'm misunderstanding some part of your report:

  1. S3 path style requests expect the request in the form https://s3.region-code.amazonaws.com/bucket-name/key-name
  2. The PROVIDE_INDEX_PAGE configuration option is intended to serve a page called index.html from the requested directory. For example, if I have a bucket called test-bucket, then a request to <s3 gateway host>/ would be expected to be proxied to s3 as https://s3.region-code.amazonaws.com/test-bucket/index.html

I think that's the shared understanding we need. Testing the code I didn't see any indication that it's not doing that but I could be missing a key part of your report so just wanting to be sure.

Here are some logs from my local testing requesting /foo with the configuration you specified (I added some additional debug prints):

2024/02/15 22:00:15 [info] 75#75: *35 js: Using path style uri : /bucket-1
2024/02/15 22:00:15 [info] 75#75: *35 js: s3uri: Internal: true | uri: / | uriPath: / | basePath: /bucket-1
2024/02/15 22:00:15 [info] 75#75: *35 js: S3 Request URI: GET /bucket-1/index.html
2024/02/15 22:00:15 [info] 75#75: *37 js: Using path style uri : /bucket-1
2024/02/15 22:00:15 [info] 75#75: *37 js: s3uri: Internal: false | uri: /bucket-1/index.html | uriPath: /bucket-1/index.html | basePath: /bucket-1
2024/02/15 22:00:15 [info] 75#75: *37 js: S3 Request URI: GET /bucket-1/bucket-1/index.html
127.0.0.1 - - [15/Feb/2024:22:00:15 +0000] "GET /bucket-1/index.html HTTP/1.1" 404 146 "-" "-" "-" "/bucket-1/bucket-1/index.html" "/bucket-1/index.html"
2024/02/15 22:00:15 [info] 75#75: *35 js: Using path style uri : /bucket-1
2024/02/15 22:00:15 [info] 75#75: *35 js: s3uri: Internal: true | uri: / | uriPath: / | basePath: /bucket-1
2024/02/15 22:00:15 [info] 75#75: *35 js: S3 Request URI: GET /bucket-1?delimiter=%2F
192.168.65.1 - - [15/Feb/2024:22:00:15 +0000] "GET / HTTP/1.1" 200 1100 "-" "curl/8.4.0" "-" "/bucket-1?delimiter=%2F" "/"

I think I see the error here, when we redirect to /bucket-name/index.html the helper prepends the bucket name again:
2024/02/15 22:00:15 [info] 75#75: *37 js: S3 Request URI: GET /bucket-1/bucket-1/index.html

Is this the bug you're referring to or have I found a different bug? 😓

@HighOnMikey
Copy link
Author

First of all, sorry for the delayed response!

I think that's the shared understanding we need.

Correct. The core functionality of path style requests works as intended. The bug is limited to attempting to find an index page when PROVIDE_INDEX_PAGE is enabled and S3_STYLE=path.

Is this the bug you're referring to or have I found a different bug? 😓

Yes, this is directly related to the bug. The loadContent function will use the local nginx proxy to test fetch to see if an index page exists, but it uses the s3uri function which is designed for building a URI to request from the S3 API directly.

@4141done
Copy link
Collaborator

Thanks for confirming and for your patience. I'm going to try to put together a fix in the coming days. I'll tag you for a test/review when it's up.

@4141done 4141done linked a pull request Apr 19, 2024 that will close this issue
Open
@4141done
Copy link
Collaborator

Hey @HighOnMikey sorry again for the slowness on this. Can you take a look at #230 and see if it solves your issue?

Let me know if you need a container image built for you to test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@4141done @HighOnMikey