Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix S3 CompleteMultipartUpload and add related UTs #17874

Closed
wants to merge 27 commits into from

Conversation

007DXR
Copy link
Contributor

@007DXR 007DXR commented Aug 2, 2023

What changes are proposed in this pull request?

  1. fix bugs in alluxio.proxy.s3.S3ObjectTask.CompleteMultipartUploadTask#validateParts
  2. CompleteMultipartUpload executes the specified parts with the specified upload ID rather than all parts.
  3. add initiate / complete / abort MPU and upload part unit tests.
  4. remove XAttr
  5. show all buckets/upload-parts to user while listing buckets/ upload-parts

Why are the changes needed?

there are problems in validateParts() of complete MPU

  1. can’t recognize S3 Error: MalformedXML
  2. throw unexpected error: InvalidPartOrder

we need to fix validateParts() of complete MPU to keep s3 proxy behaviors as consistent with aws as possible.

Does this PR introduce any user facing changes?

  • can't get the bucket name and object name of the part while executing ListMultipartUploads

  • now complete MPU's response is like

part list response(before) response(now)
[] 500 MalformedXML
[1,3] InvalidPartOrder 200
[2,1] InvalidPartOrder InvalidPartOrder
[3,3] InvalidPartOrder InvalidPartOrder

@007DXR 007DXR changed the title Multipartupload [SMALLFIX] Fix S3ObjectTask.CompleteMultipartUploadTask Aug 2, 2023
@007DXR 007DXR closed this Aug 3, 2023
@007DXR 007DXR force-pushed the multipartupload branch 2 times, most recently from 520eea5 to 17091b3 Compare August 3, 2023 07:54
…pload

# Conflicts:
#	dora/tests/src/test/java/alluxio/client/rest/RestApiTest.java
@007DXR 007DXR reopened this Aug 3, 2023
@007DXR 007DXR changed the title [SMALLFIX] Fix S3ObjectTask.CompleteMultipartUploadTask Fix S3 CompleteMultipartUpload and add related UTs Aug 9, 2023
@007DXR 007DXR changed the title Fix S3 CompleteMultipartUpload and add related UTs [WIP]Fix S3 CompleteMultipartUpload and add related UTs Aug 9, 2023
@007DXR 007DXR changed the title [WIP]Fix S3 CompleteMultipartUpload and add related UTs Fix S3 CompleteMultipartUpload and add related UTs Aug 10, 2023
@pkuweblab
Copy link
Contributor

@Jackson-Wang-7 ask for review

Copy link
Contributor

@Jackson-Wang-7 Jackson-Wang-7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good, left some comments.

Copy link
Contributor

@Jackson-Wang-7 Jackson-Wang-7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Contributor

@Jackson-Wang-7 Jackson-Wang-7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left a small question

@Jackson-Wang-7
Copy link
Contributor

can be closed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants