Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix S3 multipart upload and add related UTs #18046

Merged
merged 2 commits into from
Aug 22, 2023

Conversation

007DXR
Copy link
Contributor

@007DXR 007DXR commented Aug 22, 2023

What changes are proposed in this pull request?

  1. fix bugs in alluxio.proxy.s3.S3ObjectTask.CompleteMultipartUploadTask#validateParts
  2. CompleteMultipartUpload executes the specified parts with the specified upload ID rather than all parts.
  3. add initiate / complete / abort MPU and upload part unit tests.
  4. remove XAttr

Why are the changes needed?

there are problems in validateParts() of complete MPU

  1. can’t recognize S3 Error: MalformedXML
  2. throw unexpected error: InvalidPartOrder

we need to fix validateParts() of complete MPU to keep s3 proxy behaviors as consistent with aws as possible.

Does this PR introduce any user facing changes?

  • can't get the bucket name and object name of the part while executing ListMultipartUploads

  • now complete MPU's response is like

part list response(before) response(now)
[] 500 MalformedXML
[1,3] InvalidPartOrder 200
[2,1] InvalidPartOrder InvalidPartOrder
[3,3] InvalidPartOrder InvalidPartOrder

@007DXR
Copy link
Contributor Author

007DXR commented Aug 22, 2023

@Jackson-Wang-7 ask for review

Copy link
Contributor

@Jackson-Wang-7 Jackson-Wang-7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Jackson-Wang-7 Jackson-Wang-7 added the type-debt This issue is about tech debt label Aug 22, 2023
@Jackson-Wang-7
Copy link
Contributor

alluxio-bot, merge this please

@alluxio-bot alluxio-bot merged commit 07f0dfb into Alluxio:main Aug 22, 2023
12 checks passed
@007DXR 007DXR deleted the multipartupload2 branch August 22, 2023 07:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-debt This issue is about tech debt
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants