Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-12094. Update BasicKeyInfo to include isFile payload #7740

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

ptlrs
Copy link
Contributor

@ptlrs ptlrs commented Jan 23, 2025

What changes were proposed in this pull request?

The name of directories and subdirectories do not have a trailing slash.
This results in them being interpreted as files instead of directories.
As a result isFile() incorrectly returns true even for directories.

This PR :

  • adds the isFile payload to the BasicKeyInfo proto message
  • updates the protobuf (de)serialization in BasicOmKeyInfo to get/set the isFile status
  • adds a new integration test

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-12094

How was this patch tested?

Added new integration test in OzoneRpcClientTests#testListKey.

CI: https://github.com/ptlrs/ozone/actions/runs/12945031804

@ptlrs ptlrs force-pushed the HDDS-12094-isfile branch from 0cb669b to c5e588f Compare January 24, 2025 02:33
Comment on lines +2842 to +2844
@Test
public void testListKeyDirectoriesAreNotFiles()
throws IOException {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could not add this test to testListKey due to checkstyle limit of 150 lines per method. Created a new test instead.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to use ParameterizedTest in testListKey to include the NotFile cases you added here?

@ptlrs ptlrs marked this pull request as ready for review January 24, 2025 08:27
Copy link
Contributor

@jojochuang jojochuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to be careful about backward compatibility.

@@ -1182,6 +1182,7 @@ message BasicKeyInfo {
optional hadoop.hdds.ECReplicationConfig ecReplicationConfig = 7;
optional string eTag = 8;
optional string ownerName = 9;
optional bool isFile = 10;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BasicKeyInfo was added in HDDS-9079 (1.4.0).

@@ -254,7 +255,7 @@ public static BasicOmKeyInfo getFromProtobuf(String volumeName,
basicKeyInfo.getFactor(),
basicKeyInfo.getEcReplicationConfig()))
.setETag(basicKeyInfo.getETag())
.setIsFile(!keyName.endsWith("/"))
.setIsFile(basicKeyInfo.getIsFile())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If an old client send a request containing BasicKeyInfo to a new OM, the isFile field will not be set and a file would always be identified as not a file.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ptlrs can you please add test case in hadoop-ozone/dist/src/main/smoketest/compatibility/read.robot?

@adoroszlai adoroszlai marked this pull request as draft January 30, 2025 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants