Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync writes __source__ falsely into frozen layers #130

Merged
merged 7 commits into from
Sep 10, 2024
Merged

Conversation

Wiseqube
Copy link
Contributor

@Wiseqube Wiseqube commented Sep 4, 2024

This PR fixes an issue where zodbsync was overwriting the contents of a source-file in a frozen layer.

The test-case is as follows:

  1. We record an object into the custom layer
  2. The filesystem objects are moved to the frozen layer manually
  3. The content in the Data.FS is modified, in this case we edith both the meta aswell as the content data
  4. A record is executed.

The expected behaviour would be that the changed meta and source files are both written into the custom-layer. The actual behaviour is that the __meta__-File is correctly written into the custom-layer but the __source__-File is modified in the frozen layer below!

The issue was that the path which was generated for the source-file was based on the layer-path while the path for the meta-file was generated based upon the custom upper layer!

@Wiseqube Wiseqube added the bug Something isn't working label Sep 4, 2024
@Wiseqube Wiseqube marked this pull request as draft September 4, 2024 20:34
@Wiseqube Wiseqube changed the title WIP: Sync writes __source__ falsely into frozen layers Sync writes __source__ falsely into frozen layers Sep 6, 2024
@Wiseqube Wiseqube marked this pull request as ready for review September 6, 2024 06:19
Copy link
Member

@gandie gandie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One typo, also i suggest being extra suspicious:

diff --git a/perfact/zodbsync/tests/test_sync.py b/perfact/zodbsync/tests/test_sync.py
index 0c27784..f545d6c 100644
--- a/perfact/zodbsync/tests/test_sync.py
+++ b/perfact/zodbsync/tests/test_sync.py
@@ -2179,6 +2179,10 @@ class TestSync():
             # both meta and source file are in custom layer
             assert os.path.exists(os.path.join(root, 'blob/__meta__'))
             assert os.path.exists(os.path.join(root, 'blob/__source__.txt'))
-            with open('{}/__root__/blob/__source__.txt'.format(layer)) as f:
+            source_fmt = '{}/__root__/blob/__source__.txt'
+            with open(source_fmt.format(layer)) as f:
                 # source in layer should still be empty
                 assert f.read() == ''
+            with open(source_fmt.format(self.repo.path)) as f:
+                # ... content is in custom layer!
+                assert f.read() == 'text_content'


def test_layer_frozen(self):
"""
Verify that changed files are propery written into the custom
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

properly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the input. I've implemented both changes!

@Wiseqube Wiseqube requested a review from gandie September 6, 2024 18:00
Copy link
Member

@gandie gandie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about this one to make things a little more clear (tests seem cool with it):

diff --git a/perfact/zodbsync/zodbsync.py b/perfact/zodbsync/zodbsync.py
index c0eaf01..85cf1c0 100644
--- a/perfact/zodbsync/zodbsync.py
+++ b/perfact/zodbsync/zodbsync.py
@@ -476,7 +476,7 @@ class ZODBSync:
         if old_data != new_data:
             # Path in top layer, might be different than the one where we read
             # the content
-            write_base = os.path.join(self.app_dir, path.lstrip('/'))
+            write_base = self.fs_path(path)
             os.makedirs(write_base, exist_ok=True)

             self.logger.debug("Will write %d bytes of metadata" % len(fmt))

Comment and docstring of fs_path then match nicely imo

@Wiseqube
Copy link
Contributor Author

Wiseqube commented Sep 9, 2024

@gandie accidentially put this additional reafactoring into a branch which is already merged now, so I guess ready for review again!

Copy link
Collaborator

@viktordick viktordick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

with self.runner.sync.tm:
self.app.manage_addProduct['OFSP'].manage_addFile(id='blob')

with self.addlayer(frozen=True) as layer:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty sure that frozen = True is the default for layers below the top-most custom layer. So not sure why this was necessary, but it still is OK.

@Wiseqube Wiseqube merged commit 803e4c8 into master Sep 10, 2024
3 checks passed
@Wiseqube Wiseqube deleted the frozen-overwrite branch September 10, 2024 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants