Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Master loads metadata very slowly without libjudy #239

Closed
uristdwarf opened this issue Nov 7, 2024 · 2 comments
Closed

Master loads metadata very slowly without libjudy #239

uristdwarf opened this issue Nov 7, 2024 · 2 comments
Assignees
@uristdwarf
Labels
dependencies Pull requests that update a dependency file enhancement New feature or request Priority: Medium Performance degradation, bugs

Comments

@uristdwarf
Copy link
Collaborator

uristdwarf commented Nov 7, 2024 •
edited
Loading

Version

4.6.0-rc3

Description

When upgrading an existing installation from 4.5.0 master starts very slowly. It took about 19 hours to finish loading the metadata. 4.5.0 doesn't have this issue, as downgrading it is much faster (around a few minutes on our test cluster)

It seems the final section of the metadata (FREE 1.0) takes the longest, as everything else is much faster.

This is also related to uraft, as uraft won't assign a floating IP address until master loads the metadata and starts.

Reproduction

  1. Setup a 4.5.0 cluster and write a lot of files.
  2. Upgrade to 4.6.0
  3. Start master from 4.6.0
@uristdwarf uristdwarf added bug Something isn't working Priority: High Heavy performance degradation, major bug regression Something worked in previous version(s) labels Nov 7, 2024
@uristdwarf uristdwarf pinned this issue Nov 7, 2024
@uristdwarf
Copy link
Collaborator Author

uristdwarf commented Nov 7, 2024 •
edited
Loading

This issue happened because of a refactor in our packaging due to missing libjudy-dev dependency, not any bug with the source code. Regardless, due the abnormal load time for metadata without libjudy, we should require this in the CMakeLists.txt

@uristdwarf uristdwarf added enhancement New feature or request dependencies Pull requests that update a dependency file Priority: Medium Performance degradation, bugs and removed bug Something isn't working Priority: High Heavy performance degradation, major bug regression Something worked in previous version(s) labels Nov 7, 2024
@uristdwarf uristdwarf changed the title Master loads metadata very slowly Master loads metadata very slowly without libjudy Nov 7, 2024
@uristdwarf uristdwarf self-assigned this Nov 7, 2024
uristdwarf added a commit that referenced this issue Nov 8, 2024
@uristdwarf
build: Require Judy library
79304fc 
This library was considered optional, however it is required for
performance reasons, as when loading the FREE section it would take a
long time (depending on the size). This prevents compilations without it

Fixes #239
uristdwarf added a commit that referenced this issue Nov 8, 2024
@uristdwarf
build: Require Judy for building
64a7483 
If Judy is not present, the master could take a very long time to load
(depending on FREE section size). This fixes that by requiring Judy for
building.

Fixes #239
@uristdwarf uristdwarf mentioned this issue Nov 8, 2024
Closed
@uristdwarf
Copy link
Collaborator Author

Specific compile time directives were added to disable for certain data structures, Judy will remain optional for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@uristdwarf uristdwarf

Labels
dependencies Pull requests that update a dependency file enhancement New feature or request Priority: Medium Performance degradation, bugs
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@uristdwarf