Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vlv index keys inconsistencies #6028

Closed
progier389 opened this issue Jan 8, 2024 · 4 comments · Fixed by #6031
Closed

vlv index keys inconsistencies #6028

progier389 opened this issue Jan 8, 2024 · 4 comments · Fixed by #6031
Labels
lmdb LMDB related

Comments

@progier389
Copy link
Contributor

progier389 commented Jan 8, 2024

Issue Description
When working on issue #6022 I discovered some inconsistencies that impact both bdb and lmdb

Package Version and Platform:

  • Platform: Fedora
  • Package and version: main branch

Steps to Reproduce
Steps to reproduce the behavior:

  1. Go to 389-ds-base/dirsrvtests/tests/suites/vlv
  2. Edit regression_test.py and comment M2.restart() in test_vlv_cache_subdb_names
  3. Run ( DEBUGGING=1 pytest -x $PWD -k test_vlv_cache_subdb_names)
  4. Run dbscan -f '$DBDIR/userroot/vlv#vlvidx.db' > keys.1
  5. Run ( DEBUGGING=1 pytest -x $PWD -k 'test_bulk_import_when_the_backend_with_vlv_was_recreated or test_vlv_cache_subdb_names')
  6. Run dbscan -f '$DBDIR/userroot/vlv#vlvidx.db' > keys.2
  7. compare keys.1 and keys.2
    See that keys.1 are the keys for the entries that matches (uid=)
    See that keys.2 are the keys for the entries that matches (|(uid=
    )(cn=*))

Expected results
Should have keys matching the same entries

@progier389 progier389 added needs triage The issue will be triaged during scrum priority_low valuable but very complex/risky/minor labels Jan 8, 2024
@tbordaz
Copy link
Contributor

tbordaz commented Jan 8, 2024

The patch itself looks good but I am very novice on vlv index/cache. Something surprising to me is that the description states inconsistency on both bdb and mdb but the PR is only fixing mdb.

@progier389
Copy link
Contributor Author

The reason is that PR #6026 is fixing the issue #6022 which only impact mdb.
While testing that PR I hit this issue #6028 and used a work around (restarting the instance) so that the CI test is passing both on bdb and mdb.
FYI I open this issue for tracking purpose but IMHO it is low priority as it is not a regression and no one has reported it.

@progier389 progier389 added lmdb LMDB related and removed priority_low valuable but very complex/risky/minor labels Jan 10, 2024
@progier389
Copy link
Contributor Author

I made a mistake in my test (run the wrong testcase), the inconsistency exists only purely with lmdb
There is a different problem on bdb (bdb_remove hangs in some case) but I will create a new issue for it.

@progier389 progier389 changed the title vlv index inconsistencies vlv index keys inconsistencies Jan 10, 2024
@progier389
Copy link
Contributor Author

Inconsistency is due to the fact that when reindexing, vlv index sub db is not cleared before rebuilding the keys

@progier389 progier389 linked a pull request Jan 10, 2024 that will close this issue
@progier389 progier389 removed the needs triage The issue will be triaged during scrum label Jan 10, 2024
progier389 added a commit that referenced this issue Jan 11, 2024
* Issue 6028 - Inconsistency among vlv keys
The issue is that reindexed vlv database are not cleared, so old keys remains
Solution: truncate the reindexed vlv sub database and its cache before starting the import engine.
Note: this is tested by: dirsrvtests/tests/suites/vlv/regression_test.py::test_vlv_cache_subdb_names CI test

Issue #6028

Reviewed by: @droideck (Thanks!)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lmdb LMDB related
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants