Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to RocksDB #2098

Merged
merged 14 commits into from
Apr 9, 2024
Merged

Migrate to RocksDB #2098

merged 14 commits into from
Apr 9, 2024

Conversation

jellymlg
Copy link
Collaborator

Migrate from LevelDB to RocksDB

Closes #2096

@kushti kushti changed the base branch from master to v5.1.0 February 16, 2024 21:25
@jellymlg jellymlg requested a review from kushti February 18, 2024 12:51
@ccellado
Copy link
Collaborator

Running sbt test failed for me

j1px2g_p2fy25w0000gn/T/z9gL6Plw2N1692963984190461516/history/extra exists and is not used by some other active node
[debug] Summary for ScalaCheck not available.
[info] ScalaTest
[info] Run completed in 14 minutes, 38 seconds.
[info] Total number of tests run: 467
[info] Suites: completed 57, aborted 0
[info] Tests: succeeded 467, failed 0, canceled 0, ignored 3, pending 0
[info] All tests passed.
[error] Error: Total 0, Failed 0, Errors 0, Passed 0
[error] Error during tests:
<long java command>
[error] (Test / test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 881 s (14:41), completed Feb 27, 2024, 5:41:42 PM

@jellymlg
Copy link
Collaborator Author

Running sbt test failed for me

j1px2g_p2fy25w0000gn/T/z9gL6Plw2N1692963984190461516/history/extra exists and is not used by some other active node
[debug] Summary for ScalaCheck not available.
[info] ScalaTest
[info] Run completed in 14 minutes, 38 seconds.
[info] Total number of tests run: 467
[info] Suites: completed 57, aborted 0
[info] Tests: succeeded 467, failed 0, canceled 0, ignored 3, pending 0
[info] All tests passed.
[error] Error: Total 0, Failed 0, Errors 0, Passed 0
[error] Error during tests:
<long java command>
[error] (Test / test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 881 s (14:41), completed Feb 27, 2024, 5:41:42 PM

All tests run fine for me, maybe try clean build with cleared Temp folder

@ccellado
Copy link
Collaborator

ccellado commented Feb 29, 2024

Running sbt test failed for me

j1px2g_p2fy25w0000gn/T/z9gL6Plw2N1692963984190461516/history/extra exists and is not used by some other active node
[debug] Summary for ScalaCheck not available.
[info] ScalaTest
[info] Run completed in 14 minutes, 38 seconds.
[info] Total number of tests run: 467
[info] Suites: completed 57, aborted 0
[info] Tests: succeeded 467, failed 0, canceled 0, ignored 3, pending 0
[info] All tests passed.
[error] Error: Total 0, Failed 0, Errors 0, Passed 0
[error] Error during tests:
<long java command>
[error] (Test / test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 881 s (14:41), completed Feb 27, 2024, 5:41:42 PM

All tests run fine for me, maybe try clean build with cleared Temp folder

I've used sbt clean and still unsuccessful

master branch sbt test works fine on same settings same machine

What Temp folder are you proposing then?

System is Macos m1

@ccellado
Copy link
Collaborator

ccellado commented Feb 29, 2024

Tested on linux-vm, first got OOM, increased memory tests went ok

@ccellado
Copy link
Collaborator

ccellado commented Feb 29, 2024

Running sbt test failed for me

j1px2g_p2fy25w0000gn/T/z9gL6Plw2N1692963984190461516/history/extra exists and is not used by some other active node
[debug] Summary for ScalaCheck not available.
[info] ScalaTest
[info] Run completed in 14 minutes, 38 seconds.
[info] Total number of tests run: 467
[info] Suites: completed 57, aborted 0
[info] Tests: succeeded 467, failed 0, canceled 0, ignored 3, pending 0
[info] All tests passed.
[error] Error: Total 0, Failed 0, Errors 0, Passed 0
[error] Error during tests:
<long java command>
[error] (Test / test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 881 s (14:41), completed Feb 27, 2024, 5:41:42 PM

All tests run fine for me, maybe try clean build with cleared Temp folder

I've used sbt clean and still unsuccessful

master branch sbt test works fine on same settings same machine

What Temp folder are you proposing then?

System is Macos m1

There is a problem with RocksDB and ulimit -n on MacOS which I couldn't fix still
It could potentially be a problem with node on Mac, Windows need testing too

@jellymlg
Copy link
Collaborator Author

Running sbt test failed for me

j1px2g_p2fy25w0000gn/T/z9gL6Plw2N1692963984190461516/history/extra exists and is not used by some other active node
[debug] Summary for ScalaCheck not available.
[info] ScalaTest
[info] Run completed in 14 minutes, 38 seconds.
[info] Total number of tests run: 467
[info] Suites: completed 57, aborted 0
[info] Tests: succeeded 467, failed 0, canceled 0, ignored 3, pending 0
[info] All tests passed.
[error] Error: Total 0, Failed 0, Errors 0, Passed 0
[error] Error during tests:
<long java command>
[error] (Test / test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 881 s (14:41), completed Feb 27, 2024, 5:41:42 PM

All tests run fine for me, maybe try clean build with cleared Temp folder

I've used sbt clean and still unsuccessful
master branch sbt test works fine on same settings same machine
What Temp folder are you proposing then?
System is Macos m1

There is a problem with RocksDB and ulimit -n on MacOS which I couldn't fix still It could potentially be a problem with node on Mac, Windows need testing too

I am not well versed with MacOS, but it works fine on Windows. What is the problem with ulimit and RocksDB?

@ccellado
Copy link
Collaborator

ccellado commented Mar 1, 2024

Test run fails with macos-runner, so it looks like a specific problem with macos and RocksDB

The logs show max open files as the culprit, it is known to cause problems with RocksDB in different scenarios

@kushti kushti merged commit 49df825 into v5.1.0 Apr 9, 2024
3 of 4 checks passed
@kushti kushti deleted the rocksdb branch April 9, 2024 09:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migrate to RocksDB
3 participants