Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added MetaStorage to replace MetadataMap #159

Merged
merged 3 commits into from
Jul 13, 2024
Merged

Added MetaStorage to replace MetadataMap #159

merged 3 commits into from
Jul 13, 2024

Conversation

LeeGodSRC
Copy link
Member

@LeeGodSRC LeeGodSRC commented Jul 12, 2024

Made a replacement for MetadataMap.
MetadataMap is proven to be extremely slow, saw it here and there in Spark Report.
MetaStorage were made to replace MetadataMap, even faster than ConcurrentHashMap in single thread operations and slightly worse in multi thread performance.

overall, MetaStorage is
400 times faster in single threaded,
162 times faster in multi threaded
over MetadataMap.

Benchmark                                               Mode  Cnt      Score      Error  Units
MetaStorageImplBenchmark.concurrentHashMap             thrpt   10  23842.312 ±  255.226  ops/s
MetaStorageImplBenchmark.concurrentHashMap_concurrent  thrpt   10    898.415 ±  517.752  ops/s
MetaStorageImplBenchmark.metaStorage                   thrpt   10  77488.548 ± 1049.076  ops/s
MetaStorageImplBenchmark.metaStorageConcurrent         thrpt   10    717.150 ±  414.137  ops/s
MetaStorageImplBenchmark.metadataMap                   thrpt   10    193.100 ±    3.899  ops/s
MetaStorageImplBenchmark.metadataMapConcurrent         thrpt   10      4.485 ±    1.442  ops/s

@LeeGodSRC LeeGodSRC added the enhancement New feature or request label Jul 12, 2024
chore: Replace MetadataMap to MetaStorage usage in fairy itself
@LeeGodSRC LeeGodSRC merged commit 68f1645 into v0.7 Jul 13, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request size/XXL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant