Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rust/cardano-chain-follower): add thread and mmap file stats #150

Merged
merged 25 commits into from
Jan 21, 2025
Merged
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
7f2b8f0
feat(cardano-chain-follower): add thread and mmap file stats
bkioshn Jan 15, 2025
8fcaacb
fix(cardano-chain-follower): remove log
bkioshn Jan 15, 2025
c9d478e
fix(cardano-chain-follower): fix mmapfile name
bkioshn Jan 15, 2025
1bf8f75
fix(cardano-chain-follower): thread stat call
bkioshn Jan 15, 2025
3cc0d46
fix(cardano-chain-follower): thread logic
bkioshn Jan 15, 2025
8edb23f
fix(cardano-chain-follower): thread stat call
bkioshn Jan 15, 2025
476e131
fix(cardano-chain-follower): update forever task thread stat
bkioshn Jan 15, 2025
329455f
fix(cardano-chain-follower): linter and format
bkioshn Jan 15, 2025
2edc0e3
fix(cardano-chain-follower): thread stat name constants to its own file
bkioshn Jan 16, 2025
7a06a97
Merge branch 'main' into feat/thread_mmap_stats
stevenj Jan 16, 2025
6b68b3c
fix(cardano-chain-follower): thread stat calling
bkioshn Jan 17, 2025
d105c6c
fix(cardano-chain-follower): move mmap file to cat-types
bkioshn Jan 17, 2025
5276b16
fix(cardano-blockchain-types): format
bkioshn Jan 19, 2025
68dfa90
fix(cardano-chain-follower): remove unused dep
bkioshn Jan 19, 2025
4a6ab32
fix(cardano-chain-follower): thread stat calling
bkioshn Jan 19, 2025
7968e86
fix: mmapfile
bkioshn Jan 19, 2025
454c37e
Merge branch 'main' into feat/thread_mmap_stats
bkioshn Jan 19, 2025
d156158
Merge branch 'main' into feat/thread_mmap_stats
stevenj Jan 20, 2025
60960df
fix(cardano-blockchain-types): use RwLock for mmap stat
bkioshn Jan 20, 2025
56bb35a
fix(cardano-blockchain-types): add size function
bkioshn Jan 20, 2025
60c347e
fix(catalyst-types): mmap file error and structure
bkioshn Jan 21, 2025
6b1f1fb
fix(cardano-chain-follower): stat visibility
bkioshn Jan 21, 2025
8ba010f
fix(cardano-chain-follower): thread stat naming
bkioshn Jan 21, 2025
69479f4
fix(cardano-chain-follower): thread stat worker
bkioshn Jan 21, 2025
4aa61cf
Merge branch 'main' into feat/thread_mmap_stats
stevenj Jan 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix(cardano-chain-follower): thread stat calling
Signed-off-by: bkioshn <[email protected]>
bkioshn committed Jan 19, 2025

Verified

This commit was signed with the committer’s verified signature.
bkioshn bkioshn
commit 4a6ab32eaba730f32e0e003494ce59ab417eb7a8
2 changes: 0 additions & 2 deletions rust/cardano-chain-follower/src/stats/thread/name.rs
Original file line number Diff line number Diff line change
@@ -18,5 +18,3 @@ pub(crate) const VALIDATE_MITHRIL_SNAPSHOT: &str = "ValidateMithrilSnapshot";
pub(crate) const MITHRIL_DL_DEDUP: &str = "MithrilDlDedup";
/// Parallel Download Processor Worker.
pub(crate) const PARALLEL_DL_WORKER: &str = "ParallelDlWorker";
/// Parallel Download Processor Get Content Length.
pub(crate) const PARALLEL_DL_GET_CONTENT_LENGTH: &str = "ParallelDlGetContentLength";
17 changes: 3 additions & 14 deletions rust/cardano-chain-follower/src/turbo_downloader/mod.rs
Original file line number Diff line number Diff line change
@@ -375,7 +375,7 @@ impl ParallelDownloadProcessor {
MIN_CHUNK_SIZE
);
}
let file_size = get_content_length_async(url, chain).await?;
let file_size = get_content_length_async(url).await?;

// Get the minimum number of workers we need, just in case the chunk size is bigger than
// the requested workers can process.
@@ -701,20 +701,9 @@ impl std::io::Read for ParallelDownloadProcessor {
///
/// This exists because the `Probe` call made by Mithril is Async, and this makes
/// interfacing to that easier.
async fn get_content_length_async(url: &str, chain: Network) -> anyhow::Result<usize> {
async fn get_content_length_async(url: &str) -> anyhow::Result<usize> {
let url = url.to_owned();
match tokio::task::spawn_blocking(move || {
stats::start_thread(
chain,
stats::thread::name::PARALLEL_DL_GET_CONTENT_LENGTH,
false,
);
let result = get_content_length(&url);
stats::stop_thread(chain, stats::thread::name::PARALLEL_DL_GET_CONTENT_LENGTH);
result
})
.await
{
match tokio::task::spawn_blocking(move || get_content_length(&url)).await {
Ok(result) => result,
Err(error) => {
error!("get_content_length failed");