Skip to content

Commit

Permalink
Improve performance due to slow indexing (#273)
Browse files Browse the repository at this point in the history
Newer versions of tantivy require more memory during the indexing phase.
Otherwise the indexing phase will be a lot slower than in previous
versions.

See quickwit-oss/tantivy#2156 for details.
  • Loading branch information
martinmr authored Sep 18, 2023
1 parent 3641fe8 commit 3c09922
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 2 deletions.
4 changes: 2 additions & 2 deletions src/course_library.rs
Original file line number Diff line number Diff line change
Expand Up @@ -550,8 +550,8 @@ impl LocalCourseLibrary {
reader: None,
};

// Initialize the search index writer with an initial arena size of 50 MB.
let mut index_writer = library.index.writer(50_000_000)?;
// Initialize the search index writer with an initial arena size of 150 MB.
let mut index_writer = library.index.writer(150_000_000)?;

// Convert the list of paths to ignore into absolute paths.
let absolute_root = library_root.canonicalize()?;
Expand Down
4 changes: 4 additions & 0 deletions src/scheduler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -746,11 +746,15 @@ impl DepthFirstScheduler {
if let Some(superseded_by) = superseded_by {
all_superseded_by.extend(superseded_by);
}

let superseded_by = self.data.get_superseded_by(&candidate.course_id);
if let Some(superseded_by) = superseded_by {
all_superseded_by.extend(superseded_by);
}
}
if all_superseded_by.is_empty() {
return candidates;
}

// Filter out the superseding lessons and courses with scores lower than the superseding
// score.
Expand Down

0 comments on commit 3c09922

Please sign in to comment.