Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance improvements for referential integrity checking #980

Merged
merged 4 commits into from
Aug 28, 2024

Commits on Aug 23, 2024

  1. performance improvement

    Step one: removed the tallest tent pole by fixing the ValidationTarget cache. Not everything was being cached. In fact, little to none of it was. Implementing this sped up the final step that counts on this by 10x. The 11000 strong bundle went from 73 seconds to 7.3 seconds. The overall time remains fixed at 1500 seconds. Not sure if the 73 seconds were simply redistributed or if that is just normal variation of an uncertain process (depends on network download times).
    Al Niessner authored and Al Niessner committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    49dd150 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. correct identifier

    Al Niessner authored and Al Niessner committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    e71890e View commit details
    Browse the repository at this point in the history
  2. fix up collections with Identifier

    Al Niessner authored and Al Niessner committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    b950791 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. clean up last bits

    TargetExaminer does not need to read the same file over and over. Do it once, collecting the information needed and then store for later. Yes, another cache.
    
    The look in referential integrity was overly complicated. Reduced it to a HashSet.contains() which really speeds things up.
    Al Niessner authored and Al Niessner committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    6244bb6 View commit details
    Browse the repository at this point in the history