Lazy multiprocess target extraction #730
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When using longer datasets or larger image resolutions, kalibr uses a large amount of memory during corner extraction.
That's because all inputs needed for the corner extraction (including all images in the dataset) are duplicated
numProcesses
times when each process forks.This PR simplifies the code and uses lazy multiprocessing to feed inputs to the process pool only when needed (one-by-one).
See a comparison below of the memory consumption before and after this PR with a dataset of 1920x1080 images:
Peak memory usage is reduced by more than 50%.
CPU usage is more or less the same - seems less "spiky" now:
I've also ran the code in the PR on the sample dataset to make sure there's no unintended degradation:
Command:
Camera chain before this PR:
Camera chain after this PR:
There's small difference but I think that's just the run-to-run variance.
Maybe this addresses #22, hard to say since that issue is very old.