Make creation of extraction opportunities faster. #96
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changed algorithm of creation of extraction opportunities to speed it up.
Now algorithm is working in the following way:
Applying new version of algorithm we get the following gain:
InternalMetaDataParser
with 1721 methods the average speed up ofcreate_extraction_opportunities
step was 88.6% or 0.0086 seconds. The total time saved on that step is 14.8 seconds. The total processing of this file with SEMI algorithm takes 2.5 minutes.TomlParser
with 87 methods the average speed up ofcreate_extraction_opportunities
step was 68.3% or 0.0052 seconds. The total time saved on that step is 0.45 seconds. The total processing of this file with SEMI algorithm takes 7 seconds.The relative speed up us quite good, while in absolute numbers it is quite irrelevant.
Further speeding up the algorithms might be done through seeding up other steps and, may be,
ast framework
.Here is comparison of time taken by
create_extraction_opportunities
to other steps.old version
new version
old version
new version