Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change tripleTemplate rule to an equivalent non-recursive definition #1816

Merged
merged 3 commits into from
Feb 18, 2025

Conversation

Qup42
Copy link
Member

@Qup42 Qup42 commented Feb 18, 2025

In the original SPARQL grammar, the tripleTemplate rule is defined recursively. This leads to stack overflows in the ANTLR-v4 based parser for large inputs, in particular large INSERT DATA requests. This PR changes the definition of the triplesTemplate to an equivalent formulation that is not recursive. This massively improves the performance of parsing large triple payloads. With this change we can process the whole olympics dataset as a single UPDATE request (1.78M triples, 323M).

To test this, curl with --data-binary can be used. Prepare the Update to execute into a file foo. Execute the query with curl --data-binary @foo .... Some more info on the different curl options.

Size Time before (s) Time after (s)
1781625 N/A 65
800000 N/A 27
200000 N/A 7.5
100000 N/A 4.2
92000 N/A 4.0
88000 N/A 3.8
84000 69 3.7
75000 59 3.4
50000 13 2.6
10000 1.7 1.3

Copy link

codecov bot commented Feb 18, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.14%. Comparing base (6c909fe) to head (727b2cb).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1816      +/-   ##
==========================================
+ Coverage   90.12%   90.14%   +0.02%     
==========================================
  Files         399      399              
  Lines       38200    38188      -12     
  Branches     4280     4280              
==========================================
- Hits        34429    34426       -3     
  Misses       2473     2473              
+ Partials     1298     1289       -9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@joka921 joka921 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only a small suggestion, That looks really promising.

@sparql-conformance
Copy link

@joka921 joka921 merged commit f13fafa into ad-freiburg:master Feb 18, 2025
23 of 24 checks passed
@Qup42 Qup42 deleted the nonRecursiveTriplesTemplate branch February 18, 2025 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants