perf(sqlite data): switch OR to a UNION in data attributes query PE-6806 #210
Conversation
The OR in the previous query was preventing effective index usage. This change modifies the query to use a UNION of lookup by ID and data root. That's a much simpler query for the optimizer to understand, and it uses indexes for both.
WalkthroughWalkthroughThe pull request modifies an SQL query in the Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant SQLQuery
participant ContiguousData
participant DataRoots
User->>SQLQuery: Request data attributes
SQLQuery->>ContiguousData: SELECT from contiguous_data where cdi.id matches
ContiguousData-->>SQLQuery: Return matching entry
SQLQuery->>DataRoots: SELECT from data_roots where dr.data_root matches
DataRoots-->>SQLQuery: Return matching entry
SQLQuery-->>User: Return combined results
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #210 +/- ##
========================================
Coverage 70.09% 70.09%
========================================
Files 32 32
Lines 7877 7877
Branches 429 429
========================================
Hits 5521 5521
Misses 2356 2356 ☔ View full report in Codecov by Sentry. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Outside diff range and nitpick comments (1)
src/database/sql/data/content-attributes.sql (1)
39-64: Excellent performance optimization!The restructuring of the
selectDataAttributesquery from using OR to UNION is a great improvement. This change allows for better index utilization, potentially enhancing query performance significantly. The structure is clear and aligns well with the PR objectives.Consider using UNION ALL for potential further optimization
If you can guarantee that the results from both subqueries will never overlap (i.e., no duplicates), or if duplicate results are acceptable, consider using UNION ALL instead of UNION. UNION ALL is generally faster as it doesn't need to perform duplicate elimination.
Here's how you could modify the query:
SELECT * FROM ( -- First subquery (unchanged) SELECT cd.hash, cd.data_size, cd.original_source_content_type, cdi.verified FROM contiguous_data cd JOIN contiguous_data_ids cdi ON cdi.contiguous_data_hash = cd.hash WHERE cdi.id = :id LIMIT 1 ) UNION ALL -- Changed from UNION to UNION ALL SELECT * FROM ( -- Second subquery (unchanged) SELECT cd.hash, cd.data_size, cd.original_source_content_type, cdi.verified FROM data_roots dr JOIN contiguous_data cd ON dr.contiguous_data_hash = cd.hash JOIN contiguous_data_ids cdi ON cdi.contiguous_data_hash = cd.hash WHERE dr.data_root = :data_root LIMIT 1 ) LIMIT 1This change could provide a slight performance boost, especially for larger datasets.
The OR in the previous query was preventing effective index usage. This change modifies the query to use a UNION of lookup by ID and data root. That's a much simpler query for the optimizer to understand, and it uses indexes for both.