Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query execution updates for cpu metrics #2130

Merged
merged 1 commit into from
Jan 22, 2025

Conversation

joshua-spacetime
Copy link
Collaborator

Description of Changes

This patch is a prerequisite for computing query cpu metrics.

Why is a new executor needed?

Tuple at a time iterators are not well suited to gathering per operator metrics.

API and ABI breaking changes

None

Expected complexity level and risk

2

Adding new query executors is relatively straightforward since they work off of the same physical plan. We can rely on our current test coverage to catch any issues.

Testing

Describe any testing you've done, and any testing you'd like your reviewers to do,
so that you're confident that all the changes work as expected!

No new tests have been added. We already have correctness tests for the query and subscription code that are executor agnostic.

@joshua-spacetime joshua-spacetime force-pushed the joshua/materialized-query-execution branch 2 times, most recently from 89276dc to bc1fcdc Compare January 17, 2025 20:28
@joshua-spacetime joshua-spacetime force-pushed the joshua/materialized-query-execution branch from bc1fcdc to 816f481 Compare January 17, 2025 22:23
@joshua-spacetime joshua-spacetime self-assigned this Jan 17, 2025
@joshua-spacetime joshua-spacetime force-pushed the joshua/materialized-query-execution branch from 816f481 to 8a36603 Compare January 17, 2025 22:47
Copy link
Contributor

@mamcx mamcx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@joshua-spacetime joshua-spacetime force-pushed the joshua/materialized-query-execution branch 2 times, most recently from c33250b to 22af59b Compare January 21, 2025 06:39
@joshua-spacetime joshua-spacetime changed the title Introduce a pipelined materializing query executor A pipelined materializing query executor Jan 21, 2025
@joshua-spacetime joshua-spacetime added this pull request to the merge queue Jan 21, 2025
@joshua-spacetime joshua-spacetime removed this pull request from the merge queue due to a manual request Jan 21, 2025
Fully evaluates each operator before evaluating downstream ones.

This is in contrast to a tuple at a time iterator,
where each row is fully processed by all operators,
before processing the next row.

It is called "materializing" because each operator materializes its output.
In practice however, non-blocking operators will be pipelined,
so as not to perform unnecessary materialization.
@joshua-spacetime joshua-spacetime force-pushed the joshua/materialized-query-execution branch from 5ebb2c6 to cc4f397 Compare January 22, 2025 23:09
@joshua-spacetime joshua-spacetime changed the title A pipelined materializing query executor Executor for query cpu metrics Jan 22, 2025
@joshua-spacetime joshua-spacetime changed the title Executor for query cpu metrics Query execution updates for cpu metrics Jan 22, 2025
@joshua-spacetime joshua-spacetime added this pull request to the merge queue Jan 22, 2025
Merged via the queue into master with commit 149ff97 Jan 22, 2025
11 checks passed
@joshua-spacetime joshua-spacetime deleted the joshua/materialized-query-execution branch January 22, 2025 23:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants