Skip to content

Add performance testing start #220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 48 commits into
base: master
Choose a base branch
from
Open

Add performance testing start #220

wants to merge 48 commits into from

Conversation

ArBridgeman
Copy link
Collaborator

@ArBridgeman ArBridgeman commented Aug 15, 2025 •
edited
Loading

closes #211

Checklist

Note: If any of the items in the checklist are not relevant to your PR, leave the box unchecked.

For any Pull Request

Is the following correct:

  • the title of the Pull Request?
  • the title of the corresponding issue?
  • there are no other open Pull Requests for the same update/change?
  • that the issue which this Pull Request fixes ("Fixes...") is mentioned?

When Changes Were Made

Did you:

  • update the changelog?
  • update the implementation?
  • check coverage and add tests: unit tests and, if relevant, integration tests?
  • update the User Guide & other documentation?
  • resolve any failing CI criteria (incl. Sonar quality gate)?

When Preparing a Release

Have you:

  • thought about version number (major, minor, patch)?
  • checked Exasol packages for updates and resolved open vulnerabilities, if easily possible?

ArBridgeman and others added 2 commits August 11, 2025 13:30
@ArBridgeman @tkilias
Feature/208 add import from parquet (#209)
d316100 
* Add parquet import and update documentation

---------

Co-authored-by: Torsten Kilias <[email protected]>
@ArBridgeman
Merge branch 'master' into develop
b49265e
ArBridgeman
ArBridgeman commented Aug 15, 2025
View reviewed changes
tkilias
tkilias reviewed Aug 15, 2025
View reviewed changes
@ArBridgeman ArBridgeman force-pushed the feature/211-benchmark-test-import-from-parquet branch from 480285c to 833dafe Compare August 18, 2025 07:11
@ArBridgeman ArBridgeman force-pushed the feature/211-benchmark-test-import-from-parquet branch from 3396fab to 5371a3e Compare August 25, 2025 12:07
tkilias
tkilias reviewed Aug 26, 2025
View reviewed changes
test/performance/.benchmarks/0001_performance.json Outdated
1.3755199900000434,
2.322247650999998,
2.1706722630000286,
7.116984646999981,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The imports outlier's don't look random, mostly the round before the last is slow. Maybe we force a flush to disk of the uncommitted data at this point, because we don't cleanup.

tkilias
tkilias reviewed Aug 26, 2025
View reviewed changes
test/performance/.benchmarks/0001_performance.json Outdated
5.422284263999927,
5.473687115000075,
10.43289915999992,
5.8234761429999935
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again

tkilias
tkilias reviewed Aug 26, 2025
View reviewed changes
test/performance/.benchmarks/0001_performance.json Outdated
6.998409852999998,
7.840330088999963,
7.807797928000014,
12.746238112000015,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again

tkilias
tkilias reviewed Aug 26, 2025
View reviewed changes
test/performance/.benchmarks/0001_performance.json Outdated
2.009181237000007,
2.81157267399999,
2.7745194739999306,
7.796403220000002,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again

tkilias
tkilias reviewed Aug 26, 2025
View reviewed changes
tkilias
tkilias reviewed Aug 26, 2025
View reviewed changes
test/performance/connection/http_transport_import_test.py Outdated
import_params={"columns": ["SALES_TIMESTAMP", "PRICE", "CUSTOMER_NAME"]},
)

benchmark.pedantic(func_to_be_measured, iterations=1, rounds=benchmark_specs.rounds)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw a setup or teardown parameter

setup (callable) –

A function to call right before calling the target function in the first iteration of every round.

teardown (callable) –

A function to call after the last iteration of every round.

https://share.google/HYkd8TJRkLTfIVe8H

Because we use iterations 1 and multiple rounds, we should be able to cleanup for each measurement

tkilias
tkilias reviewed Sep 7, 2025
View reviewed changes
@ArBridgeman ArBridgeman force-pushed the develop branch 2 times, most recently from 1428eb4 to 6eed5a1 Compare September 15, 2025 08:19
Base automatically changed from develop to master September 15, 2025 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tkilias tkilias tkilias left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ArBridgeman @tkilias