Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

increase insert xxx select xxx speed #461

Conversation

osdba
Copy link

@osdba osdba commented Nov 24, 2023

when increase polar_bulk_read_size to 64, but "insert xxx select" not getting faster, because ring_size is low,so increase ring_size in function GetAccessStrategy.

test:

postgres=# reset polar_bulk_read_size;
RESET
Time: 0.200 ms
postgres=# show polar_bulk_read_size;
 polar_bulk_read_size
----------------------
 128kB
(1 row)

Time: 0.216 ms
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 96621.768 ms (01:36.622)

postgres=# set polar_bulk_read_size to 64;
SET
postgres=# set max_parallel_workers_per_gather to 0;
SET
postgres=# \timing
Timing is on.
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 73271.417 ms (01:13.271)

postgres=# set max_parallel_workers_per_gather to 8;
SET
Time: 0.173 ms
postgres=# explain select count(*) from test2;
                                         QUERY PLAN
---------------------------------------------------------------------------------------------
 Finalize Aggregate  (cost=9009314.43..9009314.44 rows=1 width=8)
   ->  Gather  (cost=9009313.60..9009314.41 rows=8 width=8)
         Workers Planned: 8
         ->  Partial Aggregate  (cost=9008313.60..9008313.61 rows=1 width=8)
               ->  Parallel Seq Scan on test2  (cost=0.00..8945812.48 rows=25000448 width=0)
(5 rows)

Time: 0.454 ms
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 92410.190 ms (01:32.410)

postgres=# set polar_bulk_read_size to 64;
SET
Time: 0.168 ms
postgres=# show polar_bulk_read_size;
 polar_bulk_read_size
----------------------
 512kB
(1 row)

Time: 0.154 ms
postgres=# select count(*) from test2;
   count
-----------
 200000000
(1 row)

Time: 75722.028 ms (01:15.722)

@polardb-bot
Copy link

polardb-bot bot commented Nov 24, 2023

Hi @osdba ~ Thanks for your contribution in this PR. ❤️

Please make sure that your PR conforms the standard, and has passed all the checks.

We will review your PR as soon as possible.

@polardb-bot
Copy link

polardb-bot bot commented Nov 24, 2023

Hey @osdba :

Congratulations~ 🎉 Your commit has passed all the checks. Please wait for further manual review.

@polardb-bot polardb-bot bot added the ci/success CI status is success label Nov 24, 2023
@mrdrivingduck
Copy link
Member

/close

@polardb-bot polardb-bot bot closed this Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/success CI status is success
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants