Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: crud insert many #282

Merged
merged 9 commits into from
Mar 19, 2024
Merged

refactor: crud insert many #282

merged 9 commits into from
Mar 19, 2024

Conversation

flower-of-the-bridges
Copy link
Member

Pull Request Type

  • Bugfix
  • Feature
  • Code style update (formatting, local variables)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • CI related changes
  • Documentation content changes
  • Other... Please describe:

Description

This PR improves the performances of the POST /bulk operation, by avoiding to create multiple copies of the batch that has to be inserted.

PR Checklist

  • The commit message follows our guidelines included in the CONTRIBUTING.md
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)

Does this PR introduce a breaking change?

  • Yes
  • No

Other information

A k6 test has been added on the bench/scripts folder, named bulk-test.js. In this test, 10k records are inserted in batches into the records folder, using up to 80 concurrent requests.

This is the outcome for the version from the current branch:

✓ checks.........................: 100.00% ✓ 14       ✗ 0   
data_received..................: 4.9 MB  18 kB/s
data_sent......................: 65 MB   235 kB/s
http_req_blocked...............: avg=180.18ms min=6.55µs   med=141.76ms max=499.23ms p(90)=416.23ms p(95)=449.42ms
http_req_connecting............: avg=179.51ms min=0s       med=141.63ms max=498.16ms p(90)=414.17ms p(95)=447.17ms
http_req_duration..............: avg=2.29s    min=19.39ms  med=1.96s    max=5.56s    p(90)=4.1s     p(95)=4.73s   
  { expected_response:true }...: avg=2.29s    min=19.39ms  med=1.96s    max=5.56s    p(90)=4.1s     p(95)=4.73s   
✓ { type:bulk }................: avg=2.46s    min=787.93ms med=2.2s     max=5.56s    p(90)=4.16s    p(95)=4.79s   
✓ http_req_failed................: 0.00%   ✓ 0        ✗ 14  
http_req_receiving.............: avg=220.31ms min=910.4µs  med=4.75ms   max=2s       p(90)=519.01ms p(95)=1.07s   
http_req_sending...............: avg=993.43ms min=42.51µs  med=701.65ms max=3.56s    p(90)=2.33s    p(95)=2.85s   
http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
http_req_waiting...............: avg=1.08s    min=17.93ms  med=1.15s    max=1.62s    p(90)=1.46s    p(95)=1.52s   
http_reqs......................: 14      0.050999/s
iteration_duration.............: avg=47.84s   min=332.17µs med=34.37s   max=3m3s     p(90)=1m39s    p(95)=2m6s    
iterations.....................: 13      0.047356/s
vus............................: 0       min=0      max=50
vus_max........................: 50      min=50     max=50

@danibix95 danibix95 merged commit 820261d into main Mar 19, 2024
10 of 11 checks passed
@danibix95 danibix95 deleted the improve/crud-insert-many branch March 19, 2024 09:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants