Skip to content

Commit

Permalink
feat: increase maximum message size (#61)
Browse files Browse the repository at this point in the history
A batch can contain up to 1024 rows. If the columns contain large
amounts of data (json blogs or free text), the total batch size can
exceed the max message size. By doubling this we should be able to
accommodate most use cases.

TODO: if we hear of people reaching this max, maybe we should make it
configurable.
  • Loading branch information
aiguofer authored Dec 13, 2024
1 parent f5e4eb2 commit 07706df
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 0 deletions.
3 changes: 3 additions & 0 deletions .changes/unreleased/Features-20241213-104739.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
kind: Features
body: Double the max message size
time: 2024-12-13T10:47:39.843647-07:00
2 changes: 2 additions & 0 deletions dbtsl/api/adbc/client/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ def _extra_db_kwargs(cls) -> Dict[str, str]:
return {
DatabaseOptions.WITH_COOKIE_MIDDLEWARE.value: "true",
f"{DatabaseOptions.RPC_CALL_HEADER_PREFIX.value}user-agent": env.PLATFORM.user_agent,
# double the default max msg size in case of queries with large batches
DatabaseOptions.WITH_MAX_MSG_SIZE.value: f"{1024 * 1024 * 32}",
}

def __init__( # noqa: D107
Expand Down

0 comments on commit 07706df

Please sign in to comment.