RS-70: Optimize parsing batch records in Python SDK (#99)

* Ignore vscode config * Optimized batch read for better memory efficiency * Update changelog * Update changelog
reductstore · Dec 14, 2023 · 90ff139 · 90ff139
1 parent 0770840
commit 90ff139
Show file tree

Hide file tree

Showing 3 changed files with 6 additions and 3 deletions.
diff --git a/.gitignore b/.gitignore
@@ -3,3 +3,5 @@ reduct/VERSION
 *.egg-info
 .venv
 .gitignore
+*.pyc
+.vscode
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Changed:
 
 - docs: update link to new website, [PR-98](https://github.com/reductstore/reduct-py/pull/98)
+- Optimize batch read for better memory efficiency, [PR-99](https://github.com/reductstore/reduct-py/pull/99)
 
 ## [1.7.1] - 2023-10-09
 

diff --git a/reduct/record.py b/reduct/record.py
@@ -204,14 +204,14 @@ async def parse_batched_records(resp: ClientResponse) -> AsyncIterator[Record]:
 
 
 async def _read_response(resp, content_length):
-    buffer = b""
+    chunks = []
     count = 0
     while True:
         n = min(CHUNK_SIZE, content_length - count)
         chunk = await resp.content.read(n)
-        buffer += chunk
+        chunks.append(chunk)
         count += len(chunk)
 
         if count == content_length:
             break
-    return buffer
+    return b"".join(chunks)