Skip to content

Commit

Permalink
Update v3-roadmap-and-design.md
Browse files Browse the repository at this point in the history
Co-authored-by: Sanket Verma <[email protected]>
  • Loading branch information
jhamman and MSanKeys963 authored Jan 4, 2024
1 parent 80c528c commit ab3f781
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion v3-roadmap-and-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ With the `Store` and core `AsyncArray`/ `AsyncGroup` classes being predominantly
c. Splitting the two will allow us to clearly define behavior on the `AsyncObj` and simply wrap it in the `SyncObj`.
2. What if a store is only has a synchronous backend?
a. First off, this is expected to be a fairly rare occurrence. Most storage backends have async interfaces.
b. But in the even a storage backend doesn’t have a async interface, there is nothing wrong with putting synchronous code in `async` methods. There are approaches to enabling concurrent action through wrappers like AsyncIO's `loop.run_in_executor` ([ref 1](https://stackoverflow.com/questions/38865050/is-await-in-python3-cooperative-multitasking ), [ref 2](https://stackoverflow.com/a/43263397/732596), [ref 3](https://bbc.github.io/cloudfit-public-docs/asyncio/asyncio-part-5.html), [ref 4](https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor).
b. But in the event a storage backend doesn’t have a async interface, there is nothing wrong with putting synchronous code in `async` methods. There are approaches to enabling concurrent action through wrappers like AsyncIO's `loop.run_in_executor` ([ref 1](https://stackoverflow.com/questions/38865050/is-await-in-python3-cooperative-multitasking ), [ref 2](https://stackoverflow.com/a/43263397/732596), [ref 3](https://bbc.github.io/cloudfit-public-docs/asyncio/asyncio-part-5.html), [ref 4](https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor).
3. Will Zarr help manage the async contexts encouraged by some libraries (e.g. [AioBotoCore](https://aiobotocore.readthedocs.io/en/latest/tutorial.html#using-botocore))?
a. Many async IO libraries require entering an async context before interacting with the API. We expect some experimentation to be needed here but the initial design will follow something close to what fsspec does ([example in s3fs](https://github.com/fsspec/s3fs/blob/949442693ec940b35cda3420c17a864fbe426567/s3fs/core.py#L527)).
4. Why not provide a synchronous Store interface?
Expand Down

0 comments on commit ab3f781

Please sign in to comment.