diff --git a/docs/concepts/index.rst b/docs/concepts/index.rst index 25fc88da..f4aaba77 100644 --- a/docs/concepts/index.rst +++ b/docs/concepts/index.rst @@ -24,5 +24,5 @@ Overview of the concepts -- read these to get a mental model for how Burr works. additional-visibility parallelism recursion - planned-capabilities sync-vs-async + planned-capabilities diff --git a/docs/concepts/sync-vs-async.rst b/docs/concepts/sync-vs-async.rst index 96b8eaa2..ba1f68d3 100644 --- a/docs/concepts/sync-vs-async.rst +++ b/docs/concepts/sync-vs-async.rst @@ -4,36 +4,53 @@ Sync vs Async Applications TL;DR ------ -1. For applications with horizontal scaling: multiple users, lots of i/o throughput,... go with async. Check out: + +Burr gives you the ability to write synchronous (standard python) and asynchronous (``async def``/``await``) Burr applications. You then run these applications in some sort of python context (e.g. script, web-service, aws lambda, etc). Whether you choose to write your Burr application using Burr's synchronous or asynchronous features depends on where you plan to run your Burr application. At a high level: + +1. Use the `async` interfaces when you have I/O-heavy applications that require horizontal scaling, and have avaialble asynchronous APIs (E.G. async LLM APIs in a web-service like FastAPI) * :py:meth:`.abuild() <.ApplicationBuilder.abuild()>` * :py:meth:`.aiterate() <.Application.aiterate()>` * :py:meth:`.arun() <.Application.arun()>` * :py:meth:`.astream_result() <.Application.astream_result()>` -2. For prototyping applications or high stakes operations,... go with sync.Check out: +2. Use the synchronous interfaces otherwise -- when you have high CPU-bound applications (running models locally), or do not have async APIs to use: * :py:meth:`.build() <.ApplicationBuilder.build()>` * :py:meth:`.iterate() <.Application.iterate()>` * :py:meth:`.run() <.Application.run()>` * :py:meth:`.stream_result() <.Application.stream_result()>` +Comparison +---------- + +A synchronous Python application processes tasks sequentially for every thread/process it executes, blocking entirely on the result +of the prior call. When using Burr, this means that two (or more) separate Burr applications, if they are to run concurrently, have to be run in separate threads/processes which you manage / control. + +Specifically for Burr, this means that you have a 1:1 app -> thread/process mapping (unless you're using :ref:`parallelism ` and explicitly multithreading sub-actions). -Burr does both sync and async ------------------------------- -Bellow we give a short breakdown when to consider each case. +An asynchronous application can parallelize multiple I/O bound tasks within the confines of a single thread. At the time that +a task blocks on I/O it can give control back to the process, allow it to run other tasks simultaneously. -In general, Burr is equipped to handle both synchronous and asynchronous runs. We usually do that by +In the case of Burr, Burr supports this model in running multiple Burr applications, in parallel, on the same as thread (i.e. the asyncio event loop). +Note, however, that you have to ensure your Burr application is async all the way down -- E.G. that every blocking call +is called using `await` -- if it blocks the event loop through a slow, synchronous call, it will block *all* current +applications. + +In general, Burr gives you the constructs for synchronous and asynchronous execution. We usually do that by providing both methods (see specific references for more detail and reach out if you feel like we -are missing a specific implementation). +are missing a specific implementation). Furthermore, Burr suports the following APIs for both synchronous/asynchronous interfaces: + +- :ref:`hooks ` +- :ref:`persisters ` -We hope the switch from one to another is as convenient as possible; you only need to substitute -functions/adapters/method calls. +Nuances of Sync + Async together +-------------------------------- -We highly encourage to make a decision to either commit fully to sync or async. Having said that, +We encourage to make a decision to either commit fully to sync or async. That said, there are cases where a hybrid situation may be desirable or unavoidable (testing, prototyping, legacy code, ...) and we give some options to handle that. The table bellow shows the -possibilities Burr now supports. +possibilities Burr now supports -- combining the set of synchronous/asynchronous. .. table:: Cases Burr supports @@ -73,67 +90,3 @@ possibilities Burr now supports. | | | should really use the sync | | | | builder | +------------------------------------------------+----------+----------------------------------+ - - -Synchronous Applications --------------------------- -A synchronous application processes tasks sequentially, where the user or calling system must wait for -the agent to finish a task before proceeding. - -Advantages -~~~~~~~~~~~~ - -1. Simplicity: - * Easier to design and implement as the logic flows linearly. - * Debugging and maintenance are more straightforward. -2. Deterministic Behavior: - * Results are predictable and occur in a defined sequence. - * Ideal for workflows where steps must be completed in strict order. -3. Low Overhead: - * Useful for tasks that don’t require extensive multitasking or background processing. - * Rapid prototyping. - - -Use Cases -~~~~~~~~~~~ - -1. Short, straightforward tasks: - * For example, fetching a single database entry or making a quick calculation. -2. High-stakes operations: - * When the process must proceed step-by-step without the risk of race conditions or overlapping tasks. -3. Interactive applications: - * Situations where the user must receive immediate feedback before taking the next step, like form validations. - -Asynchronous Application --------------------------- -An asynchronous application can perform tasks in parallel or handle multiple requests without waiting for -one to finish before starting another. - -Advantages -~~~~~~~~~~~~ - -1. Efficiency: - * Makes better use of system resources by handling multiple tasks simultaneously. - * Reduces idle time, especially when dealing with I/O-bound or network-bound operations. -2. Scalability: - * Handles large volumes of concurrent tasks more effectively. - * Useful in systems requiring high throughput, like web servers or chatbots. -3. Non-blocking Execution: - * Allows other operations to continue while waiting for longer processes to complete. - * Provides a smoother experience in real-time systems. - - -Use Cases -~~~~~~~~~~~~ - -1. Long-running processes: - * Training machine learning models. - * Data processing pipelines. -2. I/O-bound tasks: - * Calling APIs. - * Retrieving data from remote servers or databases. -3. High-concurrency systems: - * Chatbots serving multiple users. - * Customer support systems. -4. Background processing: - * Notifications, logs, and analytics tasks running while the main application continues. diff --git a/docs/reference/persister.rst b/docs/reference/persister.rst index 3f7ee868..e6068023 100644 --- a/docs/reference/persister.rst +++ b/docs/reference/persister.rst @@ -29,6 +29,30 @@ Internally, this interface combines the following two interfaces: .. automethod:: __init__ +Note we also have the corresponding async implementations: + +.. autoclass:: burr.core.persistence.AsyncBaseStatePersister + :members: + :show-inheritance: + :inherited-members: + + .. automethod:: __init__ + +Internally, this interface combines the following two interfaces: + +.. autoclass:: burr.core.persistence.AsyncBaseStateLoader + :members: + + .. automethod:: __init__ + +.. autoclass:: burr.core.persistence.AsyncBaseStateSaver + :members: + :show-inheritance: + :inherited-members: + + .. automethod:: __init__ + + Supported Sync Implementations ================================