Releases: temporalio/sdk-python
1.6.0
Get from PyPI
Highlights
Safe Eviction
The internal eviction logic has been overhauled to make sure that asyncio tasks are completed before a workflow is removed from the cache and garbage collected. For all properly running workflows this should not cause any issues. However, workflows that may have been developed in invalid ways in the past could have a problem being evicted from cache and will log as such. Users should keep an eye out for logs when upgrading to this version.
Experimental Workflow Failure Type Customization
When a workflow raises an exception that wasn't ApplicationError
or bubbled from a Temporal call (e.g. activity failure), it would be a "task failure" which puts the workflow in a suspended state retrying the task until a code fix is deployed. An experimental feature is now available that lets users set which exception types should instead fail the workflow instead of suspending it. Users can now set the per-workflow failure_exception_types
on the @workflow.defn
decorator or set the worker-level workflow_failure_exception_types
when creating the Worker
. If an exception extends from one of these when raised, it will fail the workflow when raised.
HTTP CONNECT Proxy Support
Python clients now officially support HTTP CONNECT proxies. The http_connect_proxy_config
argument can now be set when connecting a Client
.
Worker Client Replacement
Workers can now have the client they use replaced without shutting down the worker. The client
property on the Worker
can be set to a different connected client. This is useful for users that need to change options like mTLS client certificate without worker downtime. The worker will not interrupt any polling calls but will start using the new client for subsequent calls.
API Key Client Option
If you use API keys in your Temporal server (i.e. Bearer
tokens on Authorization
headers), you can now set api_key
when connecting a Client
or set the api_key
property on an existing client to update it.
Floats and Durations in Metrics
Users using metric_meter()
directly to create metrics (off of runtime, activity, or workflow) can now create float-based histograms and gauges, and duration-based histograms. Similarly users using MetricBuffer
to retrieve metrics can say how they want to retrieve durations (float seconds, int milliseconds, or timedelta
). A durations_as_seconds
option was added to telemetry options for users that prefer seconds as float-based durations to OpenTelemetry and Prometheus instead of the default of integer milliseconds.
Counting Workflows
A new count_workflows
call has been added to the Client
that takes the same query as list_workflows
but is optimized just to perform counts.
💥 Activity/Workflow extra
on LogRecord
Changed
Activity logs and workflow logs used to set activity_info
and workflow_info
on LogRecord
respectively as the full info dataclasses by default. This was not usable by many third-party logging tools that can only work with dict
s and not dataclass
es. Therefore the default was changed to set temporal_activity
and temporal_workflow
on the respective log records to dict
s of commonly needed information. temporalio.activity.logger.full_activity_info_on_extra
and/or temporalio.workflow.logger.full_workflow_info_on_extra
can be set to True
to put those info entries back on extra
.
This only affects those using advanced log handlers that leverage contextual state. The log messages themselves are unaffected.
Specific Changes
2024-03-04 - 477aa31 - Fix execute_child_workflow apidoc (#483)
2024-03-13 - f3d1b85 - API key client option (#486)
2024-03-26 - 36fe961 - Remove experimental flag from start_delay (#492)
2024-04-01 - 13d18ca - Update core and add durations-as-seconds metric option (#498)
2024-04-02 - b07e75e - Change default "extra" contents of activity/workflow logs (#490)
2024-04-05 - 1001653 - HTTP CONNECT proxy support (#501)
2024-04-05 - 466da16 - Safe Eviction (#499)
2024-04-12 - b45447e - Add macOS ARM runner (#506)
2024-04-12 - cf4c7cb - Ensure extra data on task fail logs (#502)
2024-04-18 - 50c2033 - Support float and duration metrics (#508)
2024-04-19 - ecd703d - Add Client.count_workflows (#510)
2024-04-30 - 0bb94f8 - Ability for certain task failure types to fail workflow (#516)
2024-05-03 - 0687151 - Worker client replacement (#517)
1.5.1
Get from PyPI
Highlights
This is mostly a bug-fix and small-issue release. There are no large notable highlights.
Specific Changes
2024-01-10 - e3630ef - Fix line number in workflow.logger (#460)
2024-01-11 - 0bff022 - Add wheel
to dev-dependencies (#457)
2024-01-12 - 10d099b - Fix typing of methods for pyright (#461)
2024-01-12 - 4904054 - Add build id to workflow info (#458)
2024-01-29 - 02a974c - Modularize gen-protos script (#465)
2024-02-01 - 5fe85be - Prefix some errors with rule identifiers (#464)
2024-02-02 - 50768df - Apply eviction before completing activation (#466)
2024-02-13 - 8dce5b5 - Fix SingleParam types for pyright checking (#471)
2024-02-14 - b0a7b4e - Flag MD5 as non-security related usage for FIPS compatibility (#472)
2024-02-20 - 1cfdaa8 - Add workflow ID on not-found activity error message (#428)
2024-02-22 - f31f927 - Ensure update input and responses are run through converters (#478)
2024-02-27 - 73e832d - Update core, minor README/doc updates, and put delete-workflow on proper service (#480)
1.5.0
Get from PyPI
Highlights
Drop Python 3.7 support
Python 3.7 is EOL and is not supported by this library. In addition, we also added CI to confirm 3.12 works properly.
Fix upserting new search attributes
Last release, a bug was introduced that caused an error when upserting search attributes that weren't set on start. This has been fixed.
Specific Changes
2023-11-16 - 06cfd03 - Drop Python 3.7 support, ensure 3.12 support, update dependencies, and implement asyncio.timeout (#422)
2023-11-17 - 75e528b - Delete .github/workflows/semgrep.yml (#431)
2023-11-27 - c47e3f1 - Enable Eager Workflow Start (#430)
2023-11-30 - 6f966c7 - Fix bugs in upsert search attribs (#440)
1.4.0
Get from PyPI
Highlights
Typed Search Attributes
Previously, search attributes were untyped dictionaries with string keys and list values. This causes ambiguity on what types a search attribute key represents and leads to errors. Also, the way search attributes were created or upserted allowed mistakes to occur and was not clear to the server exactly what was expected.
Among other classes, temporalio.common.TypedSearchAttributes
and temporalio.common.SearchAttributeKey
have been added. Search attributes are now best used by creating a global SearchAttributeKey
(e.g. SearchAttributeKey.for_int("my-key")
) and then referencing that key when getting or mutating search attributes. The TypedSearchAttributes
collection (and workflow.typed_search_attributes()
) should now be used when creating/accessing search attributes or a warning will emitted. Users should use the keys when accessing the collection or call value_set
or value_unset
on them to build update objects to pass to temporalio.workflow.upsert_search_attributes
when upserting.
Advanced Metrics Support
Previously, metrics were generated and exposed from our Core layer in a way that made them neither reusable nor exportable in a flexible way. We have now both exposed the ability to record metrics via Core and to received buffered metrics from Core.
To record metrics via Core, we have exposed a temporalio.common.MetricMeter
via temporalio.activity.metric_meter()
, temporalio.workflow.metric_meter()
, and temporalio.runtime.Runtime.metric_meter
. This meter provides a limited interface to generate metrics. It is not meant to abstract everything one can do with a metric system and users are still encouraged to use their own metrics libraries as needed. The workflow metric meter is built to skip recording meters during replay.
To configure metrics in the SDK, a new runtime must be created with telemetry config. To capture metrics emitted by Core, temporalio.runtime.TelemetryConfig.metrics
now accepts a temporalio.runtime.MetricBuffer
instance. That same instance must then be used to call retrieve_updates
on repeatedly and frequently to get metrics coming out of Core. These come as raw events that can then filtered and/or sent to any other metrics system of choice.
Log Forwarding from Core (experimental)
To configure Core logging in the SDK, a new runtime must be created with telemetry config. Previously, Core logs would be sent to the console which wasn't always desirable. Now, the temporalio.runtime.LoggingConfig.forwarding
field can be set with a temporalio.runtime.LogForwardingConfig
class which accepts a logging.Logger
to send core logs to. This is currently experimental while we gather feedback.
Several Minor Additions
temporalio.workflow.Info.get_current_history_size()
added to get history size (get_current_history_length()
was already present)temporalio.workflow.Info.is_continue_as_new_suggested()
added to know whether server has suggested to continue as new- Client keep alive enabled by default (30s interval, 15s timeout), configurable via
keep_alive_config
when connecting - Added
start_delay
totemporalio.client.Client.start_workflow
andexecute_workflow
to support delaying a workflow start - Added experimental support for workflow update which is only available in bleeding edge open source server environments
💥 BREAKING CHANGES
ScheduleActionStartWorkflow.search_attributes
replaced with ScheduleActionStartWorkflow.typed_search_attributes
The rarely used ScheduleActionStartWorkflow.search_attributes
has been replaced by ScheduleActionStartWorkflow.typed_search_attributes
since that object is used in both directions and we can't determine user intent. Users using the old field name in either direction will see an exception immediately on client side (i.e. no behavior changes or accidental misuse). We also added an untyped_search_attributes
field to this class to let untyped ones stay present on update.
Removed temporalio.runtime.TelemetryConfig.tracing
temporalio.runtime.TelemetryConfig.tracing
and its associated class have been removed. This was not general purpose tracing, this was advanced tracing for debugging internal Core logc only and should not have been used by any users. All user-facing tracing remains untouched.
Specific Changes
2023-08-01 - 393e5c8 - Provide Span.Kind for TracingInterceptor (#356)
2023-08-02 - 24fea4c - Expose history size and continue-as-new-suggested (#361)
2023-08-02 - 63f63ae - Upgrade grcpio to 1.53.0 (#359)
2023-08-07 - 59fbccf - Fix interceptors in testing environment (#364)
2023-08-16 - 31358d1 - Add ignore_unknown_fields as an argument to JSONProtoPayloadConverter (#365)
2023-08-24 - 40daaaa - Wait for activity completions on worker shutdown (#370)
2023-08-31 - aa829d3 - Update core/tonic, update metric options, remove core tracing (#380)
2023-09-25 - 3fade95 - Custom metric support (#384)
2023-09-28 - d5edb71 - Fix logging of error on activation failure (#389)
2023-10-05 - d03f356 - Buffered metrics (#391)
2023-10-06 - ded3747 - Encourage threaded activities, warn when max_workers too low, and other small changes (#387)
2023-10-09 - 00878ad - Client keep alive support (#396)
2023-10-24 - 4242dfb - README updates and CI fixes (#403)
2023-10-24 - 97814c2 - Workflow start delay (#406)
2023-10-25 - 2c15ed3 - Workflow Update (#400)
2023-10-30 - fd938c4 - Update last cleanup items (#410)
2023-11-02 - 063b9bf - Update core to get bugfix (#411)
2023-11-02 - 7c0a464 - Typed search attributes (#366)
2023-11-06 - 4802d2f - Log forwarding support (#413)
2023-11-06 - 6dbe2f4 - Fix docstring (#414)
2023-11-07 - d6646a2 - Fix pyd file name for Windows (#417)
1.3.0
Get from PyPI
Highlights
Dynamic Workflows, Activities, Signals, and Queries
dynamic=True
can be added to @workflow.defn
, @activity.defn
, @workflow.signal
, and @workflow.query
to make them "dynamic". This means they are unnamed and are called if no other workflow/activity/signal/query match a name from the server when called. The workflow run or activity signature must be a single parameter typed with Sequence[temporalio.common.RawValue]
whereas the signal or query signature must be two parameters typed with str
and Sequence[temporalio.common.RawValue]
. Signals and queries that used the old form of dynamic=True
which didn't require specific signatures are deprecated and will emit a warning.
To support this we added workflow.payload_converter()
and activity.payload_converter()
functions to obtain a payload converter to convert raw values. We also added a temporalio.common.RawValue
class which, when accepted or returned, will just use the raw payload (but it will still use codecs).
See the README for more information.
💥 NOTE: A slight behavior change was made to interceptors to support this. Previously, we would call signal/query interceptor and then check if there was a valid signal/query with that name. Now we check that a valid signal/query handler exists before calling the interceptor. This should not effect most users.
Use Latest Temporal CLI Dev Server Instead of Temporalite
temporalio.testing.WorkflowEnvironment.start_local
now uses the latest Temporal CLI release (which is up to date with latest server) instead of Temporalite. Any use of advanced, documented-experimental temporalite_
-prefixed parameters should now use the dev_server_
-prefixed ones.
Experimental Worker Versioning Support
Support has been added for the brand new Worker Versioning feature. This feature is currently experimental and can only be enabled when running the open source server and setting some special options. Samples for the versioning feature will be present soon.
Schedules GA
The experimental tag for the scheduling API is removed.
General Improvements
- Can now set
result_type
when not using type-safe activity, child workflow, and query calls to get properly typed results (but users should always prefer the type-safe forms) - We now eagerly error if an invalid workflow call is attempted inside a query. In some cases this may have technically worked before but is invalid and will now properly error.
Specific Changes
2023-05-31 - b0daaa3 - Create CODEOWNERS (#323)
2023-06-06 - e1d983e - Add Semgrep scanning (#324)
2023-06-16 - 6a3662c - Remove experimental tag for schedules and add limited action check (#330)
2023-07-05 - 07b2043 - Update core and update Rust dependencies (#337)
2023-07-05 - b75a567 - Allow converter failures to fail workflow and other minor things (#329)
2023-07-06 - aff198b - Set result types for string-based activity, child workflow, and query calls (#334)
2023-07-07 - 677f8be - Move from Temporalite to Temporal CLI dev server (#339)
2023-07-10 - d7238cd - Worker Versioning (#340)
2023-07-11 - 317dd9b - Add Versioning Intents to Commands (#342)
2023-07-11 - b902ec8 - Swallow Python exceptions better on workflow GC from eviction (#341)
2023-07-12 - 34681ca - Reachability type wasn't passed through all the way (#343)
2023-07-14 - 83bbc36 - Dynamic workflows, activities, signals, and queries (#346)
2023-07-17 - 60b72c6 - Link to Python SDK launch blog post (#350)
2023-07-18 - a17c0ef - Disallow most workflow operations in read-only context (#351)
2023-07-18 - b9df212 - Log and drop signals whose params can't be deserialized (and other error handling improvements) (#349)
1.2.0
Get from PyPI
Highlights
No significant changes from previous version, mostly bug fixes.
Specific Changes
2023-03-30 - 4ad2e64 - Use CDN for README image, fixes #302 (#308)
2023-03-30 - 61166af - Minor doc update (#305)
2023-03-30 - 72bdca7 - Update core (#307)
2023-03-30 - d790468 - Call JSON type converters recursively (#304)
2023-04-03 - dbf96fd - Pass through global telemetry tags (#303)
2023-05-01 - a4224a2 - Update core and release version (#314)
1.1.0
Get from PyPI
Highlights
Scheduling API
Python client now has support for creating and working with schedules.
💥 Breaking Changes
When a workflow is already running, we will now raise a WorkflowAlreadyStartedError
error when trying to start instead of the default RPCError
we do for all other errors.
Specific Changes
2023-01-13 - 1a260c3 - Mark local activities as experimental (#252)
2023-01-13 - fdf8b65 - Fail activity worker on broken executor (#253)
2023-01-17 - 677262a - Add missing TERMINATE_IF_RUNNING
to WorkflowIDReusePolicy
(#257)
2023-01-23 - 82f723b - Properly encode failure encoded attributes (#251)
2023-01-23 - f22efea - Support PEP 604 union annotations (#256)
2023-02-07 - 832c375 - Add warning and direct to Pydantic converter sample if Pydantic models are detected (#249)
2023-02-08 - 3c8ee0a - Minor fixes (#268)
2023-02-08 - b52e04e - Add JSONTypeConverter (#269)
2023-02-09 - ae9038c - Fix gRPC health package clash (#270)
2023-02-13 - 1fa36c2 - Force Poetry setup file (#271)
2023-02-15 - 7a2d703 - Change types-protobuf dependency spec to match protobuf (#278)
2023-02-16 - 2ce6d9f - Update Rust dependencies (#281)
2023-02-16 - 852ff2f - Keep extra log attributes (#280)
2023-02-17 - fbcba65 - Schedules (#279)
2023-02-21 - c6e204a - Minor updates (#283)
1.0.0
First Stable Release! 🎉
This is the first stable release of the Python SDK! Since Python is now GA, we provide the same stability guarantees as all other SDKs - primarily that we vow not to break workflow history in newer versions.
Get from PyPI.
💥 Breaking Changes
Minor change since last beta - for those configuring a custom runtime for telemetry purposes via the experimental temporal.bridge.runtime.Runtime
, they must now use the also-experimental temporal.runtime.Runtime
class instead.
Specific Changes
2022-12-08 - 1f72c98 - Added more context to the README sections (#225)
2023-01-03 - 22e8d92 - Update min proto version and fix gRPC optional dependency (#231)
2023-01-04 - 57bf9e6 - Minor type improvements (#239)
2023-01-04 - f252e36 - Expose core "runtime" as temporalio.runtime.Runtime (#240)
0.1b4
Get from PyPI
Highlights
Support for both Protobuf 3.x and 4.x libraries
We now generate Protobuf via 3.x and ensure it works with 3.x and 4.x Python libraries.
Sync activities now raise exception in thread on cancellation
Now, unless opted out, sync multithread activities will raise an exception in thread when cancellation is received.
Simplify designating passthrough modules in the sandbox
Since we are encouraging passing through third party modules, we have made it simpler to do so. Now the imports can simply be wrapped with with workflow.unsafe.imports_passed_through():
or the SandboxRunner
's restrictions
can just have passthrough_modules
as a set of strings (or just SandboxRunner(restrictions=SandboxRestrictions.default.with_passthrough_modules("other.module")
).
💥 Breaking Changes
- Sync threaded activities now have exceptions raised inside of them on cancellation whereas before you had to explicitly check for cancellation
SandboxRestriction
'spassthrough_modules
changed fromSandboxMatcher
type to aSet[str]
Specific Changes
2022-11-15 - 99e36cb - Sandbox and type fixes (#202)
2022-11-18 - a7d6fc3 - Add banner to README (#205)
2022-11-18 - b04813c - Add sdk-features CI trigger (#206)
2022-11-22 - e37b500 - Add table of contents to README (#209)
2022-11-29 - 89b6e66 - Support both protobuf 3 and 4 (#215)
2022-12-01 - 6a43919 - Support raising cancellation in sync multithreaded activities (#217)
2022-12-02 - cc679b9 - Sandbox Improvements (#219)
2022-12-05 - 06828ea - Update SDK core and minor updates (#221)
0.1b3
Get from PyPI
Highlights
Workflow Sandbox
There is now an enabled-by-default workflow sandbox that attempts to ensure determinism via workflow isolation and
restricted access to known non-deterministic standard library calls. If you are importing third-party libraries inside
the same file as your workflow, this can incur a noticeable perf/mem penalty.
Also, if your workflow previously worked out of the sandbox, it may no longer work.
These issues can be mitigated. See https://github.com/temporalio/sdk-python#workflow-sandbox.
Listing Workflows and Multi-History Replayer
API improvements:
Client.list_workflows()
- returns an async iterator of workflows based on
visibility filterWorkflowHandle.fetch_history_events()
- returns an async iterator of history eventsWorkflowHandle.fetch_history()
- helper for the above that returns a fullWorkflowHistory
WorkflowExecutionAsyncIterator.map_histories()
- returns an async iterator ofWorkflowHistory
based on async
iterator of workflowsReplayWorkflow.replay_workflow()
- replays singleWorkflowHistory
ReplayWorkflow.replay_workflows()
- replays async iterator ofWorkflowHistory
collecting resultsReplayWorkflow.workflow_replay_iterator()
- async context manager providing an async iterator over replay results
from an async iterator ofWorkflowHistory
Using these APIs, users can now easily load many workflows and run their histories through a replayer. Among other
benefits, this can help users know how code changes might react on previous workflow executions.
Python 3.11 and Linux ARM Support
Python 3.11 is properly supported and early results show some speed improvement. Also, PyPI releases will now include
Linux ARM wheels.
Failure Converter
A custom temporalio.converter.FailureConverter
can now be set as the DataConverter.failure_converter_class
. In
addition to custom error conversion, this is also useful to set to
temporalio.converter.DefaultFailureConverterWithEncodedAttributes
to force all error messages and stack traces to be
put in the encoded section of the failure, allowing them to run through a PayloadCodec
for encryption.
Worker Fatal Errors
Worker fatal errors are rare, but previously they were not surfaced when using async with
. Now, similar to
asyncio.timeout, the current task will be
cancelled on fatal error and that cancel will be caught at the end of the async context manager and the reason for fatal
will be raised. The alternative to async with
, Worker.run()
, now also raises fatal errors.
Note, fatal errors only occur after a while of internal retries.
💥 Breaking Changes
Not many and they are on less-often used parts.
⚠️ Multiple patched
History Incompatibility
If you called patched
for the same patch ID multiple times in the same workflow, this new release will be incompatible
with that workflow and you'll get non-determinism errors. patched
is now memoized.
API Breaking Changes
- Workflow sandbox may break registering/running workflows that it didn't used to
temporalio.worker.Replayer.replay_workflow
now takes atemporalio.client.WorkflowHistory
object instead of a proto
object or JSON str/dicttemporalio.client.WorkflowExecutionDescription.raw
renamed toraw_description
- Failed queries now raise
temporalio.client.WorkflowQueryFailedError
instead oftemporalio.client.RPCError
- Configuring telemetry has changed from using the global
temporalio.bridge.telemetry.init_telemetry
package to making
a newtelemetryio.bridge.runtime.Runtime
and passing that around.
Specific Changes
2022-10-03 - 0dd2312 - Move API reference site to Vercel (#148)
2022-10-19 - e436ecc - Multiple History Replay support (#158)
2022-10-28 - 0126e6e - Workflow sandbox (#164)
2022-10-28 - 5259f4c - Preload type hints and remove global lookup cache (#168)
2022-10-28 - 657a13d - Don't require timer presence when fired (#169)
2022-10-28 - bc62186 - Remove Go server from Python tests (#170)
2022-10-31 - e16bbc3 - ARM runner and Python 3.11 support (#172)
2022-11-01 - 656b77b - Add simple benchmark script (#180)
2022-11-01 - b4b3f82 - Support for list workflow, fetch workflow history, and iterable replay results (#175)
2022-11-03 - cfdc548 - Add StrEnum conversion support (#177)
2022-11-04 - 65c4025 - Prefix private modules with underscore (#183)
2022-11-07 - 2546871 - Minor updates (#186)
2022-11-07 - 6b9f554 - Propagate fatal worker errors (#188)
2022-11-07 - 87fd193 - Clarity on client thread safety and sync activity cancel (#189)
2022-11-07 - 929dc81 - Failure converter (#185)
2022-11-07 - a75256e - Feature/datetime conversion fix (#179)
2022-11-08 - 3901df8 - Memoize patched calls and support UUID conversion (#192)
2022-11-09 - 87b28a7 - Add missing UUID test case (#193)
2022-11-10 - eacdca8 - Sdk core related updates (#191)