Skip to content

Commit

Permalink
docs(python): Adds cancellable queries
Browse files Browse the repository at this point in the history
  • Loading branch information
thomasjpfan committed Aug 4, 2024
1 parent 0c2b5d8 commit 792ae51
Show file tree
Hide file tree
Showing 2 changed files with 45 additions and 0 deletions.
30 changes: 30 additions & 0 deletions docs/src/python/user-guide/lazy/execution.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,34 @@
.fetch(n_rows=int(100))
)
# --8<-- [end:partial]
# --8<-- [start:background]
import time
background_query = (
pl.scan_csv("docs/data/reddit.csv")
.with_columns(pl.col("name").str.to_uppercase())
.filter(pl.col("comment_karma") > 0)
.collect(background=True)
)
def wait_for_query(background_query, *, max_wait_time):
start_time = time.monotonic()
while True:
result = background_query.fetch()
if result is not None:
return result # query successfully completed
current_time = time.monotonic()
if (current_time - start_time) > max_wait_time:
background_query.cancel()
msg = f"Background query took more than {max_wait_time} s"
raise RuntimeError(msg)
# continue waiting
time.sleep(1)
result = wait_for_query(background_query, max_wait_time=10)
# --8<-- [end:background]
"""
15 changes: 15 additions & 0 deletions docs/user-guide/lazy/execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,3 +77,18 @@ shape: (27, 6)
│ 77768 ┆ TINHEADNED ┆ 1139665457 ┆ 1536497404 ┆ 4434 ┆ 103 │
└───────┴───────────────────────────┴─────────────┴────────────┴───────────────┴────────────┘
```

### Cancellable Executions

With background mode, you can run a query in the background and get a handler for
an the in process query. The handler can be used to to do something else while
the query is running or be canceled.

Here we run a background query and cancel the query if it took more than
`max_wait_time` seconds to complete.

!!! warning "background mode is **unstable**"

Background mode may be changed at any point as a breaking change.

{{code_block('user-guide/lazy/execution','background',['scan_csv','collect'])}}

0 comments on commit 792ae51

Please sign in to comment.