Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Add new page for jupyter notebook interactive mode #6036

Merged
merged 26 commits into from
Dec 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/user_guide/flyte_fundamentals/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ use cases.
- Develop and deploy workflows to a local Flyte demo cluster.
* - {doc}`⏱ Running and scheduling workflows <running_and_scheduling_workflows>`
- Execute workflows programmatically and schedule them as cron jobs.
* - {doc}`📙 Jupyter notebook interaction <jupyter_notebook_interaction>`
- Develop and debug Flyte workflows interactively in Jupyter notebooks.
* - {doc}`📊 Visualizing task input and output <visualizing_task_input_and_output>`
- Create rich, customizable static reports for increased visibility into tasks.
* - {doc}`🏎 Optimizing tasks <optimizing_tasks>`
Expand All @@ -45,6 +47,7 @@ cluster, see the {ref}`Deployment Guide <deployment>`.
tasks_workflows_and_launch_plans
registering_workflows
running_and_scheduling_workflows
jupyter_notebook_interaction
visualizing_task_input_and_output
optimizing_tasks
extending_flyte
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
kernelspec:
display_name: Python 3
language: python
name: python3
---

(getting_started_jupyter_notebook_interaction)=

# Running and developing workflows in Jupyter notebooks

Flyte supports the development, running, and debugging of tasks and workflows in an interactive
Jupyter notebook environment, which accelerates the iteration speed when building data-
or machine learning-driven applications.

```{admonition} Attention
:class: attention

This feature requires the `flytekit` version `1.14.0` or higher.
```

```{admonition} Prerequisites
:class: important

This guide assumes that you've completed the previous guides for
{ref}`Running and Scheduling Workflows <getting_started_run_and_schedule>`.
The code snippets in this guide are intended to be run in a Jupyter notebook.
```

The code of this guide can be found in the [flytesnacks](https://github.com/flyteorg/flytesnacks/blob/master/examples/basics/basics/basic_interactive_mode.ipynb)

## Create an interactive `FlyteRemote` object

In {ref}`Running and Scheduling Workflows <getting_started_run_and_schedule>`, you learned
how to run registered Flyte workflows from a Python runtime using the
{py:class}`~flytekit.remote.remote.FlyteRemote` client.

When developing workflows in a Jupyter notebook, `FlyteRemote` provides an
interactive interface to register and run workflows on a Flyte cluster. Let's
create an interactive `FlyteRemote` object:

```{code-cell} ipython3
:tags: [remove-output]

from flytekit.configuration import Config
from flytekit.remote import FlyteRemote

remote = FlyteRemote(
config=Config.auto(),
default_project="flytesnacks",
default_domain="development",
interactive_mode_enabled=True,
)
```

```{admonition} Note
:class: Note

The `interactive_mode_enabled` flag is automatically set to `True` when running
in a Jupyter notebook environment, enabling interactive registration and execution
of workflows.
```

## Running a task or a workflow

You can run entities (tasks or workflows) using the `FlyteRemote`
{py:meth}`~flytekit.remote.remote.FlyteRemote.execute` method.
During execution, `flytekit` first checks if the entity is registered with the
Flyte backend, and if not, registers it before execution.

```{code-block} python
execution = remote.execute(my_task, inputs={"name": "Flyte"})
execution = remote.execute(my_wf, inputs={"name": "Flyte"})
```

You can then fetch the inputs and outputs of the execution by following the steps
in {ref}`getting_started_run_and_schedule_fetch_execution`.

## When Does Interactive `FlyteRemote` Re-register an Entity?

The interactive `FlyteRemote` client re-registers an entity whenever it's
redefined in the notebook, including when you re-execute a cell containing the
entity definition, even if the entity remains unchanged. This behavior facilitates
iterative development and debugging of tasks and workflows in a Jupyter notebook.

## What's next?

In this guide, you learned how to develop and run tasks and workflows in a
Jupyter Notebook environment using interactive `FlyteRemote`.

In the next guide, you'll learn how to visualize tasks using Flyte Decks.
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,8 @@ execution = remote.execute(flyte_task, inputs={"name": "Kermit"})
You can also launch tasks via `flytectl`, learn more in the {ref}`User Guide <remote_task>`
```

(getting_started_run_and_schedule_fetch_execution)=

## Fetching inputs and outputs of an execution

By default, {py:meth}`FlyteRemote.execute <flytekit.remote.remote.FlyteRemote.execute>`
Expand Down Expand Up @@ -342,4 +344,5 @@ In this guide, you learned about how to:
- Run tasks, workflows, and launch plans using `FlyteRemote`.
- Create a cron schedule to run a launch plan at a specified time interval.

In the next guide, you'll learn how to visualize tasks using Flyte Decks.
In the next guide, you'll learn how to develop and run tasks and workflows in
a Jupyter Notebook environment.
Loading