Skip to content

Commit

Permalink
Align dir structure and URL structure with left nav hierarchy (#4843)
Browse files Browse the repository at this point in the history
* Correct heading

Signed-off-by: Peeter Piegaze <[email protected]>

* Align directory and URL structure with left nav

Signed-off-by: Peeter Piegaze <[email protected]>

* Sentence case and logically named files

Signed-off-by: Peeter Piegaze <[email protected]>

---------

Signed-off-by: Peeter Piegaze <[email protected]>
  • Loading branch information
ppiegaze authored Feb 6, 2024
1 parent 4223a83 commit 7abb38a
Show file tree
Hide file tree
Showing 19 changed files with 94 additions and 94 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Flyte is ideal for data cleaning, statistical summarization, and plotting
because with `flytekit` you can leverage the rich Python ecosystem of data
processing and visualization tools.

## Cleaning Data
## Cleaning data

In this example, we are going to analyze some covid vaccination data:

Expand Down Expand Up @@ -41,7 +41,7 @@ As you can see, we're using `pandas` for data processing, and in the task
below we use `plotly` to create a choropleth map of the percent of a country's
population that has received at least one COVID-19 vaccination.

## Rendering Plots
## Rendering plots

We can use {ref}`Flyte Decks <decks>` for rendering a static HTML report
of the map. In this case, we normalize the `people_vaccinated` by the
Expand Down Expand Up @@ -158,7 +158,7 @@ logger.removeFilter(deck_filter)
HTML(filename=cp_deck(deck_filter.deck_files["plot"]))
```

## Custom Flyte Deck Renderers
## Custom Flyte deck renderers

You can also create your own {ref}`custom Flyte Deck renderers <getting_started_customer_renderers>`
to visualize data with any plotting/visualization library of your choice, as
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ kernelspec:

(getting_started_data_engineering)=

# Data Engineering
# Data engineering

Flyte is well-suited for data engineering use cases, where you can interleave
SQL queries with data processing logic implemented in Python with whichever
Expand Down Expand Up @@ -100,7 +100,7 @@ def load(df: pd.DataFrame) -> CSVFile:
return CSVFile(path=csv_file)
```

## ETL Workflow
## ETL workflow

Putting all the pieces together, we create an `etl_workflow` that produces a
dataset based on the parameters you give it.
Expand Down Expand Up @@ -130,7 +130,7 @@ csv_file = etl_workflow(limit=5)
pd.read_csv(csv_file)
```

## Workflows as Reusable Components
## Workflows as reusable components

Because Flyte tasks and workflows are simply functions, we can embed
`etl_workflow` as part of a larger workflow, where it's used to create a
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
(getting_started_core_use_cases)=

# Core Use Cases
# Core use cases

This section of the **Getting Started** documentation will take you through the
This section of the documentation will take you through the
core use cases for which Flyte is designed. Within the context of these guides,
we're going to assume that the discipline of data science can be broken down into
at least three specializations: data engineering, machine learning (or
Expand All @@ -16,9 +16,9 @@ practice.
:header-rows: 0
:widths: 10 30
* - {doc}`🛠 Data Engineering <data_engineering>`
* - {doc}`🛠 Data engineering <data_engineering>`
- Create an ETL workflow for processing data with SQLAlchemy and Pandas.
* - {doc}`🤖 Machine Learning <machine_learning>`
* - {doc}`🤖 Machine learning <machine_learning>`
- Train a classifier with Scikit-Learn and Pandas.
* - {doc}`📈 Analytics <analytics>`
- Develop a data cleaning and plotting pipeline with Plotly and Pandas.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,13 @@ kernelspec:

(getting_started_machine_learning)=

# Machine Learning
# Machine learning

Flyte can handle a full spectrum of machine learning workloads, from
training small models to gpu-accelerated deep learning and hyperparameter
optimization.

## Getting the Data
## Getting the data

In this simple example, we train a binary classification model on the
[wine dataset](https://scikit-learn.org/stable/datasets/toy_dataset.html#wine-dataset)
Expand All @@ -34,7 +34,7 @@ def get_data() -> pd.DataFrame:
return load_wine(as_frame=True).frame
```

## Define a Training Workflow
## Define a training workflow

Then, we define `process_data` and `train_model` tasks along with a
`training_workflow` to put all the pieces together for a model-training
Expand Down Expand Up @@ -76,7 +76,7 @@ type-safe launch forms on the Flyte UI. Learn more in the
{ref}`Extending Flyte <customizing_flyte_types>` guide.
```

## Computing Predictions
## Computing predictions

Executing this workflow locally, we can call the `model.predict` method to make
sure we can use our newly trained model to make predictions based on some
Expand All @@ -88,7 +88,7 @@ X, _ = load_wine(as_frame=True, return_X_y=True)
model.predict(X.sample(10, random_state=41))
```

## Extending your ML Workloads
## Extending your ML workloads

There are many ways to extend your workloads:

Expand Down Expand Up @@ -119,7 +119,7 @@ There are many ways to extend your workloads:
supports ScikitLearn, TensorFlow, and PyTorch.
```

```{admonition} Learn More
```{admonition} Learn more
:class: important
See the {ref}`Tutorials <tutorials>` for more machine learning examples.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ the Flyte type system and Flyte tasks.

(customizing_flyte_types)=

## Customizing Flyte Types
## Customizing Flyte types

Flyte has a {ref}`rich type system <data_types_and_io>` that automatically
handles the serialization and deserialization of objects so that when you pass
Expand Down Expand Up @@ -80,7 +80,7 @@ See the {ref}`User Guide <advanced_custom_types>` for an example of a custom
type.
```

## Customizing Flyte Tasks
## Customizing Flyte tasks

The easiest way to extend Flyte tasks is to use Python decorators. Since Flyte
tasks are simply functions, you can wrap the task function in a custom
Expand Down Expand Up @@ -130,7 +130,7 @@ allows you to enable/disable services at the beginning/end of your workflows.
See the {ref}`User Guide <decorating_workflows>` for more details.
```

## The Plugin Hierarchy of Needs
## The plugin hierarchy of needs

The decorator approach is great for many surface-level use cases, but there are
many more ways to customize Flyte tasks:
Expand All @@ -154,7 +154,7 @@ many more ways to customize Flyte tasks:
external services like Sagemaker, Snowflake, BigQuery, etc.
```

## What's Next?
## What's next?

Congratulations! 🎉 You've just completed the Flyte Fundamentals tour.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
(getting_started_fundamentals)=

# Flyte Fundamentals
# Flyte fundamentals

This section of the documentation will take you through the
fundamental concepts of Flyte: tasks, workflows, and launch plans.
Expand All @@ -15,16 +15,16 @@ use cases.
:header-rows: 0
:widths: 20 30
* - {doc}`🔀 Tasks, Workflows and LaunchPlans <tasks_and_workflows>`
* - {doc}`🔀 Tasks, workflows and launch plans <tasks_workflows_and_launch_plans>`
- Create tasks as building blocks, compose them into workflows, and schedule
them with launchplans.
* - {doc}`🗄 Registering Workflows <package_register>`
them with launch plans.
* - {doc}`🗄 Registering workflows <registering_workflows>`
- Develop and deploy workflows to a local Flyte demo cluster.
* - {doc}`⏱ Running and Scheduling Workflows <run_schedule>`
* - {doc}`⏱ Running and scheduling workflows <running_and_scheduling_workflows>`
- Execute workflows programmatically and schedule them as cron jobs.
* - {doc}`📊 Visualizing Task Input and Output <visualizing_task_input_and_output>`
* - {doc}`📊 Visualizing task input and output <visualizing_task_input_and_output>`
- Create rich, customizable static reports for increased visibility into tasks.
* - {doc}`🏎 Optimizing Tasks <optimizing_tasks>`
* - {doc}`🏎 Optimizing tasks <optimizing_tasks>`
- Make tasks scalable, performant, and robust to unexpected failures.
* - {doc}`🔌 Extending Flyte <extending_flyte>`
- Customize Flyte types and tasks to fit your needs.
Expand All @@ -42,9 +42,9 @@ cluster, see the {ref}`Deployment Guide <deployment>`.
:maxdepth: -1
:hidden:
tasks_and_workflows
package_register
run_schedule
tasks_workflows_and_launch_plans
registering_workflows
running_and_scheduling_workflows
visualizing_task_input_and_output
optimizing_tasks
extending_flyte
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ kernelspec:

(getting_started_optimizing_tasks)=

# Optimizing Tasks
# Optimizing tasks

There are many ways to optimize your tasks and workflows in Flyte, and this guide
will take you through some of the common methods for doing so.
Expand Down Expand Up @@ -94,7 +94,7 @@ def compute_mean(data: List[float]) -> float:
Notice that the `timeout` argument takes a built-in Python
{py:class}`~datetime.timedelta` object.

## Map Tasks
## Map tasks

If you need to parallelize a task, you can use the {py:func}`~flytekit.map_task`
construct. A mappable task is one that takes in a single argument and produces
Expand Down Expand Up @@ -145,7 +145,7 @@ def parallelized_compute_mean(data: List[float], n_partitions: int = 10) -> floa
parallelized_compute_mean(data=[float(x) for x in range(10_000)])
```

## Resource Allocation
## Resource allocation

As one of the core features of Flyte, workflows can be composed of tasks that
potentially have heterogeneous resource requirements. You can express this with
Expand All @@ -165,7 +165,7 @@ def heavy_task() -> float:
...
```

## Multi-image Workflows
## Multi-image workflows

In addition to task-level resource configuration, you can
also specify different images per task. This is particularly useful if some
Expand Down Expand Up @@ -211,7 +211,7 @@ You can also configure the container images dynamically. See the
```


## Declarative Infrastructure
## Declarative infrastructure

Finally, staying with the theme of Flyte's ability to handle heterogeneous
workloads at the most granular level, you can configure tasks to leverage
Expand Down Expand Up @@ -252,7 +252,7 @@ so its plugin system enables you to call out to third-party services and
infrastructure when needed so that you can embed existing workloads into the
Flyte programming paradigm.

## What's Next?
## What's next?

In this guide, you learned the various ways in which you can optimize your
tasks and workflows to make them more scalable and robust. In the final stop of
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
(getting_started_package_register)=

# Registering Workflows
# Registering workflows

In this guide, you'll learn how to package and register (that is, deploy) your tasks and
workflows to a Flyte cluster. This will enable you to scale your workloads with
Expand All @@ -17,7 +17,7 @@ This guide assumes that you:
guide to create a minimal Flyte project.
```

## Custom Dependencies
## Custom dependencies

If you have custom Python dependencies, update the `requirements.txt` file that
ships with the {ref}`project template <getting_started_python_dependencies>`
Expand All @@ -27,7 +27,7 @@ You can also update the Dockerfile (if using a Dockerfile) or the [ImageSpec con
want to use a different base image or if the additional Python dependencies
require installing binaries or packages from other languages.

# Registration Patterns
## Registration patterns

There are different methods of registering your workflows to a Flyte cluster
where each method fulfills a particular use case during the workflow development
Expand All @@ -45,7 +45,7 @@ The following diagram provides a summarized view of the different registration p

(getting_started_register_pyflyte_run)=

## Iterating on a Single Task or Workflow
## Iterating on a single task or workflow

The quickest way to register a task or workflow to a Flyte cluster is with the
`pyflyte run` CLI command. Assuming that you're inside the `my_project` directory
Expand Down Expand Up @@ -80,7 +80,7 @@ contain additional tasks or workflows.

(getting_started_pyflyte_register)=

### Iterating on a Flyte Project
### Iterating on a Flyte project

One of Flyte's benefits is its functional design, which means that you can
import and reuse tasks and workflows like you would Python functions when you
Expand Down Expand Up @@ -137,7 +137,7 @@ Once you've successfully registered your workflows, you can execute them by
going to the Flyte console. If you're using a local Flyte demo cluster, you can
go to the browser at `localhost:30080/console` and do the following:

- Navigate to the **flytesnacks** > **development** domain.
- Navigate to the **flytesnacks** > **development** domain.
- Click on the **Workflows** section of the left-hand sidebar.
- Click on the **workflows.example.wf** card on the workflows list.
- Click on the **Launch Workflow** button on the top-right corner.
Expand All @@ -149,7 +149,7 @@ go to the browser at `localhost:30080/console` and do the following:
In the next guide you'll learn about how to run your workflows programmatically.
```

#### Fast Registration
#### Fast registration

`pyflyte register` packages up your code through a mechanism called
**fast registration**. Fast registration is useful when you already have a
Expand Down Expand Up @@ -182,7 +182,7 @@ You can do so by specifying these files in a `.gitignore` or `.dockerignore`
file in the root directory of your project.
```

### Productionizing your Workflows
### Productionizing your workflows

Flyte's core design decision is to make workflows reproducible and repeatable.
One way it achieves this is by providing a way for you to bake-in user-defined
Expand All @@ -200,7 +200,7 @@ are immutable.

(containerizing_your_project)=

#### Containerizing your Project
#### Containerizing your project

Flyte relies on OCI-compatible containers to package up your code and third-party
dependencies. When you invoke `pyflyte init`, the resulting template project
Expand Down Expand Up @@ -242,7 +242,7 @@ docker login ghcr.io
docker push <tag>
```

```{admonition} Pulling Private Images
```{admonition} Pulling private images
:class: important
For many projects it's convenient to make your images public, but in the case
Expand All @@ -252,7 +252,7 @@ metadata/configuration, it's more secure if they're private.
Learn more about how to pull private image in the {ref}`User Guide <private_images>`.
```

#### Package your Project with `pyflyte package`
#### Package your project with `pyflyte package`

You can package your project with the `pyflyte package` command like so:

Expand Down Expand Up @@ -350,7 +350,7 @@ two GitHub actions that facilitates this:
of Flyte packages, for example, the `.tgz` archives that are created by
`pyflyte package`.

## What's Next?
## What's next?

In this guide, you learned about the Flyte demo cluster, Flyte configuration, and
the different registration patterns you can leverage during the workflow
Expand Down
Loading

0 comments on commit 7abb38a

Please sign in to comment.