Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Update for image spec/fast register notes #5726

Merged
merged 4 commits into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion docs/flyte_fundamentals/registering_workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ run it with the supplied arguments. As you can see from the expected output, you
can visit the link to the Flyte console to see the progress of your running
execution.

You may also run `run --remote --copy-all`, which is very similar to the above command. As the name suggests, this will copy the source tree rooted at the top-level `__init__.py` file. With this strategy, any modules discoverable on the `PYTHONPATH` will be importable.
You may also run `run --remote --copy all`, which is very similar to the above command. As the name suggests, this will copy the source tree rooted at the top-level `__init__.py` file. With this strategy, any modules discoverable on the `PYTHONPATH` will be importable.

```{note}
`pyflyte run` supports Flyte workflows that import any other user-defined modules that
Expand Down Expand Up @@ -260,6 +260,14 @@ metadata/configuration, it's more secure if they're private.
Learn more about how to pull private image in the {ref}`User Guide <private_images>`.
```

##### Relationship between ImageSpec and Fast Registration
wild-endeavor marked this conversation as resolved.
Show resolved Hide resolved
The `ImageSpec` construct available in flytekit also has a mechanism to copy files into the image being built. By default, the way it works is
wild-endeavor marked this conversation as resolved.
Show resolved Hide resolved
* if fast register is used, then it's assumed that you don't also want to copy source files into built image.
wild-endeavor marked this conversation as resolved.
Show resolved Hide resolved
* if fast register is not used (which is the default for `pyflyte package`, or if `pyflyte register --copy none` is specified), then it's assumed that
wild-endeavor marked this conversation as resolved.
Show resolved Hide resolved
you do want source files copied.
wild-endeavor marked this conversation as resolved.
Show resolved Hide resolved

In any case, if your `ImageSpec` constructor specifies a `source_root` and the `copy` argument set to something other than `CopyFileDetection.NO_COPY`, then files will be copied regardless of fast registration status.
wild-endeavor marked this conversation as resolved.
Show resolved Hide resolved

#### Package your project with `pyflyte package`

You can package your project with the `pyflyte package` command like so:
Expand Down
6 changes: 6 additions & 0 deletions docs/user_guide/customizing_dependencies/imagespec.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,3 +103,9 @@ or
image = ImageSpec(registry="ghcr.io/flyteorg", packages=["pandas"]).force_push()
```
[flytesnacks]: https://github.com/flyteorg/flytesnacks/tree/master/examples/customizing_dependencies/

## Getting source files into ImageSpec
Typically, getting source code files into a task's image at run time on a live Flyte backend is done through the fast registration mechanism.
wild-endeavor marked this conversation as resolved.
Show resolved Hide resolved
If the `source_root` and `copy` fields to an `ImageSpec` are left blank, then whether or not your source files are copied into the built `ImageSpec` image is basically the inverse of whether fast register is used. Please see [registering workflows](https://docs.flyte.org/en/latest/flyte_fundamentals/registering_workflows.html#containerizing-your-project) for the full explanation.
wild-endeavor marked this conversation as resolved.
Show resolved Hide resolved

Keep in mind that because files are sometimes copied into the built image, the tag that is published for an ImageSpec will change based on whether fast register is enabled, and the contents of any files copied.
wild-endeavor marked this conversation as resolved.
Show resolved Hide resolved
Loading