Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Update for image spec/fast register notes #5726

Merged
merged 4 commits into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/user_guide/customizing_dependencies/imagespec.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,3 +218,11 @@ You can also force push an image in the Python code by calling the `force_push()
image = ImageSpec(registry="ghcr.io/flyteorg", packages=["pandas"]).force_push()
```
[flytesnacks]: https://github.com/flyteorg/flytesnacks/tree/master/examples/customizing_dependencies/

## Getting source files into ImageSpec
Typically, getting source code files into a task's image at run time on a live Flyte backend is done through the fast registration mechanism.
wild-endeavor marked this conversation as resolved.
Show resolved Hide resolved

However, if your `ImageSpec` constructor specifies a `source_root` and the `copy` argument is set to something other than `CopyFileDetection.NO_COPY`, then files will be copied regardless of fast registration status.
If the `source_root` and `copy` fields to an `ImageSpec` are left blank, then whether or not your source files are copied into the built `ImageSpec` image depends on whether or not you use fast registration. Please see [registering workflows](https://docs.flyte.org/en/latest/flyte_fundamentals/registering_workflows.html#containerizing-your-project) for the full explanation.

Since files are sometimes copied into the built image, the tag that is published for an ImageSpec will change based on whether fast register is enabled, and the contents of any files copied.
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ run it with the supplied arguments. As you can see from the expected output, you
can visit the link to the Flyte console to see the progress of your running
execution.

You may also run `run --remote --copy-all`, which is very similar to the above command. As the name suggests, this will copy the source tree rooted at the top-level `__init__.py` file. With this strategy, any modules discoverable on the `PYTHONPATH` will be importable.
You may also run `run --remote --copy all`, which is very similar to the above command. As the name suggests, this will copy the source tree rooted at the top-level `__init__.py` file. With this strategy, any modules discoverable on the `PYTHONPATH` will be importable.

```{note}
`pyflyte run` supports Flyte workflows that import any other user-defined modules that
Expand Down Expand Up @@ -260,6 +260,13 @@ metadata/configuration, it's more secure if they're private.
Learn more about how to pull private image in the {ref}`User Guide <private_images>`.
```

##### Relationship between ImageSpec and fast registration
The `ImageSpec` construct available in flytekit also has a mechanism to copy files into the image being built. Its behavior depends on the type of registration used:
* If fast register is used, then it's assumed that you don't also want to copy source files into the built image.
* If fast register is not used (which is the default for `pyflyte package`, or if `pyflyte register --copy none` is specified), then it's assumed that you do want source files copied into the built image.

If your `ImageSpec` constructor specifies a `source_root` and the `copy` argument is set to something other than `CopyFileDetection.NO_COPY`, then files will be copied regardless of fast registration status.

#### Package your project with `pyflyte package`

You can package your project with the `pyflyte package` command like so:
Expand Down
Loading