Skip to content

Commit

Permalink
[Docs] Update for image spec/fast register notes (#5726)
Browse files Browse the repository at this point in the history
Signed-off-by: Yee Hing Tong <[email protected]>
Co-authored-by: Nikki Everett <[email protected]>
  • Loading branch information
wild-endeavor and neverett authored Oct 16, 2024
1 parent effbdec commit 6c4f8db
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 1 deletion.
8 changes: 8 additions & 0 deletions docs/user_guide/customizing_dependencies/imagespec.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,3 +218,11 @@ You can also force push an image in the Python code by calling the `force_push()
image = ImageSpec(registry="ghcr.io/flyteorg", packages=["pandas"]).force_push()
```
[flytesnacks]: https://github.com/flyteorg/flytesnacks/tree/master/examples/customizing_dependencies/

## Getting source files into ImageSpec
Typically, getting source code files into a task's image at run time on a live Flyte backend is done through the fast registration mechanism.

However, if your `ImageSpec` constructor specifies a `source_root` and the `copy` argument is set to something other than `CopyFileDetection.NO_COPY`, then files will be copied regardless of fast registration status.
If the `source_root` and `copy` fields to an `ImageSpec` are left blank, then whether or not your source files are copied into the built `ImageSpec` image depends on whether or not you use fast registration. Please see [registering workflows](https://docs.flyte.org/en/latest/flyte_fundamentals/registering_workflows.html#containerizing-your-project) for the full explanation.

Since files are sometimes copied into the built image, the tag that is published for an ImageSpec will change based on whether fast register is enabled, and the contents of any files copied.
9 changes: 8 additions & 1 deletion docs/user_guide/flyte_fundamentals/registering_workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ run it with the supplied arguments. As you can see from the expected output, you
can visit the link to the Flyte console to see the progress of your running
execution.

You may also run `run --remote --copy-all`, which is very similar to the above command. As the name suggests, this will copy the source tree rooted at the top-level `__init__.py` file. With this strategy, any modules discoverable on the `PYTHONPATH` will be importable.
You may also run `run --remote --copy all`, which is very similar to the above command. As the name suggests, this will copy the source tree rooted at the top-level `__init__.py` file. With this strategy, any modules discoverable on the `PYTHONPATH` will be importable.

```{note}
`pyflyte run` supports Flyte workflows that import any other user-defined modules that
Expand Down Expand Up @@ -260,6 +260,13 @@ metadata/configuration, it's more secure if they're private.
Learn more about how to pull private image in the {ref}`User Guide <private_images>`.
```

##### Relationship between ImageSpec and fast registration
The `ImageSpec` construct available in flytekit also has a mechanism to copy files into the image being built. Its behavior depends on the type of registration used:
* If fast register is used, then it's assumed that you don't also want to copy source files into the built image.
* If fast register is not used (which is the default for `pyflyte package`, or if `pyflyte register --copy none` is specified), then it's assumed that you do want source files copied into the built image.

If your `ImageSpec` constructor specifies a `source_root` and the `copy` argument is set to something other than `CopyFileDetection.NO_COPY`, then files will be copied regardless of fast registration status.

#### Package your project with `pyflyte package`

You can package your project with the `pyflyte package` command like so:
Expand Down

0 comments on commit 6c4f8db

Please sign in to comment.