Skip to content

Commit

Permalink
[Docs] .skyignore doc (#4114)
Browse files Browse the repository at this point in the history
* [Docs] .skyignore doc

* Correct typos

Co-authored-by: Zongheng Yang <[email protected]>

---------

Co-authored-by: Zongheng Yang <[email protected]>
  • Loading branch information
yika-luo and concretevitamin authored Oct 18, 2024
1 parent 3042a27 commit a34ccb7
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 27 deletions.
53 changes: 28 additions & 25 deletions docs/source/examples/syncing-code-artifacts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,31 +46,7 @@ VMs. The task is invoked under that working directory (so that it can call
scripts, access checkpoints, etc.).

.. note::

**Exclude files from syncing**

For large, multi-gigabyte workdirs, uploading may be slow because they
are synced to the remote VM(s). To exclude large files in
your workdir from being uploaded, add them to a :code:`.skyignore` file
under your workdir. :code:`.skyignore` follows RSYNC filter rules.

Example :code:`.skyignore` file:

.. code-block::
# Files that match pattern under ONLY CURRENT directory
/hello.py
/*.txt
/dir
# Files that match pattern under ALL directories
*.txt
hello.py
# Files that match pattern under a directory ./dir/
/dir/*.txt
Do NOT use ``.`` to indicate local directory (e.g. ``./hello.py``).
To exclude large files from being uploaded, see :ref:`exclude-uploading-files`.

.. note::

Expand Down Expand Up @@ -140,6 +116,33 @@ file_mount may be slow because they are processed by ``rsync``. Use
:ref:`SkyPilot bucket mounting <sky-storage>` to efficiently handle
large files.

.. _exclude-uploading-files:

Exclude uploading files
--------------------------------------
By default, SkyPilot uses your existing :code:`.gitignore` and :code:`.git/info/exclude` to exclude files from syncing.

Alternatively, you can use :code:`.skyignore` if you want to separate SkyPilot's syncing behavior from Git's.
If you use a :code:`.skyignore` file, SkyPilot will only exclude files based on that file without using the default Git files.

Any :code:`.skyignore` file under either your workdir or source paths of file_mounts is respected.

:code:`.skyignore` follows RSYNC filter rules, e.g.

.. code-block::
# Files that match pattern under CURRENT directory
/file.txt
/dir
/*.jar
/dir/*.jar
# Files that match pattern under ALL directories
*.jar
file.txt
Do _not_ use ``.`` to indicate local directory (e.g., instead of ``./file``, write ``/file``).

.. _downloading-files-and-artifacts:

Downloading files and artifacts
Expand Down
4 changes: 2 additions & 2 deletions docs/source/reference/yaml-spec.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ Available fields:
# If a relative path is used, it's evaluated relative to the location from
# which `sky` is called.
#
# To exclude files from syncing, add them to a .skyignore file under your working directory.
# Details: https://skypilot.readthedocs.io/en/latest/examples/syncing-code-artifacts.html#uploading-code-and-project-files
# To exclude files from syncing, see
# https://skypilot.readthedocs.io/en/latest/examples/syncing-code-artifacts.html#exclude-uploading-files
workdir: ~/my-task-code
# Number of nodes (optional; defaults to 1) to launch including the head node.
Expand Down

0 comments on commit a34ccb7

Please sign in to comment.