From 228a1b7c136afcb64716b424830b538fb94dfc62 Mon Sep 17 00:00:00 2001 From: Scott Todd Date: Tue, 12 Nov 2024 17:02:40 -0800 Subject: [PATCH 1/5] Iterate on shortfin/README.md. --- shortfin/README.md | 50 +++++++++++++++++++++++++++++++--------------- 1 file changed, 34 insertions(+), 16 deletions(-) diff --git a/shortfin/README.md b/shortfin/README.md index 3e7901342..1168674ae 100644 --- a/shortfin/README.md +++ b/shortfin/README.md @@ -1,20 +1,38 @@ -# shortfin - SHARK C++ inference library +# shortfin - SHARK inference library and serving engine -## Simple User Installation +The shortfin project is SHARK's open source, high performance inference library +and serving engine. Shortfin consists of these major components: -Install: +* The "libshortfin" inference library written in C/C++ and built on + [IREE](https://github.com/iree-org/iree) +* Python bindings for the underlying inference library +* Example applications 'shortfin_apps' built using the python bindings +## Simple user installation + +Install the latest stable version: + +```bash +python -m pip install shortfin ``` + +## Developer guides + +### Install local packages + +After cloning this repository, from the `shortfin/` directory: + +```bash python -m pip install . ``` Run tests: -``` +```bash python -m pytest -s tests/ ``` -## Simple Dev Setup +### Simple dev setup 1. Check out this repository as a sibling to [IREE](https://github.com/iree-org/iree) if you already have an IREE source checkout. Otherwise, a pinned version will @@ -36,7 +54,7 @@ python -m pytest -s tests/ Refer to the advanced build options below for other scenarios. -## Advanced Build Options +### Advanced build options 1. Native C++ build 2. Local Python release build @@ -48,7 +66,7 @@ Prerequisites * A modern C/C++ compiler, such as clang 18 or gcc 12 * A modern Python, such as Python 3.12 -### Native C++ Builds +#### Native C++ builds ```bash cmake -GNinja -S. -Bbuild \ @@ -61,13 +79,13 @@ If Python bindings are enabled in this mode (`-DSHORTFIN_BUILD_PYTHON_BINDINGS=O then `pip install -e build/` will install from the build dir (and support build/continue). -### Local Python Release Builds +#### Local Python release builds ```bash pip install -v -e . ``` -### Package Python Release Builds +#### Package Python release builds * To build wheels for Linux using a manylinux Docker container: @@ -86,7 +104,7 @@ pip install -v -e . python3 -m pip install dist/*.whl ``` -### Python Dev Builds +#### Python dev builds ```bash # Install build system pre-reqs (since we are building in dev mode, this @@ -124,7 +142,7 @@ Several optional environment variables can be used with setup.py: * `SHORTFIN_RUN_CTESTS=ON` : Runs `ctest` as part of the build. Useful for CI as it uses the version of ctest installed in the pip venv. -### Running Tests +#### Running tests The project uses a combination of ctest for native C++ tests and pytest. Much of the functionality is only tested via the Python tests, using the @@ -139,7 +157,7 @@ does mean that the C++ core of the library must always be built with the Python bindings to test the most behavior. Given the target of the project, this is not considered to be a significant issue. -#### Python tests +##### Python tests Run platform independent tests only: @@ -156,7 +174,7 @@ pytest tests/ --system amdgpu \ --compile-flags="--iree-hal-target-backends=rocm --iree-hip-target=gfx1100" ``` -# Production Library Building +## Production library building In order to build a production library, additional build steps are typically recommended: @@ -167,16 +185,16 @@ recommended: * Enable LTO builds of libshortfin * Set flags to enable symbol versioning -# Miscellaneous Build Topics +## Miscellaneous build topics -## Free-threaded Python +### Free-threaded Python Support for free-threaded Python builds (aka. "nogil") is in progress. It is currently being tested via dev builds of CPython 3.13 with the `--disable-gil` option set. There are multiple ways to acquire such an environment. If using `pyenv`, here is a way: -``` +```bash # Build a free-threaded 3.13 version. pyenv install --debug 3.13t-dev From 84ee0d2e4472d4d08d428c65112e1e18198eab75 Mon Sep 17 00:00:00 2001 From: Scott Todd Date: Wed, 13 Nov 2024 10:55:12 -0800 Subject: [PATCH 2/5] Adjust heading levels. --- shortfin/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/shortfin/README.md b/shortfin/README.md index 1168674ae..62735ca21 100644 --- a/shortfin/README.md +++ b/shortfin/README.md @@ -142,7 +142,7 @@ Several optional environment variables can be used with setup.py: * `SHORTFIN_RUN_CTESTS=ON` : Runs `ctest` as part of the build. Useful for CI as it uses the version of ctest installed in the pip venv. -#### Running tests +### Running tests The project uses a combination of ctest for native C++ tests and pytest. Much of the functionality is only tested via the Python tests, using the @@ -157,7 +157,7 @@ does mean that the C++ core of the library must always be built with the Python bindings to test the most behavior. Given the target of the project, this is not considered to be a significant issue. -##### Python tests +#### Python tests Run platform independent tests only: From 4ecadaab5f8317ca6491edc2d455790de42ee5dd Mon Sep 17 00:00:00 2001 From: Scott Todd Date: Wed, 13 Nov 2024 11:03:36 -0800 Subject: [PATCH 3/5] Add link to shortfin_apps, list Python 3.11 as a prerequisite. --- shortfin/README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/shortfin/README.md b/shortfin/README.md index 62735ca21..59b08a752 100644 --- a/shortfin/README.md +++ b/shortfin/README.md @@ -6,7 +6,13 @@ and serving engine. Shortfin consists of these major components: * The "libshortfin" inference library written in C/C++ and built on [IREE](https://github.com/iree-org/iree) * Python bindings for the underlying inference library -* Example applications 'shortfin_apps' built using the python bindings +* Example applications in + ['shortfin_apps'](https://github.com/nod-ai/SHARK-Platform/tree/main/shortfin/python/shortfin_apps) + built using the python bindings + +## Prerequisites + +* Python 3.11+ ## Simple user installation From 818cd4aabc87b9f16867a0e1a70a03f1ba9a22a2 Mon Sep 17 00:00:00 2001 From: Scott Todd Date: Wed, 13 Nov 2024 14:26:46 -0800 Subject: [PATCH 4/5] Restructure "Install local packages" and "Free-threaded Python" steps. --- shortfin/README.md | 42 ++++++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 20 deletions(-) diff --git a/shortfin/README.md b/shortfin/README.md index 59b08a752..86cf77d3f 100644 --- a/shortfin/README.md +++ b/shortfin/README.md @@ -24,12 +24,18 @@ python -m pip install shortfin ## Developer guides -### Install local packages +### Quick start: install local packages and run tests After cloning this repository, from the `shortfin/` directory: ```bash -python -m pip install . +python -m pip install -e . +``` + +Install test requirements: + +```bash +python -m pip install -r requirements-tests.txt ``` Run tests: @@ -40,6 +46,8 @@ python -m pytest -s tests/ ### Simple dev setup +We recommend this development setup for core contributors: + 1. Check out this repository as a sibling to [IREE](https://github.com/iree-org/iree) if you already have an IREE source checkout. Otherwise, a pinned version will be downloaded for you @@ -85,12 +93,6 @@ If Python bindings are enabled in this mode (`-DSHORTFIN_BUILD_PYTHON_BINDINGS=O then `pip install -e build/` will install from the build dir (and support build/continue). -#### Local Python release builds - -```bash -pip install -v -e . -``` - #### Package Python release builds * To build wheels for Linux using a manylinux Docker container: @@ -196,18 +198,18 @@ recommended: ### Free-threaded Python Support for free-threaded Python builds (aka. "nogil") is in progress. It -is currently being tested via dev builds of CPython 3.13 with the -`--disable-gil` option set. There are multiple ways to acquire such an -environment. If using `pyenv`, here is a way: +is currently being tested via CPython 3.13 with the `--disable-gil` option set. +There are multiple ways to acquire such an environment: -```bash -# Build a free-threaded 3.13 version. -pyenv install --debug 3.13t-dev +* Generally, see the documentation at + +* If using `pyenv`: -# Test (should print "False"). -pyenv shell 3.13t-dev -python -c 'import sys; print(sys._is_gil_enabled())' -``` + ```bash + # Install a free-threaded 3.13 version. + pyenv install 3.13t -Further ways of installing a free-threaded CPython interpreter are documented at -[py-free-threading.github.io](https://py-free-threading.github.io/installing_cpython/). + # Test (should print "False"). + pyenv shell 3.13 + python -c 'import sys; print(sys._is_gil_enabled())' + ``` From 8b7262bd8a565989433f443fdfcccdca6ebe54f2 Mon Sep 17 00:00:00 2001 From: Scott Todd Date: Thu, 14 Nov 2024 08:26:00 -0800 Subject: [PATCH 5/5] Apply suggestions from code review Co-authored-by: Marius Brehler --- shortfin/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/shortfin/README.md b/shortfin/README.md index 86cf77d3f..13ee20966 100644 --- a/shortfin/README.md +++ b/shortfin/README.md @@ -19,7 +19,7 @@ and serving engine. Shortfin consists of these major components: Install the latest stable version: ```bash -python -m pip install shortfin +pip install shortfin ``` ## Developer guides @@ -29,19 +29,19 @@ python -m pip install shortfin After cloning this repository, from the `shortfin/` directory: ```bash -python -m pip install -e . +pip install -e . ``` Install test requirements: ```bash -python -m pip install -r requirements-tests.txt +pip install -r requirements-tests.txt ``` Run tests: ```bash -python -m pytest -s tests/ +pytest -s tests/ ``` ### Simple dev setup @@ -210,6 +210,6 @@ There are multiple ways to acquire such an environment: pyenv install 3.13t # Test (should print "False"). - pyenv shell 3.13 + pyenv shell 3.13t python -c 'import sys; print(sys._is_gil_enabled())' ```