diff --git a/shortfin/README.md b/shortfin/README.md index 9818e05d3..13ee20966 100644 --- a/shortfin/README.md +++ b/shortfin/README.md @@ -1,20 +1,52 @@ # shortfin - SHARK inference library and serving engine -## Simple User Installation +The shortfin project is SHARK's open source, high performance inference library +and serving engine. Shortfin consists of these major components: -Install: +* The "libshortfin" inference library written in C/C++ and built on + [IREE](https://github.com/iree-org/iree) +* Python bindings for the underlying inference library +* Example applications in + ['shortfin_apps'](https://github.com/nod-ai/SHARK-Platform/tree/main/shortfin/python/shortfin_apps) + built using the python bindings +## Prerequisites + +* Python 3.11+ + +## Simple user installation + +Install the latest stable version: + +```bash +pip install shortfin ``` -python -m pip install . + +## Developer guides + +### Quick start: install local packages and run tests + +After cloning this repository, from the `shortfin/` directory: + +```bash +pip install -e . ``` -Run tests: +Install test requirements: +```bash +pip install -r requirements-tests.txt ``` -python -m pytest -s tests/ + +Run tests: + +```bash +pytest -s tests/ ``` -## Simple Dev Setup +### Simple dev setup + +We recommend this development setup for core contributors: 1. Check out this repository as a sibling to [IREE](https://github.com/iree-org/iree) if you already have an IREE source checkout. Otherwise, a pinned version will @@ -36,7 +68,7 @@ python -m pytest -s tests/ Refer to the advanced build options below for other scenarios. -## Advanced Build Options +### Advanced build options 1. Native C++ build 2. Local Python release build @@ -48,7 +80,7 @@ Prerequisites * A modern C/C++ compiler, such as clang 18 or gcc 12 * A modern Python, such as Python 3.12 -### Native C++ Builds +#### Native C++ builds ```bash cmake -GNinja -S. -Bbuild \ @@ -61,13 +93,7 @@ If Python bindings are enabled in this mode (`-DSHORTFIN_BUILD_PYTHON_BINDINGS=O then `pip install -e build/` will install from the build dir (and support build/continue). -### Local Python Release Builds - -```bash -pip install -v -e . -``` - -### Package Python Release Builds +#### Package Python release builds * To build wheels for Linux using a manylinux Docker container: @@ -86,7 +112,7 @@ pip install -v -e . python3 -m pip install dist/*.whl ``` -### Python Dev Builds +#### Python dev builds ```bash # Install build system pre-reqs (since we are building in dev mode, this @@ -124,7 +150,7 @@ Several optional environment variables can be used with setup.py: * `SHORTFIN_RUN_CTESTS=ON` : Runs `ctest` as part of the build. Useful for CI as it uses the version of ctest installed in the pip venv. -### Running Tests +### Running tests The project uses a combination of ctest for native C++ tests and pytest. Much of the functionality is only tested via the Python tests, using the @@ -156,7 +182,7 @@ pytest tests/ --system amdgpu \ --compile-flags="--iree-hal-target-backends=rocm --iree-hip-target=gfx1100" ``` -# Production Library Building +## Production library building In order to build a production library, additional build steps are typically recommended: @@ -167,23 +193,23 @@ recommended: * Enable LTO builds of libshortfin * Set flags to enable symbol versioning -# Miscellaneous Build Topics +## Miscellaneous build topics -## Free-threaded Python +### Free-threaded Python Support for free-threaded Python builds (aka. "nogil") is in progress. It -is currently being tested via dev builds of CPython 3.13 with the -`--disable-gil` option set. There are multiple ways to acquire such an -environment. If using `pyenv`, here is a way: +is currently being tested via CPython 3.13 with the `--disable-gil` option set. +There are multiple ways to acquire such an environment: -``` -# Build a free-threaded 3.13 version. -pyenv install --debug 3.13t-dev +* Generally, see the documentation at + +* If using `pyenv`: -# Test (should print "False"). -pyenv shell 3.13t-dev -python -c 'import sys; print(sys._is_gil_enabled())' -``` + ```bash + # Install a free-threaded 3.13 version. + pyenv install 3.13t -Further ways of installing a free-threaded CPython interpreter are documented at -[py-free-threading.github.io](https://py-free-threading.github.io/installing_cpython/). + # Test (should print "False"). + pyenv shell 3.13t + python -c 'import sys; print(sys._is_gil_enabled())' + ```