Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iterate on shortfin/README.md. #501

Merged
merged 9 commits into from
Nov 14, 2024
88 changes: 57 additions & 31 deletions shortfin/README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,52 @@
# shortfin - SHARK inference library and serving engine

## Simple User Installation
The shortfin project is SHARK's open source, high performance inference library
and serving engine. Shortfin consists of these major components:

Install:
* The "libshortfin" inference library written in C/C++ and built on
[IREE](https://github.com/iree-org/iree)
* Python bindings for the underlying inference library
* Example applications in
['shortfin_apps'](https://github.com/nod-ai/SHARK-Platform/tree/main/shortfin/python/shortfin_apps)
built using the python bindings

## Prerequisites

* Python 3.11+
ScottTodd marked this conversation as resolved.
Show resolved Hide resolved

## Simple user installation

Install the latest stable version:

```bash
python -m pip install shortfin
ScottTodd marked this conversation as resolved.
Show resolved Hide resolved
```
python -m pip install .

## Developer guides

### Quick start: install local packages and run tests

After cloning this repository, from the `shortfin/` directory:

```bash
python -m pip install -e .
ScottTodd marked this conversation as resolved.
Show resolved Hide resolved
```

Run tests:
Install test requirements:

```bash
python -m pip install -r requirements-tests.txt
ScottTodd marked this conversation as resolved.
Show resolved Hide resolved
```

Run tests:

```bash
python -m pytest -s tests/
ScottTodd marked this conversation as resolved.
Show resolved Hide resolved
```

## Simple Dev Setup
### Simple dev setup

We recommend this development setup for core contributors:

1. Check out this repository as a sibling to [IREE](https://github.com/iree-org/iree)
if you already have an IREE source checkout. Otherwise, a pinned version will
Expand All @@ -36,7 +68,7 @@ python -m pytest -s tests/

Refer to the advanced build options below for other scenarios.

## Advanced Build Options
### Advanced build options

1. Native C++ build
2. Local Python release build
Expand All @@ -48,7 +80,7 @@ Prerequisites
* A modern C/C++ compiler, such as clang 18 or gcc 12
* A modern Python, such as Python 3.12

### Native C++ Builds
#### Native C++ builds

```bash
cmake -GNinja -S. -Bbuild \
Expand All @@ -61,13 +93,7 @@ If Python bindings are enabled in this mode (`-DSHORTFIN_BUILD_PYTHON_BINDINGS=O
then `pip install -e build/` will install from the build dir (and support
build/continue).

### Local Python Release Builds

```bash
pip install -v -e .
```

### Package Python Release Builds
#### Package Python release builds

* To build wheels for Linux using a manylinux Docker container:

Expand All @@ -86,7 +112,7 @@ pip install -v -e .
python3 -m pip install dist/*.whl
```

### Python Dev Builds
#### Python dev builds

```bash
# Install build system pre-reqs (since we are building in dev mode, this
Expand Down Expand Up @@ -124,7 +150,7 @@ Several optional environment variables can be used with setup.py:
* `SHORTFIN_RUN_CTESTS=ON` : Runs `ctest` as part of the build. Useful for CI
as it uses the version of ctest installed in the pip venv.

### Running Tests
### Running tests

The project uses a combination of ctest for native C++ tests and pytest. Much
of the functionality is only tested via the Python tests, using the
Expand Down Expand Up @@ -156,7 +182,7 @@ pytest tests/ --system amdgpu \
--compile-flags="--iree-hal-target-backends=rocm --iree-hip-target=gfx1100"
```

# Production Library Building
## Production library building

In order to build a production library, additional build steps are typically
recommended:
Expand All @@ -167,23 +193,23 @@ recommended:
* Enable LTO builds of libshortfin
* Set flags to enable symbol versioning

# Miscellaneous Build Topics
## Miscellaneous build topics

## Free-threaded Python
### Free-threaded Python

Support for free-threaded Python builds (aka. "nogil") is in progress. It
is currently being tested via dev builds of CPython 3.13 with the
`--disable-gil` option set. There are multiple ways to acquire such an
environment. If using `pyenv`, here is a way:
is currently being tested via CPython 3.13 with the `--disable-gil` option set.
There are multiple ways to acquire such an environment:

```
# Build a free-threaded 3.13 version.
pyenv install --debug 3.13t-dev
* Generally, see the documentation at
<https://py-free-threading.github.io/installing_cpython/>
* If using `pyenv`:

# Test (should print "False").
pyenv shell 3.13t-dev
python -c 'import sys; print(sys._is_gil_enabled())'
```
```bash
# Install a free-threaded 3.13 version.
pyenv install 3.13t

Further ways of installing a free-threaded CPython interpreter are documented at
[py-free-threading.github.io](https://py-free-threading.github.io/installing_cpython/).
# Test (should print "False").
pyenv shell 3.13
ScottTodd marked this conversation as resolved.
Show resolved Hide resolved
python -c 'import sys; print(sys._is_gil_enabled())'
```
Loading