README

<div align="center">
  <a href="http://www.openucx.org/"><img src="./doc/doxygen/UCX_Logo_930x933.png" width="200"></a>
  <br>
  <a href="https://twitter.com/openucx"><img src="doc/doxygen/twitter.svg"></a>
  <a href="http://openucx.github.io/ucx/api/v1.6/html/"><img src="doc/doxygen/api.svg"></a>
  <a href="https://github.com/openucx/ucx/releases/tag/v1.6.0"><img src="doc/doxygen/release.svg"></a>
</div>

<!-- TOC generated by https://github.com/ekalinin/github-markdown-toc -->

<hr>

   * [Unified Communication X](#unified-communication-x)
      * [Using UCX](#using-ucx)
         * [Building and Running Internal Unit Tests](#building-and-running-internal-unit-tests)
         * [UCX Performance Test](#ucx-performance-test)
      * [Our Community](#our-community)
      * [Licenses](#licenses)
      * [Contributor Agreement and Guidelines](#contributor-agreement-and-guidelines)
      * [UCX Publications](#ucx-publications)
      * [UCX Architecture](#ucx-architecture)
      * [Supported Transports](#supported-transports)
      * [Supported CPU Architectures](#supported-cpu-architectures)

<hr>

# Unified Communication X

Unified Communication X (UCX) provides an optimized communication
layer for Message Passing ([MPI](https://www.mpi-forum.org/)),
[PGAS](http://www.pgas.org/)/[OpenSHMEM](http://www.openshmem.org/)
libraries and RPC/data-centric applications.

UCX utilizes high-speed networks for inter-node communication, and
shared memory mechanisms for efficient intra-node communication.

## Using UCX

### Release Builds

Building UCX is typically a combination of running "configure" and "make".
Execute the following commands to install the UCX system from within the
directory at the top of the tree:

```sh
$ ./autogen.sh
$ ./contrib/configure-release --prefix=/where/to/install
$ make -j8
$ make install
```

NOTE: Compiling support for various networks or other specific hardware may
require additional command line flags when running configure.

### Developer Builds

```bash
$ ./autogen.sh
$ ./contrib/configure-devel --prefix=$PWD/install-debug
```

*** NOTE: Developer builds of UCX typically include a large performance 
penalty at run-time because of extra debugging code.

### Running internal unit tests

```sh
$ make -C test/gtest test
```

### Build RPM package
```bash
$ contrib/buildrpm.sh -s -b
```

### Build DEB package
```bash
$ dpkg-buildpackage -us -uc
```

### Build Doxygen documentation
```bash
$ make docs
```

### OpenMPI and OpenSHMEM installation with UCX
[Wiki page](http://github.com/openucx/ucx/wiki/OpenMPI-and-OpenSHMEM-installation-with-UCX)

### MPICH installation with UCX
[Wiki page](http://github.com/openucx/ucx/wiki/MPICH-installation-with-UCX)

### UCX Performance Test

Start server:


```sh
$ ./src/tools/perf/ucx_perftest -c 0
```

Connect client:

```sh
$ ./src/tools/perf/ucx_perftest <server-hostname> -t tag_lat -c 0
```

## Our Community

* [Project Website](http://www.openucx.org/)
* [Github](http://www.github.com/openucx/ucx/)
* [Software Releases](http://www.github.com/openucx/ucx/releases)
* [Mailing List](https://elist.ornl.gov/mailman/listinfo/ucx-group)
* [Twitter](https://twitter.com/openucx)

## Licenses

UCX is licensed as:

* [BSD3](LICENSE)

## Contributor Agreement and Guidelines

In order to contribute to UCX, please sign up with an appropriate
[Contributor Agreement](http://www.openucx.org/license/).

Follow these
[instructions](https://github.com/openucx/ucx/blob/master/CONTRIBUTING.md)
when submitting contributions and changes.

## UCX Publications

To reference UCX in a publication, please use the following entry:

```bibtex
@inproceedings{shamis2015ucx,
  title={UCX: an open source framework for HPC network APIs and beyond},
  author={Shamis, Pavel and Venkata, Manjunath Gorentla and Lopez, M Graham and Baker, Matthew B and Hernandez, Oscar and Itigin, Yossi and Dubman, Mike and Shainer, Gilad and Graham, Richard L and Liss, Liran and others},
  booktitle={2015 IEEE 23rd Annual Symposium on High-Performance Interconnects},
  pages={40--43},
  year={2015},
  organization={IEEE}
}
```

To reference the UCX website:

```bibtex
@misc{openucx-website,
    title = {{The Unified Communication X Library}},
    key = {{{The Unified Communication X Library}},
    howpublished = {{\url{http://www.openucx.org}}}
}
```

## UCX Architecture

![](doc/doxygen/Architecture.png)

| Component | Role        | Description |
| :---:     | :---:       | ---         |
| UCP | Protocol          | Implements high-level abstractions such as tag-matching, streams, connection negotiation and establishment, multi-rail, and handling different memory types |
| UCT | Transport         | Implements low-level communication primitives such as active messages, remote memory access, and atomic operations |
| UCS | Services          | A collection of data structures, algorithms, and system utilities for common use |
| UCM | Memory            | Intercepts memory allocation and release events, used by the  memory registration cache |

## Supported Transports

* [Infiniband](https://www.infinibandta.org/)
* [Omni-Path](https://www.intel.com/content/www/us/en/high-performance-computing-fabrics/omni-path-driving-exascale-computing.html)
* [RoCE](http://www.roceinitiative.org/)
* [Cray Gemini and Aries](https://www.cray.com/)
* [CUDA](https://developer.nvidia.com/cuda-zone)
* [ROCm](https://rocm.github.io/)
* Shared Memory
    * posix, sysv, [cma](https://dl.acm.org/citation.cfm?id=2616532), [knem](http://knem.gforge.inria.fr/), and [xpmem](https://github.com/hjelmn/xpmem)
* TCP/IP

## Supported CPU Architectures

* [x86_64](https://en.wikipedia.org/wiki/X86-64)
* [Power8/9](https://www.ibm.com/support/knowledgecenter/en/POWER9/p9hdx/POWER9welcome.htm)
* [Arm v8](https://www.arm.com/products/silicon-ip-cpu)