Skip to content

AxoNN 0.1.0
Compare
Choose a tag to compare
@bhatele bhatele released this 25 Apr 02:53
· 58 commits to develop since this release
v0.1.0
ccd878b

AxoNN is a parallel framework for training deep neural networks.

Features:

  • Offers a hybrid of inter-layer parallelism with pipelining and data parallelism.
  • Supports both 16-bit mixed precision and 32-bit full precision training.
  • A highly efficient and scalable implementation of inter-layer parallelism with pipelining using asynchronous MPI-based communication and message-driven scheduling that achieves significant overlap of computation and communication.
  • Memory optimizations that can reduce the model state memory consumption by 5x for mixed precision training using the Adam optimizer and indirectly also increase hardware efficiency.
Assets 2
Loading