-
Notifications
You must be signed in to change notification settings - Fork 2
Home
The latest incarnation of Bifrost is now hosted here: https://github.com/ledatelescope/bifrost
Over the past year, we have developed a new pipeline processing framework designed for sustained high-throughput GPU-based applications, which we call Bifrost. By combining an efficient ring buffer implementation and multi-threaded pipeline architecture with high level interfaces to construct and monitor pipelines in real time, the framework enables rapid development and deployment of new digital telescope back-ends that traditionally require many years of work.
We are currently using Bifrost to construct the Advanced Digital Processor back-end for a new LWA radio observatory (LWA-SV) at Sevilleta, New Mexico, where it runs on a cluster of 14 GTX 980 GPUs. Soon to be completed (2015), this back-end will be significantly more powerful than the existing Digital Processor at the first LWA station (LWA1), with extended bandwidth and full cross-correlation of the 512 antennas making it a cutting edge instrument for the community.
Existing radio astronomy software:
- Pelican (http://www.oerc.ox.ac.uk/~ska/pelican/1.0/doc/user/html/)
- PSRDADA (http://psrdada.sourceforge.net)
- HASHPIPE (https://github.com/david-macmahon/hashpipe)
Pelican provides a highly abstracted API, but is designed for static, quasi-real-time processing. PSRDADA and HASHPIPE are designed for high-throughput stream processing, but suffer from rigidity and low-level C APIs.
Existing computer science software
- Caffe (http://caffe.berkeleyvision.org)
- Torch (http://torch.ch)
These packages provide high-level interfaces for the processing involved in machine learning applications, with built-in support for GPU processing. While they expose considerable generality, they are tailored to offline batch processing rather than real-time streaming.
Existing SDR software
- GNURadio (http://gnuradio.org)
The design choices made for Bifrost are remarkably similar to those of GNURadio (a good example of convergent evolution in software). However, Bifrost was designed from the ground up, using modern C++ and CUDA, with an emphasis on high-performance GPGPU processing.
Bifrost aims to combine and expand upon the functionality of such packages as follows:
- Designed for sustained high-throughput stream processing (with offline batch processing as a corollary).
- A high-level interface (text and/or highly-abstracted code) without a sacrifice in performance.
- A focus on decoupling of processing tasks (e.g., each task can request buffering parameters without constraint, and interpret data streams with shape and type).
- Built-in support for both system and CUDA memory spaces and computation.
- Built-in support for broadcasting of real-time monitoring data, plus tools for subscribing to and visualizing these data.
- A library of generic plugins for common processing operations (e.g., packetize/depacketize, FFT, BLAS; potentially many more). In a nutshell, the goals are: Easy to use, easy to debug, performant enough to compete with best-in-class, and flexible enough to implement even advanced pipeline designs.