Simba Decoder - Low Latency

Author: Mike Orr

Intro

Simba Decoder is an application to extract market financial data from Simba encoded PCAP network packets and outputs the results in JSON. The solution is split across two projects: a C++ library, and a C++ executable.

Libray header files:

• Simba/include/eqvilent/financial/order.hpp
• Simba/include/eqvilent/financial/simba.hpp
• Simba/include/eqvilent/sys/binaryIO.hpp
• Simba/include/eqvilent/sys/fileMap.hpp
• Simba/include/eqvilent/sys/network.hpp
• Simba/include/eqvilent/sys/pcap.hpp
• Simba/include/eqvilent/directives.hpp

Library source files:

• Simba/source/eqvilent/financial/order.cpp
• Simba/source/eqvilent/financial/simba.cpp
• Simba/source/eqvilent/sys/binaryIO.cpp
• Simba/source/eqvilent/sys/fileMap.cpp
• Simba/source/eqvilent/sys/pcap.cpp

Decoder executable files:

• Simba/include/eqvilent/financial/messageResult.hpp
• Simba/source/eqvilent/financial/messageResult.cpp
• Simba/source/main.cpp

Test executable files:

• tests/simbaTest.cpp
• tests/resources/test.pcap

Specification

The specification this application was produced from:

1. Develop a parser for the pcap format, which is used to store network traffic, without using any third-party libraries.

2. Create a binary protocol decoder based on the Simba format specification (Moscow exchange moex). The protocol specification can be found at http://ftp.moex.ru/pub/SIMBA/Spectra/prod/doc/spectra_simba_en.pdf. Focus on decoding messages like OrderUpdate, OrderExecution, and OrderBookSnapshot.

3. Test your decoder against the provided data at http://ftp.moex.ru/pub/SIMBA/Spectra/prod/pcap/ using the results obtained in part 1. Save the decoded output to a file in JSON format, with each input packet represented by a single line.

Environment Setup

VCPKG Setup (Windows)

VCPKG is the chosen package manager for this project.

1. Clone the vcpkg public repository and run the bootstrap script

> git clone https://github.com/microsoft/vcpkg.git C:/vcpkg
> cd C:/vcpkg
> .\bootstrap-vcpkg.bat

2. Create system environment variable VCPKG_ROOT, set the value to 'C:\vcpkg', and add the VCPKG_ROOT variable to the system PATH variable.

Doxygen Setup (Windows)

Doxygen is used for documentation generation. Unforunately Doxygen is not currently available as a package with vcpkg, so you will need to manually download and install Doxygen from the official website. Add the Doxygen bin path ('C:\Program Files\doxygen\bin') to your system PATH variable, so Doxygen can be found (may be automatically added by the installer).

Doxygen uses a Graphviz component called 'dot', so download and install Graphviz - be sure to select the install option to add GraphViz to the system PATH variable, otherwise add it manually.

Building - Debug

> cd <project_root_dir>
> cmake -S . -B build --preset "x64-debug"
> cmake --build build --config Debug

Building & Installing - Release

> cd <project_root_dir>
> cmake -S . -B build --preset "x64-release"
> cmake --build build --config Release
> cmake --install build --prefix <path_to_install>

Running Tests (Post-build)

> cd build
> ctest

Notes

1. To open the CMake project in Visual Studio: Start Visual Studio -> continue without code -> File -> Open -> CMake -> CMakeLists.txt

2. Doxygen documentation is generated during the build process and is installed to the install directory - see ../docs/html/index.html

Reasoning

1. Given the potential for large input and to prevent lengthy runtimes, the decision was made to use memory mapping for reading. Memory mapping for writing was not used as the potential upside for the project's requirements was not sufficient.

2. Functionality likely to be re-used was placed within a library sub-project, 'Simba'.

3. Created a small pcap file from a larger pcap file for testing purposes due to size limits with version control and large datastore symbolic links not being adequate.