Fuzzing

[mratsim]

The introduction of a wide-range of tests against a reference implementation https://github.com/mratsim/weave/blob/f8c80719/benchmarks/matmul_gemm_blas/test_gemm_output.nim highlighted synchronization bugs like syncRoot not handling dataflow parallelism (#97 #115) and deadlock or livelock (#119).

In general we need to introduce some fuzzing.

Note that due to the state explosion and non-determinism of multithreaded applications, fuzzing can only cover a very small portion of application state if done globally. Formal verification, model checking and race detection (#18) would still be required as some bugs have otherwise a low chance of being triggered.

Readings

• Fuzzing with ASAN notes:

  • https://github.com/mirrorer/afl/blob/master/docs/notes_for_asan.txt
  • https://countuponsecurity.com/2018/04/24/intro-to-american-fuzzy-lop-fuzzing-with-asan-and-beyond/
  • https://fuzzing-project.org/tutorial3.html

• Securing Software Systems via Fuzz Testing and Verification
  Hongxu Chen, PhD Thesis, 2019
  https://www.ntu.edu.sg/home/yangliu/publications/hongxu.pdf

[mratsim]

On Fuzzing Concurrent Program with C++ Atomics
Zachary Snyder, 2019
https://escholarship.org/content/qt5rw7n0xs/qt5rw7n0xs.pdf