Unusual Helium 0.2.1 Performance Results

Test 1

Overview

Interesting failure behavior. When running in exactly the same environment as the one used to gather Results 0, but with 128 switches instead of 64, ODL drops to very low CPU usage, CBench pegs at 100% and CBench consistently reports 0 flows/second. There are no CBench msgbuf errors, just "successful" runs with 0 flows/sec results. RAM usage increases in an interesting way, as shown on the graph below.

This seems like either an issue with ODL or CBench (I suspect ODL).

Restarting CBench (ctrl+c; ./loop_wcbench.sh -l) puts the system back in the same state. That supports the hypothesis that it's an ODL issue.

Test Details

• Up-to-date Fedora 20 VM running on an OpenStack deployment.
• 8GB RAM
• 4 VCPUs
• 128 switches
• Test command: ./loop_wcbench.sh -l
• WCBench version: bc1594d334afefed6742d72eee8ad5d4025d9aa9

Results

Discussion

CBench average flows/second results are normal for a while, then drop to 0 and stay there. RAM uses increases in a step-wise way over time. CPU usage for ODL drops to ~5%, while CBench pegs at 100% (it's single-threaded).

Graphs

Stats

[~/wcbench]$ ./stats.py -S                                                                                                                                  13:32:39
{'fifteen_load': {'max': 6,
                  'mean': 2.399,
                  'min': 0,
                  'relstddev': 89.412,
                  'stddev': 2.145},
 'five_load': {'max': 7,
               'mean': 2.393,
               'min': 0,
               'relstddev': 92.603,
               'stddev': 2.216},
 'flows': {'max': 11752,
           'mean': 2023.489,
           'min': 0,
           'relstddev': 172.08,
           'stddev': 3482.021},
 'iowait': {'max': 0, 'mean': 0.0, 'min': 0, 'relstddev': 0.0, 'stddev': 0.0},
 'max_flows': {'max': 13922,
               'mean': 2424.112,
               'min': 0,
               'relstddev': 172.123,
               'stddev': 4172.444},
 'min_flows': {'max': 8736,
               'mean': 1373.421,
               'min': 0,
               'relstddev': 174.74,
               'stddev': 2399.915},
 'one_load': {'max': 8,
              'mean': 2.356,
              'min': 0,
              'relstddev': 96.18,
              'stddev': 2.266},
 'runtime': {'max': 103,
             'mean': 101.164,
             'min': 101,
             'relstddev': 0.37,
             'stddev': 0.374},
 'sample_size': 652,
 'steal_time': {'max': 0,
                'mean': 0.0,
                'min': 0,
                'relstddev': 0.0,
                'stddev': 0.0},
 'used_ram': {'max': 3904,
              'mean': 3509.695,
              'min': 3259,
              'relstddev': 5.438,
              'stddev': 190.87}}

Raw Results

Raw results can be found here.