Skip to content

Commit

Permalink
New benchmarks for geth
Browse files Browse the repository at this point in the history
  • Loading branch information
@holiman
holiman committed Jul 14, 2018
1 parent d27bdf2 commit 9685ac4
Show file tree
Hide file tree
Showing 7 changed files with 667 additions and 0 deletions.
68 changes: 68 additions & 0 deletions constantinople/processed/geth_20180714_mhswende.txt.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
```
Name Gascost Time (ns) MGas/S Gasprice for 10MGas/S Gasprice for ECDSA eq
----------------------------------------- --------- ---------------- --------- ----------------------- -----------------------
PrecompiledEcrecover/ 3000 159077 18.8588 1590.77 3000
PrecompiledSha256/128 108 639 169.014 6.39 12.0508
PrecompiledRipeMD/128 1080 2030 532.02 20.3 38.2833
PrecompiledIdentity/128 27 17.2 1569.77 0.172 0.324371
PrecompiledModExp/eip_example1 13056 34735 375.874 347.35 655.06
PrecompiledModExp/eip_example2 13056 7713 1692.73 77.13 145.458
PrecompiledModExp/nagydani-1-square 204 2590 78.7645 25.9 48.8443
PrecompiledModExp/nagydani-1-qube 204 3357 60.7685 33.57 63.309
PrecompiledModExp/nagydani-1-pow0x10001 3276 13991 234.151 139.91 263.853
PrecompiledModExp/nagydani-2-square 665 4215 157.77 42.15 79.4898
PrecompiledModExp/nagydani-2-qube 665 6248 106.434 62.48 117.83
PrecompiledModExp/nagydani-2-pow0x10001 10649 31272 340.528 312.72 589.752
PrecompiledModExp/nagydani-3-square 1894 7558 250.595 75.58 142.535
PrecompiledModExp/nagydani-3-qube 1894 12706 149.063 127.06 239.62
PrecompiledModExp/nagydani-3-pow0x10001 30310 78718 385.045 787.18 1484.53
PrecompiledModExp/nagydani-4-square 5580 18090 308.458 180.9 341.156
PrecompiledModExp/nagydani-4-qube 5580 36116 154.502 361.16 681.104
PrecompiledModExp/nagydani-4-pow0x10001 89292 207740 429.826 2077.4 3917.73
PrecompiledModExp/nagydani-5-square 17868 45934 388.993 459.34 866.26
PrecompiledModExp/nagydani-5-qube 17868 99434 179.697 994.34 1875.21
PrecompiledModExp/nagydani-5-pow0x10001 285900 659933 433.226 6599.33 12445.5
PrecompiledBn256Add/chfast1 500 14068 35.5417 140.68 265.305
PrecompiledBn256Add/chfast2 500 14136 35.3707 141.36 266.588
PrecompiledBn256Add/cdetrio1 500 1032 484.496 10.32 19.4623
PrecompiledBn256Add/cdetrio2 500 1110 450.45 11.1 20.9333
PrecompiledBn256Add/cdetrio3 500 1189 420.521 11.89 22.4231
PrecompiledBn256Add/cdetrio4 500 1135 440.529 11.35 21.4047
PrecompiledBn256Add/cdetrio5 500 1199 417.014 11.99 22.6117
PrecompiledBn256Add/cdetrio6 500 1417 352.858 14.17 26.7229
PrecompiledBn256Add/cdetrio7 500 1495 334.448 14.95 28.1939
PrecompiledBn256Add/cdetrio8 500 1552 322.165 15.52 29.2688
PrecompiledBn256Add/cdetrio9 500 1611 310.366 16.11 30.3815
PrecompiledBn256Add/cdetrio10 500 1440 347.222 14.4 27.1567
PrecompiledBn256Add/cdetrio11 500 14456 34.5877 144.56 272.623
PrecompiledBn256Add/cdetrio12 500 14485 34.5185 144.85 273.17
PrecompiledBn256Add/cdetrio13 500 14315 34.9284 143.15 269.964
PrecompiledBn256Add/cdetrio14 500 2164 231.054 21.64 40.8104
PrecompiledBn256ScalarMul/chfast1 40000 97875 408.685 978.75 1845.8
PrecompiledBn256ScalarMul/chfast2 40000 105280 379.939 1052.8 1985.45
PrecompiledBn256ScalarMul/chfast3 40000 101911 392.499 1019.11 1921.92
PrecompiledBn256ScalarMul/cdetrio1 40000 108458 368.806 1084.58 2045.39
PrecompiledBn256ScalarMul/cdetrio6 40000 107855 370.868 1078.55 2034.01
PrecompiledBn256ScalarMul/cdetrio11 40000 108114 369.98 1081.14 2038.9
PrecompiledBn256Pairing/jeff1 260000 3.2591e+06 79.7766 32591 61462.7
PrecompiledBn256Pairing/jeff2 260000 3.20302e+06 81.1735 32030.2 60405
PrecompiledBn256Pairing/jeff3 260000 3.23077e+06 80.4762 32307.7 60928.4
PrecompiledBn256Pairing/jeff4 340000 4.32009e+06 78.7021 43200.9 81471.7
PrecompiledBn256Pairing/jeff5 340000 4.31471e+06 78.8003 43147.1 81370.2
PrecompiledBn256Pairing/jeff6 260000 3.25083e+06 79.9796 32508.3 61306.7
PrecompiledBn256Pairing/empty_data 100000 1.02727e+06 97.3457 10272.7 19373
PrecompiledBn256Pairing/one_point 180000 2.13090e+06 84.4712 21309.1 40186.3
PrecompiledBn256Pairing/two_point_match_2 260000 3.18384e+06 81.6623 31838.4 60043.5
PrecompiledBn256Pairing/two_point_match_3 260000 3.18156e+06 81.7208 31815.6 60000.5
PrecompiledBn256Pairing/two_point_match_4 260000 3.22249e+06 80.6828 32224.9 60772.3
PrecompiledBn256Pairing/ten_point_match_1 900000 1.17622e+07 76.5164 117622 221821
PrecompiledBn256Pairing/ten_point_match_2 900000 1.1861e+07 75.8787 118610 223685
PrecompiledBn256Pairing/ten_point_match_3 260000 3.18667e+06 81.5899 31866.7 60096.7
```

Columns

* `MGas/S` - Shows what MGas per second was measured on that machine at that time
* `Gasprice for 10MGas/S` shows what the gasprice should have been, in order to reach 10 MGas/second
* `Gasprice for ECDSA eq` shows what the gasprice should have been, in order to have the same cost/cycle as ecRecover

67 changes: 67 additions & 0 deletions constantinople/rawdata/geth_20180714_mhswende.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
[user@work vm]$ cat /proc/cpuinfo | grep "model name"| head
model name : Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz


[user@work vm]$ go test -bench BenchmarkPrecompiled -benchtime 5s
goos: linux
goarch: amd64
pkg: github.com/ethereum/go-ethereum/core/vm
BenchmarkPrecompiledEcrecover/-Gas=3000-2 50000 159077 ns/op
BenchmarkPrecompiledSha256/128-Gas=108-2 10000000 639 ns/op
BenchmarkPrecompiledRipeMD/128-Gas=1080-2 3000000 2030 ns/op
BenchmarkPrecompiledIdentity/128-Gas=27-2 500000000 17.2 ns/op
BenchmarkPrecompiledModExp/eip_example1-Gas=13056-2 200000 34735 ns/op
BenchmarkPrecompiledModExp/eip_example2-Gas=13056-2 1000000 7713 ns/op
BenchmarkPrecompiledModExp/nagydani-1-square-Gas=204-2 3000000 2590 ns/op
BenchmarkPrecompiledModExp/nagydani-1-qube-Gas=204-2 2000000 3357 ns/op
BenchmarkPrecompiledModExp/nagydani-1-pow0x10001-Gas=3276-2 500000 13991 ns/op
BenchmarkPrecompiledModExp/nagydani-2-square-Gas=665-2 2000000 4215 ns/op
BenchmarkPrecompiledModExp/nagydani-2-qube-Gas=665-2 1000000 6248 ns/op
BenchmarkPrecompiledModExp/nagydani-2-pow0x10001-Gas=10649-2 200000 31272 ns/op
BenchmarkPrecompiledModExp/nagydani-3-square-Gas=1894-2 1000000 7558 ns/op
BenchmarkPrecompiledModExp/nagydani-3-qube-Gas=1894-2 500000 12706 ns/op
BenchmarkPrecompiledModExp/nagydani-3-pow0x10001-Gas=30310-2 100000 78718 ns/op
BenchmarkPrecompiledModExp/nagydani-4-square-Gas=5580-2 500000 18090 ns/op
BenchmarkPrecompiledModExp/nagydani-4-qube-Gas=5580-2 200000 36116 ns/op
BenchmarkPrecompiledModExp/nagydani-4-pow0x10001-Gas=89292-2 30000 207740 ns/op
BenchmarkPrecompiledModExp/nagydani-5-square-Gas=17868-2 200000 45934 ns/op
BenchmarkPrecompiledModExp/nagydani-5-qube-Gas=17868-2 100000 99434 ns/op
BenchmarkPrecompiledModExp/nagydani-5-pow0x10001-Gas=285900-2 10000 659933 ns/op
BenchmarkPrecompiledBn256Add/chfast1-Gas=500-2 500000 14068 ns/op
BenchmarkPrecompiledBn256Add/chfast2-Gas=500-2 500000 14136 ns/op
BenchmarkPrecompiledBn256Add/cdetrio1-Gas=500-2 10000000 1032 ns/op
BenchmarkPrecompiledBn256Add/cdetrio2-Gas=500-2 10000000 1110 ns/op
BenchmarkPrecompiledBn256Add/cdetrio3-Gas=500-2 5000000 1189 ns/op
BenchmarkPrecompiledBn256Add/cdetrio4-Gas=500-2 10000000 1135 ns/op
BenchmarkPrecompiledBn256Add/cdetrio5-Gas=500-2 10000000 1199 ns/op
BenchmarkPrecompiledBn256Add/cdetrio6-Gas=500-2 5000000 1417 ns/op
BenchmarkPrecompiledBn256Add/cdetrio7-Gas=500-2 5000000 1495 ns/op
BenchmarkPrecompiledBn256Add/cdetrio8-Gas=500-2 5000000 1552 ns/op
BenchmarkPrecompiledBn256Add/cdetrio9-Gas=500-2 5000000 1611 ns/op
BenchmarkPrecompiledBn256Add/cdetrio10-Gas=500-2 5000000 1440 ns/op
BenchmarkPrecompiledBn256Add/cdetrio11-Gas=500-2 500000 14456 ns/op
BenchmarkPrecompiledBn256Add/cdetrio12-Gas=500-2 500000 14485 ns/op
BenchmarkPrecompiledBn256Add/cdetrio13-Gas=500-2 500000 14315 ns/op
BenchmarkPrecompiledBn256Add/cdetrio14-Gas=500-2 3000000 2164 ns/op
BenchmarkPrecompiledBn256ScalarMul/chfast1-Gas=40000-2 100000 97875 ns/op
BenchmarkPrecompiledBn256ScalarMul/chfast2-Gas=40000-2 100000 105280 ns/op
BenchmarkPrecompiledBn256ScalarMul/chfast3-Gas=40000-2 100000 101911 ns/op
BenchmarkPrecompiledBn256ScalarMul/cdetrio1-Gas=40000-2 100000 108458 ns/op
BenchmarkPrecompiledBn256ScalarMul/cdetrio6-Gas=40000-2 100000 107855 ns/op
BenchmarkPrecompiledBn256ScalarMul/cdetrio11-Gas=40000-2 100000 108114 ns/op
BenchmarkPrecompiledBn256Pairing/jeff1-Gas=260000-2 2000 3259099 ns/op
BenchmarkPrecompiledBn256Pairing/jeff2-Gas=260000-2 2000 3203017 ns/op
BenchmarkPrecompiledBn256Pairing/jeff3-Gas=260000-2 2000 3230769 ns/op
BenchmarkPrecompiledBn256Pairing/jeff4-Gas=340000-2 2000 4320089 ns/op
BenchmarkPrecompiledBn256Pairing/jeff5-Gas=340000-2 2000 4314707 ns/op
BenchmarkPrecompiledBn256Pairing/jeff6-Gas=260000-2 2000 3250827 ns/op
BenchmarkPrecompiledBn256Pairing/empty_data-Gas=100000-2 10000 1027267 ns/op
BenchmarkPrecompiledBn256Pairing/one_point-Gas=180000-2 3000 2130905 ns/op
BenchmarkPrecompiledBn256Pairing/two_point_match_2-Gas=260000-2 2000 3183844 ns/op
BenchmarkPrecompiledBn256Pairing/two_point_match_3-Gas=260000-2 2000 3181564 ns/op
BenchmarkPrecompiledBn256Pairing/two_point_match_4-Gas=260000-2 2000 3222494 ns/op
BenchmarkPrecompiledBn256Pairing/ten_point_match_1-Gas=900000-2 1000 11762181 ns/op
BenchmarkPrecompiledBn256Pairing/ten_point_match_2-Gas=900000-2 1000 11861035 ns/op
BenchmarkPrecompiledBn256Pairing/ten_point_match_3-Gas=260000-2 2000 3186670 ns/op
PASS
ok github.com/ethereum/go-ethereum/core/vm 510.099s
25 changes: 25 additions & 0 deletions constantinople/scripts/postprocess_all.sh
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/bash

HERE=`pwd`
cd ../rawdata
FILES=geth*
for f in $FILES
do
echo "Processing $f"
python ../scripts/postprocess_geth_v2.py $f > ../processed/$f.md
done
FILES=cpp*
for f in $FILES
do
echo "Processing $f"
python ../scripts/postprocess_cpp.py $f > ../processed/$f.md
done
FILES=parity*
for f in $FILES
do
echo "Processing $f"
python ../scripts/postprocess_parity.py $f > ../processed/$f.md
done

cd $HERE

179 changes: 179 additions & 0 deletions constantinople/scripts/postprocess_cpp.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
#!/usr/bin/python

import sys, re
from tabulate import tabulate

#bench_bn256Add/chfast1: 8508 ns
precompiledRegexp = re.compile("bench_(.*): ([\d]+) ns")

def getGas(precompile):
gasLookupTable = {
"ecrecover/" :3000 ,
"bn256Pairing/jeff1" : 260000 ,
"bn256Pairing/jeff2" : 260000 ,
"bn256Pairing/jeff3" : 260000 ,
"bn256Pairing/jeff4" : 340000 ,
"bn256Pairing/jeff5" : 340000 ,
"bn256Pairing/jeff6" : 260000 ,
"bn256Pairing/empty_data" :100000 ,
"bn256Pairing/one_point" : 180000 ,
"bn256Pairing/two_point_match_2" :260000 ,
"bn256Pairing/two_point_match_3" :260000 ,
"bn256Pairing/two_point_match_4" :260000 ,
"bn256Pairing/ten_point_match_1" :900000 ,
"bn256Pairing/ten_point_match_2" :900000 ,
"bn256Pairing/ten_point_match_3" :260000 ,
"bn256Pairing/jeff1-Gas" : 260000,
"bn256Pairing/jeff2-Gas" : 260000,
"bn256Pairing/jeff3-Gas" : 260000,
"bn256Pairing/jeff4-Gas" : 340000,
"bn256Pairing/jeff5-Gas" : 340000,
"bn256Pairing/jeff6-Gas" : 260000,
"bn256Pairing/empty_data-Gas" : 100000,
"bn256Pairing/one_point-Gas" : 180000,
"modexp/eip_example1" :2611 ,
"modexp/eip_example2" :2611 ,
"modexp/nagydani-1-square" : 40,
"modexp/nagydani-1-qube" : 40,
"modexp/nagydani-1-pow0x10001" : 655,
"modexp/nagydani-2-square" : 133,
"modexp/nagydani-2-qube" : 133,
"modexp/nagydani-2-pow0x10001" : 2129,
"modexp/nagydani-3-square" : 378,
"modexp/nagydani-3-qube" : 378,
"modexp/nagydani-3-pow0x10001" : 6062,
"modexp/nagydani-4-square" : 1116,
"modexp/nagydani-4-qube" : 1116,
"modexp/nagydani-4-pow0x10001" : 17858,
"modexp/nagydani-5-square" : 3573,
"modexp/nagydani-5-qube" : 3573,
"modexp/nagydani-5-pow0x10001" : 57180 ,
}
if precompile.find("bn256ScalarMul") == 0:
return 2000
if precompile.find("bn256Add") == 0:
return 500

if precompile in gasLookupTable.keys():
return gasLookupTable[precompile]

return 0

def calc(line):
for exp in [precompiledRegexp]:#, opRegexp]:
m = exp.search(line)
if m:
(name, ns) = ( m.group(1), m.group(2))
return {"name" : name, "gas": getGas(name), "ns" : float(ns)}


def postprocess(f):
items = []
lines = f.split("\n")
#Do first pass and find the values for the ecdsarecover
idealMGasPerS = 10


for l in lines:
out = calc(l)
if out:
if out['name'].find("ecrecover/") >= 0:
ecdsaMGperS = 1000* out['gas']/out['ns']
#print("Ecdsarecover at %f Mgas/s" % ecdsaMGperS )
ecdsaTime = out['ns']
ecdsaGas = out['gas']

for l in lines:
out = calc(l)
if out:
name = out['name']
gas = out['gas']
timeNs = out['ns']


timeS = timeNs * 1e-9
gasPerNs = float(gas) / float(timeNs)
gasPerS = 1e+9 * gasPerNs
MgasPerS = gasPerS / 1e+6

forEcdsaGasPrice = ecdsaMGperS * 1e6 * timeS
for10MGPSGasPrice = idealMGasPerS * 1e6 * timeS


item = [name, gas, "%14.02f" % float(timeNs), MgasPerS, for10MGPSGasPrice , forEcdsaGasPrice]
#break
items.append(item)
print "```"
print tabulate(items, headers=['Name', 'Gascost', 'Time (ns)', 'MGas/S', 'Gasprice for 10MGas/S','Gasprice for ECDSA eq'])
print "```"
print """
Columns
* `MGas/S` - Shows what MGas per second was measured on that machine at that time
* `Gasprice for 10MGas/S` shows what the gasprice should have been, in order to reach 10 MGas/second
* `Gasprice for ECDSA eq` shows what the gasprice should have been, in order to have the same cost/cycle as ecRecover
"""

def test():
t = """
#cat /proc/cpuinfo | grep "model name" | head -n 1
model name : Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
#cpupower -c 0 frequency-info | grep "CPU frequency"
current CPU frequency is 4.40 GHz.
#taskset 0x1 test/testeth -t 'PrecompiledTests/bench*' -- --verbosity 1
Running 4 test cases...
Test Case "bench_ecrecover":
bench_ecrecover/: 70528 ns
Test Case "bench_bn256Add":
bench_bn256Add/chfast1: 8508 ns
bench_bn256Add/chfast2: 8511 ns
bench_bn256Add/cdetrio1: 2331 ns
bench_bn256Add/cdetrio2: 2328 ns
bench_bn256Add/cdetrio3: 2341 ns
bench_bn256Add/cdetrio4: 2317 ns
bench_bn256Add/cdetrio5: 2327 ns
bench_bn256Add/cdetrio6: 4990 ns
bench_bn256Add/cdetrio7: 4987 ns
bench_bn256Add/cdetrio8: 4993 ns
bench_bn256Add/cdetrio9: 4990 ns
bench_bn256Add/cdetrio10: 4986 ns
bench_bn256Add/cdetrio11: 7602 ns
bench_bn256Add/cdetrio12: 7598 ns
bench_bn256Add/cdetrio13: 8494 ns
bench_bn256Add/cdetrio14: 6605 ns
Test Case "bench_bn256ScalarMul":
bench_bn256ScalarMul/chfast1: 113076 ns
bench_bn256ScalarMul/chfast2: 455600 ns
bench_bn256ScalarMul/chfast3: 454158 ns
Test Case "bench_bn256Pairing":
bench_bn256Pairing/jeff1: 11390929 ns
bench_bn256Pairing/jeff2: 11386363 ns
bench_bn256Pairing/jeff3: 11391780 ns
bench_bn256Pairing/jeff4: 15662968 ns
bench_bn256Pairing/jeff5: 15668956 ns
bench_bn256Pairing/jeff6: 11515980 ns
bench_bn256Pairing/empty_data: 2549477 ns
bench_bn256Pairing/one_point: 7221599 ns
bench_bn256Pairing/two_point_match_2: 11092886 ns
bench_bn256Pairing/two_point_match_3: 11393495 ns
bench_bn256Pairing/two_point_match_4: 11396027 ns
bench_bn256Pairing/ten_point_match_1: 45259353 ns
bench_bn256Pairing/ten_point_match_2: 45700240 ns
bench_bn256Pairing/ten_point_match_3: 11390441 ns
*** No errors detected
"""
postprocess(t)

if __name__ == '__main__':
#test()

if len(sys.argv) < 2:
print("usage : postprocess_geth <file>")
sys.exit(1)

with open(sys.argv[1]) as f:
postprocess(f.read())

Loading

0 comments on commit 9685ac4

Please sign in to comment.