Tune ProgPoW for similar hashrate to Ethash's

[solardiz]

We might want to either tune ProgPoW to produce a similar hashrate to what Ethash produces on same currently relevant hardware and at same DAG size, or document a rationale why we don't.

Right now, ProgPoW produces hashrate that is numerically significantly below Ethash's. This may require hard-coding a difficulty scaling factor to apply on a switchover from Ethash to ProgPoW, and it has a psychological effect of making ProgPoW appear "worse".

Why don't we reduce PROGPOW_CNT_DAG to a level where the hashrates are similar, effectively hard-coding this scaling factor into ProgPoW itself? I understand they can't be exactly the same since the current hashrate ratio between the two hashes varies across different GPUs, but at least we can reduce the difference between ProgPoW's and Ethash's numeric hashrate and eliminate the need for an external scaling factor.

To support the trick I described in #26 (comment) we might have some constraints on optimal values for PROGPOW_CNT_DAG - e.g., it'd need to be a multiple of 4 in order to avoid fetching a smaller last group of blocks in the example in that comment - but this still leaves us with a lot of freedom for adjusting the value.

[lookfirst]

While I agree with you that the differences in numbers opens questions, I disagree about a need to artificially inflate or 'trick' the numbers. Lower hashrate can easily be explained to people through education and adding tricks feels disingenuous. From the other angle, what if the hashrate was actually higher? Would you have worked to lower it? ;-)

I think the 3rd party analysis should absolutely evaluate the switchover effects on the difference in hashrate. I've brought this up before in the chats and was told it shouldn't be an issue, but I'd still like to see that simulated on a testnet. There might be a very small window of opportunity for someone to do something nefarious there.

I'd also like to see the testing of tuning knobs on a subsequent hard fork once ProgPoW is deployed. It may be needed in the future, so therefore can be tested now.

[vasilevskykv]

First of all, please, describe in details meaning of LANES?
They are PCI Lanes or simpy - necessary parameters for algorithm?

[solardiz]

@lookfirst I disagree it'd be "artificially inflate or 'trick' the numbers". They're arbitrary as they are. Right now, they're based on keeping PROGPOW_CNT_DAG fixed, but why not e.g. keep total data transfer from DAG fixed (same as Ethash's) instead? That would make more sense to me, or we can also have something inbetween. If the hashrate were actually much higher, yes we could tweak to make it lower again. If it'd be just slightly higher (say, 10%), we could keep that as-is.

Then you go on to say you do have concerns about the switchover to different hashrate. This is a reason in favor of tuning ProgPoW to have a similar hashrate.

Finally, you bring up a valid point about using this as a test of hard-forking to a different hashrate PoW, where this experience could be reused later. It is possible that some other future PoW wouldn't be easily tunable to match the previous PoW's hashrate.

@vasilevskykv I find your questions irrelevant to this discussion. I didn't mention lanes, and the tuning I suggested doesn't involve any change related to lanes. That said, they're not PCIe lanes, but are necessary parameters of the algorithm related to other properties of the GPU hardware (unrelated to PCIe).

[solardiz]

I think the primary reason to possibly not bother tuning ProgPoW to match Ethash's hashrate on GPUs is that by the time Ethereum possibly switches to ProgPoW it will probably be mined largely on ASICs rather than GPUs, so there will be a major drop in hashpower regardless of how the two PoWs' hashrates compare to each other on GPU. That said, configuring ProgPoW to match Ethash's total data transfer from DAG would make sense to me even under such circumstances, and the disappearance of ASICs would need to be dealt with separately either way (tricky - e.g., might require a dual-PoW transition period, with its associated risks).

[lookfirst]

Right now, they're based on keeping PROGPOW_CNT_DAG fixed, but why not e.g. keep total data transfer from DAG fixed (same as Ethash's) instead?

It seems like more than just that one parameter. You also bring up a very good point that I'd love @ifdefelse to provide clarity on... why was the parameter changed from the Ethash setting?

Then you go on to say you do have concerns about the switchover to different hashrate. This is a reason in favor of tuning ProgPoW to have a similar hashrate.

Thank you for calling me out on that. You're indeed correct there.

What I'm thinking though is that the testing and potential code changes for a large change in hashrate needs to happen regardless. For example, it gets deployed and some bug causes all the AMD cards to stop working.

[solardiz]

For hashrate tuning, it is just PROGPOW_CNT_DAG. We'll first tweak the code and tune other parameters for other good reasons - they will affect the hashrate as well - and once we're done tweaking the code and tuning those other parameters, we can tune PROGPOW_CNT_DAG (only) as the final step to achieve the desired hashrate. This parameter value is currently unchanged since Ethash - the corresponding ETHASH_ACCESSES was also 64. However, many other things about ProgPoW are different from Ethash - most relevantly, the data transfer per access is currently doubled (and we may consider increasing it even further to make Maxwell great again) - so it feels natural to me to compensate for that by decreasing PROGPOW_CNT_DAG.

[ifdefelse]

You're right that a simple parameter tune could make the ProgPoW hashrate pretty close to Ethash. Ethash does 64 reads each of 128 bytes, reading 8kb of memory DAG total. ProgPoW keeps the same number of reads at 64 but doubles the size to 256 bytes in order to be efficient on GDDR5x and GDDR6 memories. This results in reading 16kb of DAG total and ~1/2 the hash rate. The read count could be dropped to 32 with each read being 256 bytes to retain the 8kb of DAG read and nominally similar hashrates.

The original ethash documentation says: https://github.com/ethereum/wiki/wiki/Ethash-Design-Rationale

64 accesses was chosen because a larger number of accesses would lead to light verification taking too long, and a smaller number would mean that the bulk of the time consumption is the SHA3 at the end, not the memory reads, making the algorithm not so strongly IO-bound.

The current ProgPoW spec cuts the keccak size in half while doubling the memory accessed, making the keccak effectively 1/4th as important. This was an intentional design choice to prevent acorn-style offloading of the keccak. Reducing the access count to 32 would only make the keccak 1/2 as important, which is still better than Ethash.

We think doubling the read size to 256 bytes is important for now-current memory technologies. The pros for the two choices are:

64 reads:

• keeps the existing ethash iteration count (if it's not broken don't change it)
• better at preventing acorn-style offloading

32 reads:

• faster light client verification
• nominally the same hashrates as ethash

[solardiz]

All good points.

I think it'd take something inbetween 32 and 64 to have "nominally the same hashrates as ethash" due to ProgPoW's increased memory bandwidth usage. Maybe 40 or so.

[tromp]

I remember once trying to pick a certain cycle length PoW parameter between 32 and 64...

[lookfirst]

@shemnon Discusses this here: https://ethereum-magicians.org/t/eip-progpow-a-programmatic-proof-of-work/272/13

[shemnon]

The doubling of the required memory bandwith alone makes it impossible to match the EthHash hashrate on the same hardware.