Solid State Transfers

[igormcoelho]

I propose a NEP that will help us give much more control over token operations, specially on Neo 3 (although already applicable for Neo2, even existing NEP-5 tokens could do that without changing contracts). This proposal is simple: allow better balance tracing over storage-based assets, like NEP-5.
On UTXO, it's all connected, the past transactions, past balances, and current balance, giving great auditability capabilities.
On storage-based assets, such as NEP-5, we only execute some transfer operation on the chain, and our balance is automatically updated, but no record is kept for how much existed before (and after).

This NEP requires that assets prove that they exist, before actual transfer.

Example for NEP-5:
transfer myaddress youraddress 100

How much did I have before? zero? 1000? We don't know, unless we process the whole chain at that point (or restore the point with some state trie). This proposal is simple, yet powerful:

Transfer operation NEP-5 (using solid state records):
value = balanceOf myaddress
assert(value == 250) # ok, now I'm sure what my funds were before transfer
transfer myaddress youraddress 100
value2 = balanceOf youraddress
assert(value2 == 600) # ok, now I'm sure what your funds were after transfer

If this transaction passes on chain, we have a evidence for two things: I had 250; and I have transferred 100 to you; and now you have 600.

More elaborate versions of this could require saving previous and after balances, for both addresses, but personally I think that a single "before" register is enough to guarantee quite a few nice properties. For NFT implementations, the count before and after of transferred asset (if countable), or hash state (if hashable); so this is not just intended for NEP-5, but for all onchain assets (not mandatory standard, of course, use it if you like).

Justification:
Neo Blockchain is quite special, in the sense that it doesn't add a solid state reference to block header itself, giving fundamental bugfixing capabilities to the network (that already could have prevented hundreds of hard forks on it). There are many proposals for state tracking on Neo3, in many different formats, however, I believe this proposal here adds much guarantees and auditability to NEP-5 tokens, with or without state tracking. And this is something users (or user wallets) can do for themselves, it's easy and gives strong certainty of instantaneous states on the chain, just by looking at operations themselves.

Drawbacks:
This may not be wanted on situations where several transactions compete to enter a block, in very high throughput operations on same address (like exchanges), as this NEP also fundamentally breaks the possibility of "double-spending" of any kind of token, in any situation. Yet, for common daily transfers it looks quite nice. User wallets could use it to send assets to an exchange, and withdraw operations on that same exchange not necessarily using this (that's why only prev balance is desired, not destination).

[igormcoelho]

@vncoelho @shargon @jsolman @erikzhang @RavenXce

[igormcoelho]

@erikzhang this is a simplified version of NEP #102

If this other passes, we can close this one, as we will be able to do everything here in a much more efficient way (space efficient and more flexible). This doesn't allow correction, just canceling everything. The other allows correction on state too, and re-adaptation, in all cases I could imagine (including asset transfers).

[shargon]

Imagine an ICO process with 5000 pending TX. Only one will work, because the smart contract's balance will change after the first one

[igormcoelho]

Imagine an ICO process with 5000 pending TX. Only one will work, because the smart contract's balance will change after the first one

Not true @shargon, since it only applies to users sending tx from their accounts, not minting. That's precisely why I haven't included in the destination (only origin), as it is redundant anyway, and only reduces TPS with no extra security.

[igormcoelho]

Any thoughts EdgeLT? :)

[igormcoelho]

This could be made even stronger, by adding some post-verification or post-execution verifications. If these fail, state is FAULT, but tx gets into block anyway, which is a different behavior from typical verification.
Maybe it's a simple solution to help dealing with complex contracts, what do you think @erikzhang ?

[igormcoelho]

@shargon please let's put some energy on this... very limited energy, near infinite returns.

Like I explained on other post, even if exchange sends me funds without verification, in next moment, I can send the funds again TO MYSELF, including verification. So users can always ensure that their funds exist and are safe, on Neo2 and Neo3, regardless of any MPT.

[igormcoelho]

@shargon sometimes the best solutions are in front of our eyes... what about using "greater" instead of "equals"? This allows very flexible balance state verification even in face of parallel transactions.

Example: getbalance >= 500

This ensures user/exchange has enough funds on that precise moment, without exposing precise values (thus allowing multiple simultaneous operations).

So, we can have some "strict" verification and some "flexible". Maybe for user wallets its better to have it strict, what do you think?

[igormcoelho]

Thanks a lot @shargon for the "push" on implementation... this will help Neo a lot!

Also finished a first version of text specification: #123

[roman-khimov]

This may not be wanted on situations where several transactions compete to enter a block

But does it really prevent several transactions with the same balance-checking witness from entering the block? Suppose there is a getbalance(A) == 500 witness on transactions T1 and T2 that transfer something from account A. Both T1 and T2 are created and sent at the same time. Witness check is to be done for them independently, so both would be accepted into the mempool and then CN would pick both as valid transactions for the next block. One could recheck these witnesses for in-block transactions (like neo-go does) and even that would work just fine, but upon execution one of these will change the balance and the other one either fail or succeed depending on transfer amounts (just like before this proposal).

[igormcoelho]

The recheck is mandatory for CN and during block processing, otherwise one could make transactions live forever on mempool (if verification script has some time dependency) and also cause double spending, for other verification strategies.

But this competing situation only applies to Strict mode, since Flexible already takes this into account with a >= operation.

But I agree, if verification is done a priori, then it may mislead contracts that depend on verified information... in my opinion, such inconsistencies are quite dangerous from tx perspective and should never occur on clients. The logical order is: Verification, Invocation, (Post Verification, if accepted by community)

Anyway, even in these scenarios, if one tx gets into one block, the second is rejected on the next block, so we cannot rely on such assumptions. I guess this specific verification script situation may require some more discussion on the future, because for me its not acceptable to have a valid witness on contract, whose script is actually failed.

[igormcoelho]

@roman-khimov your analysis was perfectly correct: tx verification will only happen in batch, before actual execution. Anyway, we can never guarantee that two tx issued at same time will reach same block... so, for consistency perspective, its better to adopt the proposed strategy anyway (strict or flexible).
Another turnaround in the proposal is that it may only work in Neo 2, since Neo 3 is likely to have stateless verifications (what, in the end, I should also agree that is a good thing).

So, I think we could make Neo 2 much better with this, and even in short period of time, its worth it.. what do you think? Do you want to join us as co author?

[roman-khimov]

I have too many things on my pending list for Neo 3 at the moment, so thanks, but I'm trying to concentrate more on that one. I'll keep an eye on discussion though.