Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connect to eth node #1580

Open
wants to merge 113 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
51c6091
Initial commit of connect-to-eth-node.
smoelius Jan 5, 2020
87e3b44
Add web3 to install_requires.
smoelius Jan 5, 2020
56a7bab
Ignore web3 types.
smoelius Jan 5, 2020
b395920
Handle symbolic offsets.
smoelius Jan 6, 2020
d16736a
Add typeguard to install_requires.
smoelius Jan 6, 2020
6df79e6
Blacken.
smoelius Jan 6, 2020
1b50ce0
Better mypy integration.
smoelius Jan 6, 2020
5afa862
More blackening.
smoelius Jan 6, 2020
254c6c8
Limit paths that typeguard checks.
smoelius Jan 6, 2020
d58368c
Fix broken get_storage_data types.
smoelius Jan 6, 2020
ffa39b7
Blacken.
smoelius Jan 6, 2020
7d654aa
Revert some changes related to the callstack.
smoelius Jan 7, 2020
46f7a30
Delete accounts unconditionally.
smoelius Jan 7, 2020
b1a82f7
Do not throw an exception if deleted account is not in dictionary.
smoelius Jan 7, 2020
d40d143
Add constraint argument to delete_account call.
smoelius Jan 8, 2020
043f9c0
Back out typeguard.
smoelius Jan 8, 2020
5a66521
Use "not in".
smoelius Jan 8, 2020
9eff254
Fix minor indentation issue in ci.yml.
smoelius Jan 8, 2020
081f4b4
"fromDict" -> "from_dict".
smoelius Jan 8, 2020
bc0cb70
Add (failing) test_gas_check.
smoelius Jan 9, 2020
32a19ea
Fix gas check.
smoelius Jan 9, 2020
4cbad00
Merge commit '32a19ea' into dev-connect-to-eth-node
smoelius Jan 9, 2020
8b0a5ef
Use Storage.set in set_storage_data.
smoelius Jan 9, 2020
e18fd7c
Consider BitVecConstant not symbolic in dump.
smoelius Jan 9, 2020
912366b
Add tests.
smoelius Jan 9, 2020
6222aca
Merge commit '9eff254' into dev-connect-to-eth-node
smoelius Jan 10, 2020
eec85f9
Do not thrown an exception when "evm.trace" is not present.
smoelius Jan 10, 2020
51d4494
Reduce Storage warnings.
smoelius Jan 10, 2020
566620e
Address Web3 forking bug.
smoelius Jan 10, 2020
a1d5cdd
Use count_ready_states in test_dump.
smoelius Jan 10, 2020
80c0f1f
Add command line arguments.
smoelius Jan 10, 2020
ca7d215
Run truffle tests with ganache-cli as the backend.
smoelius Jan 10, 2020
1855689
Merge remote-tracking branch 'origin/master' into dev-connect-to-eth-…
smoelius Jan 10, 2020
244554a
Don't try to convert None to an int.
smoelius Jan 10, 2020
9c871cb
Separate out truffle tests.
smoelius Jan 10, 2020
468036e
Add '-p' to mkdir, and clean-up properly otherwise.
smoelius Jan 10, 2020
6c15093
Don't try to unbox over an already unboxed truffle box.
smoelius Jan 10, 2020
920d28d
Remove unused import in evm_world_state.py.
smoelius Jan 10, 2020
3f15747
Remove unused imports from test_general.py.
smoelius Jan 10, 2020
4944d67
Simplify ci.yml
smoelius Jan 10, 2020
e1bfb4b
Merge commit '4944d67' into dev-connect-to-eth-node
smoelius Jan 11, 2020
d298c19
Remove unnecessary assignment in _web3.
smoelius Jan 13, 2020
72c95b6
Do not enable uninitialized storage detector when using RPC.
smoelius Jan 13, 2020
758e8c7
Typo.
smoelius Apr 15, 2020
053b393
Do not require argv[0] when --rpc and --txvictim are given.
smoelius Apr 15, 2020
1d0cebd
Add tests.
smoelius Apr 15, 2020
ca67c9c
Merge remote-tracking branch 'origin/master' into dev-connect-to-eth-…
smoelius Apr 15, 2020
abdca9e
Typo.
smoelius Apr 15, 2020
ef328af
Address typing issues.
smoelius Apr 16, 2020
b0a38fd
Report errors in ci.yml.
smoelius Apr 16, 2020
913f31f
Add EXPECTED_TX to ci.yml.
smoelius Apr 16, 2020
2ff301e
Better variable name in ci.yml.
smoelius Apr 16, 2020
bc5085b
Fix types.
smoelius Apr 23, 2020
e0c7272
Create DefaultWorldState only once.
smoelius Apr 23, 2020
d15cddf
"--txvictim" -> "--txtarget".
smoelius Apr 23, 2020
cd3b7e6
Create accounts when not using RPC.
smoelius Apr 23, 2020
0d3254a
Add bad IP/port tests.
smoelius Apr 24, 2020
06da949
Update option description.
smoelius Apr 24, 2020
7454fa4
Raise EthereumError on connection failure.
smoelius Apr 28, 2020
fd562db
Move OverlayWorldState comment into docstring.
smoelius Apr 28, 2020
09879e1
Store newly created Storage in world_state.
smoelius Apr 28, 2020
462982a
Add error message in OverlayWorldState.get_storage.
smoelius Apr 28, 2020
a1b06de
Add constraints parameter to WorldState.get_storage_data.
smoelius Apr 30, 2020
fbebf32
Change how Storage objects are allocated.
smoelius Apr 30, 2020
ea1b2bd
Add EVMWorld.get_storage.
smoelius Apr 30, 2020
92a1849
Add storage tests.
smoelius Apr 30, 2020
b3089b7
Blacken.
smoelius Apr 30, 2020
a7d090a
Re-add EVMWorld.get_storage docstring.
smoelius Apr 30, 2020
954e92f
Correct EVMWorld.get_storage docstring.
smoelius Apr 30, 2020
a3e70e2
Add missing newline.
smoelius May 1, 2020
18bf648
Better error handling/reporting.
smoelius May 1, 2020
ee206c4
Fix error in ci.yml.
smoelius May 1, 2020
abe52bc
Add EVMWorld.get_storage_items.
smoelius May 1, 2020
53117d2
Adjust z3 options.
smoelius May 1, 2020
6ccbd49
Experimenting with Storage simplifications.
smoelius May 4, 2020
88ddfbc
Add nonce checks to test_gas_check.
smoelius May 4, 2020
556a492
Remove reference to map.
smoelius May 8, 2020
56a38f0
Reduce log messages.
smoelius May 8, 2020
a5c2ddc
Eliminate ci.yml conflicts.
smoelius May 8, 2020
af3cc7d
Merge branch 'master' into dev-connect-to-eth-node
smoelius May 8, 2020
eecd9b5
Update ci.yml
smoelius May 8, 2020
44e6c0a
Do not store constraints in Storage.
smoelius May 15, 2020
c3ae87f
Eliminate use of dirty.
smoelius May 15, 2020
747ad69
Rename Storage.data to Storage._data.
smoelius May 15, 2020
b3558fe
Merge remote-tracking branch 'origin/master' into dev-connect-to-eth-…
smoelius May 17, 2020
235d0b5
Add docstring to Storage.new_storage.
smoelius May 24, 2020
6054033
Move get_items into ArrayProxy.
smoelius May 25, 2020
8f59f39
Merge remote-tracking branch 'origin/master' into dev-connect-to-eth-…
smoelius May 25, 2020
49707bd
Merge remote-tracking branch 'origin/master' into dev-connect-to-eth-…
smoelius Jun 11, 2020
45b87b8
Correct bitvec width in add_to/sub_from_balance.
smoelius Jun 11, 2020
60daa52
Blacken.
smoelius Jun 11, 2020
b8880cf
Interpret coinbase as hex.
smoelius Jun 11, 2020
c4a0471
Fiz __getstae__/__setstate__ names.
smoelius Jun 11, 2020
15d5366
Add comment re disabling uninitialized storage detector.
smoelius Jun 11, 2020
3a01360
Set blockheader information.
smoelius Jun 11, 2020
1de8eba
Use self.get_storage.
smoelius Jun 11, 2020
91d48ed
Use self._set_storage.
smoelius Jun 11, 2020
02a540e
Nominal changes.
smoelius Jun 11, 2020
a4c599d
Callstack type has changed.
smoelius Jun 11, 2020
2215ba8
Eliminate unused imports.
smoelius Jun 12, 2020
6b3dbf3
Simplify Storage.get.
smoelius Jun 12, 2020
c29a24a
Fix bugs introduced by merge.
smoelius Jun 12, 2020
f257ec8
More merge bug fixes.
smoelius Jun 12, 2020
399dd08
Handle case where EXP's exponent is a symbolic constant.
smoelius Jun 12, 2020
7a38281
Turn off detectors in truffle maze test.
smoelius Jun 12, 2020
3446e21
Implement Storage.__copy__.
smoelius Jun 12, 2020
cc97a38
Convert gas to constant in make_VMTests.py.
smoelius Jun 12, 2020
94055f9
Coverage fixes in ci.yml.
smoelius Jun 12, 2020
8af6869
More ci.yml fixes.
smoelius Jun 12, 2020
4a23d8a
Up SMT timeout for metacoin tests.
smoelius Jun 12, 2020
3fc4264
Fix bug concerning balance width.
smoelius Jun 12, 2020
ebd8cc0
Balance width fix was incomplete.
smoelius Jun 12, 2020
cca9dbf
Eliminate new_storage method.
smoelius Jun 12, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Change how Storage objects are allocated.
smoelius committed Apr 30, 2020
commit fbebf327bd5a96ed4f7fde8eab898856ffd2fd08
4 changes: 2 additions & 2 deletions manticore/platforms/evm.py
Original file line number Diff line number Diff line change
@@ -2581,14 +2581,14 @@ def _get_storage(self, constraints: ConstraintSet, address: int) -> Storage:
"""Private auxiliary function to retrieve the storage"""
storage = self._world_state.get_storage(address)
if storage is None:
storage = Storage(constraints, address)
storage = self._world_state.new_storage(constraints, address)
self._world_state.set_storage(address, storage)
return storage

def _set_storage(self, address: int, storage: Union[Dict[int, int], Optional[Storage]]):
"""Private auxiliary function to replace the storage"""
if isinstance(storage, dict):
storage = Storage.from_dict(self.constraints, address, storage)
storage = self._world_state.new_storage(self.constraints, address, storage)
self._world_state.set_storage(address, storage)

def get_nonce(self, address: int) -> Union[int, BitVec]:
31 changes: 21 additions & 10 deletions manticore/platforms/evm_world_state.py
Original file line number Diff line number Diff line change
@@ -18,21 +18,30 @@

class Storage:
def __init__(self, constraints: ConstraintSet, address: int):
"""
:param constraints: the ConstraintSet with which this Storage object is associated
:param address: the address that owns this storage
"""
self.constraints = constraints
smoelius marked this conversation as resolved.
Show resolved Hide resolved
self.warned = False
self.map = constraints.new_array(
index_bits=256,
value_bits=1,
name=f"STORAGE_MAP_{address:x}",
avoid_collisions=True,
# sam.moelius: The use of default here induces a kind of "closed world assumption,"
# i.e., the only writes that occur to this storage are those that we observe. See
# ArrayProxy.get in expression.py.
default=0,
)
self.data = constraints.new_array(
index_bits=256,
value_bits=256,
name=f"STORAGE_DATA_{address:x}",
avoid_collisions=True,
default=0,
# sam.moelius: The use of default here creates unnecessary if-then-elses. See
# ArrayProxy.get in expression.py.
# default=0,
)
self.dirty = False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.dirty != ( len(self.data.written) !=0 ) ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect they are are equal.

Accessing an object's data members feels wrong to me, but I take it you prefer that to having dirty?

Let me know. I am happy to make the change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am operating on the idea that if the member is not prepended with an underscore it is good for general use. 🤷
But other than that the question I'm trying to answer is why do we need Storage instead of ArrayProxy. (Other than its better naming)

Copy link
Contributor Author

@smoelius smoelius May 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this point, Storage is just a thin wrapper around ArrayProxy. It adds a few methods and sets some defaults in the constructor, but that's about it.


@@ -51,18 +60,20 @@ def dump(self, stream: TextIOBase, state: State):
f"storage[{index:x}] = {state.solve_one(self.data[index], constrain=True):x}"
)

@staticmethod
def from_dict(constraints: ConstraintSet, address: int, items: Dict[int, int]) -> "Storage":
storage = Storage(constraints, address)
for key, value in items.items():
storage.set(key, value)
return storage


####################################################################################################


class WorldState:
def new_storage(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would be the docstring of this method? Is this a convenience method?

I'm struggling to think when do we want a storage without an account.
I expect a world state to have add_account(), delete_account(), accounts()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would be the docstring of this method? Is this a convenience method?

I added the docstring. It is a convenience method.

"""
Private auxiliary function to consruct a new storage object
:param constraints: the ConstraintSet with which this Storage object is associated
:param address: the address that owns this storageq
:param items: optional items to populate the storage with
"""

I'm struggling to think when do we want a storage without an account.
I expect a world state to have add_account(), delete_account(), accounts()

I don't think those methods make sense for the remote model, because they presume that an account "exists" or "does not exist". What is more accurate is there are things that we know about an address and things that we do not, which could include nonce, balance, storage, and code. That is why I espouse the removal of the accounts method.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They do not make sense for the special case of a remote world. I guess those will not be implemented there. And the layer on top of it will need to handle that functionality.

World State looks like something that needs a way to add, delete and list account (or to check if it contains an account)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And the layer on top of it will need to handle that functionality.

Why do you think those methods are needed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A method to add an account? So the user can add accounts to the ethereum world.
A method to del an account? I guess selfdestruct wants it.
A method to list accounts? To concretize target addresses for example.
A method to check if an account exists? You need to know if the account is already there to calculate gas in tx instructions sometimes.

self, constraints: ConstraintSet, address: int, items: Optional[Dict[int, int]] = None
) -> Storage:
storage = Storage(constraints, address)
if items is not None:
for key, value in items.items():
storage.set(key, value)
return storage

@abstractmethod
def is_remote(self) -> bool:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing I'll be better the less we actually use this method.

Copy link
Contributor Author

@smoelius smoelius May 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I totally agree.

pass
@@ -409,7 +420,7 @@ def delete_account(self, constraints: ConstraintSet, address: int):
default_world_state = DefaultWorldState()
self._nonce[address] = default_world_state.get_nonce(address)
self._balance[address] = default_world_state.get_balance(address)
self._storage[address] = Storage(constraints, address)
self._storage[address] = self.new_storage(constraints, address)
self._code[address] = default_world_state.get_code(address)
self._deleted_accounts.add(address)

@@ -434,7 +445,7 @@ def set_storage_data(
):
storage = self._storage.get(address)
if storage is None:
storage = Storage(constraints, address)
storage = self.new_storage(constraints, address)
self._storage[address] = storage
if storage.constraints != constraints:
if not storage.warned: