Skip to content

Commit

Permalink
Release next minor version
Browse files Browse the repository at this point in the history 
* Fix a typo related bug crashing when using virtual_fs (#90)

* Map real files into the virtual filesystem (#116)

* Add add_real_file() method to FileSystem

* Add FileSystem::add_real_file() to python bindings

* `EVM` architecture support (#124)

* Add minimal EVM sleigh spec files

* Add basic classes for Ethereum environment emulation

* Minimal integration for executing Ethereum byte-code, test on ADD instruction

* Add EVM support for DIV, SDIV, MOD, SMOD

* Add more EVM instructions

* More EVM instructions

* Add more EVM instructions

* Add BYTE and POP EVM instructions

* Add MSTORE instruction - failing because of memory endianess

* Add big endian support to MemConcreteBuffer

* Add big endian support for MemSegment

* Add big endian support to MemEngine

* Add big endian support to SymbolicMemEngine

* Support big endian in VarContext and in EVM Memory

* Add more EVM instructions

* Remove sla files from git

* Support PUSH<n> EVM instructions

* More EVM instructions

* Fix call to MaatEngine::process_store()

* Add basic SSTORE and SLOAD support

* Add tests for solving symbolic EVM storage read/writes

* Add smart contract loading test, need to implement more instructions

* Add more EVM operators, add helper class for keccak hashes

* Support KECCAK operation

* Test deployment of simple contract

* More EVM support:

* test executing a transaction
* support STOP, REVERT, CALLDATALOAD, CALLDATACOPY
* minor improvements

* Test return status in EVM test

* WIP: EVM exploration test

* More EVM testing and improvements

* Add copy constructors for memory classes

* Add serialization for Ethereum environment

* Add bindings for Ethereum contracts

* Fix EVM python bindings

* Expose Value type in python module, support building Cst() with big ints

* Python bindings: give 'bin' parameter a default in load(), rename Transaction in EVMTransaction

* Add EVM tx result enum in bindings

* Allow to pass arbitrary data to event callbacks in both native and python APIs

* Support bigints in as_uint() and in VarContext

* Expose more things in python bindings

* Add UIDs to MaatEngine instances

* Allow CmdlineArg to hold a buffer of non-byte abstract values

* Various improvements:

* Support EXP instruction for concrete values
* Support passing constructor arguments when deploying a contract
* Handle bignums in logical AND simplification pattern

* Include <optional> in serializer.hpp

* Bindings: add getter for transaction recipient, expose Info class

* Add constructor to duplicate MaatEngine

* Make storage a shared_ptr in EVM::Contract

* Add helper to create a new EVM runtime from an existing MaatEngine

* Expose engine duplication and EVM runtime helpers in python bindings

* Add concat simplification pattern

* Fix Number signed comparisons implementation

* Simplify code in Number::slessequal_than

* Add more simplification patterns

* Expose Zext and Sext functions in bindings

* Fix rounding for signed division and modulo

* Add abstract counters for block number & timestamp

* Expose PathManager::add() in python bindings

* Add bindings helper functions for Ethereum block info

* EVM: support TIMESTAMP and NUMBER instructions

* Add iterators to EVM::Storage

* Expose EVM Storage in python bindings

* Support EVM CALL instruction

* Support CALLCODE and DELEGATECALL

* Use same class for Transaction and InternalTransaction

* Expose new transaction fields in bindings, rename EVM enums

* Support CALLER instruction

* Support specifying contract addr and deployer through env parameter

* Make transaction addresses 160 bits

* Add VarContext::contained_vars() method

* Force addresses to be 160-bits in EVM transactions

* Support CREATE and CREATE2

* Expose outgoing_transaction to bindings

* EVM loader: allow to pass init bytecode through args, throw exception if the init bytecode doesn't return properly

* Fix CREATE handler

* Expore transaction sender to bindings

* Expose EVM stack to bindings

* Support RETURNDATASIZE

* EVM loader: add option to not run init bytecode when loading the contract

* Expose result_from_last_call in bindings

* Support EXTCODESIZE

* Improve gas support

* Various fixes related to running message calls in EVM

* Don't force args to be 1-byte buffers in bindings

* Support ADDRESS instruction

* Make path manager a shared ptr, allow to share path manager when duplicating an engine

* Fix modulo translation to z3 expressions

* Expose more EVM stuff to bindings

* Fix EVM BYTE instruction

* Support large bitvectors in absorbing element simplification

* Fix unit test for EVM BYTE

* Return bytes in python bindings for VarContext.get_as_str()

* Support checking if the solver timed out

* Fix bug when adding singleton intervals in symbolic memory

* Specify size of variables in MemEngine::make_concolic()

* Improve the API for getting path constraints related to given values/constraints

* Fix checking check() result in z3 solver

* Don't serialize static UID cnt for VarContext

* Bindings: support big unsigned values in VarContext.set()

* Merge fixes for big endian memory support

Co-authored-by: novafacing <[email protected]>
  • Loading branch information
@Boyan-MILANOV @novafacing
Boyan-MILANOV and novafacing authored Jul 7, 2022
1 parent c1fe510 commit 2876019
Show file tree
Hide file tree
Showing 86 changed files with 7,950 additions and 431 deletions.
7 changes: 7 additions & 0 deletions CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ include(cmake/variables.cmake)
# ---- Declare library ----

add_library(maat_maat
src/arch/arch_EVM.cpp
src/arch/arch_X86.cpp
src/arch/lifter.cpp
src/arch/register_aliases.cpp
Expand All @@ -34,6 +35,7 @@ add_library(maat_maat
src/env/emulated_libs/libc.cpp
src/env/emulated_syscalls/linux_syscalls.cpp
src/env/env.cpp
src/env/env_EVM.cpp
src/env/env_linux.cpp
src/env/filesystem.cpp
src/env/library.cpp
Expand All @@ -49,6 +51,7 @@ add_library(maat_maat
src/ir/instruction.cpp
src/ir/ir_cache.cpp
src/loader/loader.cpp
src/loader/loader_EVM.cpp
src/loader/loader_lief.cpp
src/loader/loader_lief_elf.cpp
src/memory/memory.cpp
Expand All @@ -60,6 +63,7 @@ add_library(maat_maat
src/solver/solver.cpp
src/solver/solver_z3.cpp
src/third-party/murmur3/murmur3.c
src/third-party/keccak/sha3.cpp
src/third-party/sleigh/native/reg_translator.cpp
src/third-party/sleigh/native/sleigh_interface.cpp
)
Expand All @@ -81,6 +85,7 @@ target_include_directories(
maat_maat ${warning_guard}
PUBLIC
"$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/src/include>"
"$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/src/third-party/keccak>"
"$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/src/third-party/murmur3>"
)

Expand Down Expand Up @@ -177,11 +182,13 @@ endmacro()

maat_sleigh_compile(x86 x86-64)
maat_sleigh_compile(x86 x86)
maat_sleigh_compile(EVM EVM)

# All of the sla spec targets are combined into this one
add_custom_target(maat_all_sla_specs DEPENDS
maat_sleigh_spec_x86-64
maat_sleigh_spec_x86
maat_sleigh_spec_EVM
)

# Add sla specs as dependencies to our targets
Expand Down
2 changes: 2 additions & 0 deletions bindings/bindings.cmake
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ add_library(maat_python MODULE
bindings/python/py_cpu.cpp
bindings/python/py_engine.cpp
bindings/python/py_env.cpp
bindings/python/py_env_EVM.cpp
bindings/python/py_event.cpp
bindings/python/py_filesystem.cpp
bindings/python/py_info.cpp
Expand Down Expand Up @@ -60,6 +61,7 @@ target_include_directories(
maat_python ${warning_guard}
PRIVATE
"$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/src/include>"
"$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/src/third-party/keccak>"
"$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/src/third-party/murmur3>"
"$<BUILD_INTERFACE:${PROJECT_BINARY_DIR}/include>"
"$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/bindings/python>"
Expand Down
1 change: 1 addition & 0 deletions bindings/python/py_arch.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ void init_arch(PyObject* module)
PyObject* arch_enum = PyDict_New();
PyDict_SetItemString(arch_enum, "X86", PyLong_FromLong((int)Arch::Type::X86));
PyDict_SetItemString(arch_enum, "X64", PyLong_FromLong((int)Arch::Type::X64));
PyDict_SetItemString(arch_enum, "EVM", PyLong_FromLong((int)Arch::Type::EVM));

PyObject* arch_class = create_class(PyUnicode_FromString("ARCH"), PyTuple_New(0), arch_enum);
PyModule_AddObject(module, "ARCH", arch_class);
Expand Down
2 changes: 2 additions & 0 deletions bindings/python/py_constraint.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,8 @@ void init_constraint(PyObject* module)
Constraint_operators.nb_and = Constraint_nb_and;
Constraint_operators.nb_or = Constraint_nb_or;
Constraint_operators.nb_invert = Constraint_nb_not;

register_type(module, (PyTypeObject*)get_Constraint_Type());
}

} // namespace py
Expand Down
81 changes: 64 additions & 17 deletions bindings/python/py_engine.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,40 @@ static void MaatEngine_dealloc(PyObject* self)
Py_TYPE(self)->tp_free((PyObject *)self);
};

static PyObject* MaatEngine_duplicate(PyObject* self, PyObject* args, PyObject* keywords)
{
static char *kwlist[] = {"duplicate", "share", NULL};
PySetObject *py_duplicate = nullptr,
*py_share = nullptr;

std::set<std::string> duplicate, share;

// Process arguments
if( ! PyArg_ParseTupleAndKeywords(
args, keywords, "|O!O!", kwlist,
&PySet_Type, &py_duplicate, &PySet_Type, &py_share
)){
return NULL;
}

if (py_duplicate)
if (not py_to_c_string_set(py_duplicate, duplicate))
return PyErr_Format(PyExc_RuntimeError, "Failed to process 'duplicate' argument");

if (py_share)
if (not py_to_c_string_set(py_share, share))
return PyErr_Format(PyExc_RuntimeError, "Failed to process 'share' argument");

// Create new engine
MaatEngine *new_engine = new MaatEngine(
*as_engine_object(self).engine,
duplicate,
share
);
// The python object will own the 'new_engine' pointer
return PyMaatEngine_FromMaatEngine(new_engine);
};

static PyObject* MaatEngine_run(PyObject* self, PyObject* args){
unsigned int max_instr = 0;
info::Stop res;
Expand Down Expand Up @@ -82,7 +116,7 @@ static PyObject* MaatEngine_restore_snapshot(PyObject* self, PyObject* args, PyO

static PyObject* MaatEngine_load(PyObject* self, PyObject* args, PyObject* keywords){
char * name;
int bin_type;
int bin_type = (int)loader::Format::NONE;
unsigned long long base = 0;
PyObject* py_cmdline_args = nullptr, *arg = nullptr;
PyObject *py_envp = nullptr;
Expand All @@ -99,7 +133,7 @@ static PyObject* MaatEngine_load(PyObject* self, PyObject* args, PyObject* keywo
char* keywd[] = {"", "", "base", "args", "envp", "libdirs", "ignore_libs", "virtual_fs", "load_interp", NULL};

if( !PyArg_ParseTupleAndKeywords(
args, keywords, "si|KOOOOOp", keywd,
args, keywords, "s|iKOOOOOp", keywd,
&name, &bin_type, &base,
&py_cmdline_args, &py_envp,
&py_libs, &py_ignore_libs,
Expand Down Expand Up @@ -141,17 +175,7 @@ static PyObject* MaatEngine_load(PyObject* self, PyObject* args, PyObject* keywo
"Command line argument specified as a 'list' should only contain 'Value' elements"
);
}
const Value& value = *as_value_object(val).value;
if (value.size() != 8)
{
return PyErr_Format(
PyExc_TypeError,
"Abstract value in command line argument %d should have a size of 8 bits (got %d)",
i,
(int)value.size()
);
}
arg_buffer.push_back(value);
arg_buffer.push_back(*as_value_object(val).value);
}
cmdline_args.push_back(loader::CmdlineArg(arg_buffer));
}
Expand Down Expand Up @@ -274,6 +298,17 @@ static PyObject* MaatEngine_load(PyObject* self, PyObject* args, PyObject* keywo
};



static PyObject* MaatEngine_get_uid(PyObject* self, void* closure){
return PyLong_FromLong(as_engine_object(self).engine->uid());
}


static PyGetSetDef MaatEngine_getset[] = {
{"uid", MaatEngine_get_uid, NULL, "Unique ID for this MaatEngine instance", NULL},
{NULL}
};

static PyObject* MaatEngine_get_inst_asm(PyObject* self, PyObject* args){
unsigned long long addr;

Expand Down Expand Up @@ -314,6 +349,7 @@ static PyMethodDef MaatEngine_methods[] = {
{"take_snapshot", (PyCFunction)MaatEngine_take_snapshot, METH_NOARGS, "Take a snapshot of the symbolic engine"},
{"restore_snapshot", (PyCFunction)MaatEngine_restore_snapshot, METH_VARARGS | METH_KEYWORDS, "Restore a snapshot of the symbolic engine"},
{"load", (PyCFunction)MaatEngine_load, METH_VARARGS | METH_KEYWORDS, "Load an executable"},
{"_duplicate", (PyCFunction)MaatEngine_duplicate, METH_VARARGS | METH_KEYWORDS, "Duplicate a symbolic engine"},
{"get_inst_asm", (PyCFunction)MaatEngine_get_inst_asm, METH_VARARGS, "Get assembly code of an instruction"},
{"get_inst_bytes", (PyCFunction)MaatEngine_get_inst_bytes, METH_VARARGS, "Get raw bytes of an instruction"},
{NULL, NULL, 0, NULL}
Expand Down Expand Up @@ -363,7 +399,7 @@ PyTypeObject MaatEngine_Type = {
0, /* tp_iternext */
MaatEngine_methods, /* tp_methods */
MaatEngine_members, /* tp_members */
0, /* tp_getset */
MaatEngine_getset, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
Expand Down Expand Up @@ -411,7 +447,7 @@ void _init_MaatEngine_attributes(MaatEngine_Object* object)
object->mem = PyMemEngine_FromMemEngine(object->engine->mem.get(), true);
object->hooks = PyEventManager_FromEventManager(&(object->engine->hooks), true);
object->info = PyInfo_FromInfoAndArch(&(object->engine->info), true, &(*object->engine->arch));
object->path = PyPath_FromPath(&(object->engine->path), true);
object->path = PyPath_FromPath(object->engine->path.get(), true);
object->env = PyEnv_FromEnvEmulator(object->engine->env.get(), true);
object->settings = PySettings_FromSettings(&(object->engine->settings), true);
object->process = PyProcessInfo_FromProcessInfo(object->engine->process.get(), true);
Expand All @@ -429,14 +465,24 @@ PyObject* maat_MaatEngine(PyObject* self, PyObject* args){
return NULL;
}

// Create object
return PyMaatEngine_FromMaatEngine(
new MaatEngine((Arch::Type)arch, (env::OS)system)
);
}


PyObject* PyMaatEngine_FromMaatEngine(MaatEngine* engine)
{
MaatEngine_Object* object;
// Create object
try
{
PyType_Ready(&MaatEngine_Type);
object = PyObject_New(MaatEngine_Object, &MaatEngine_Type);
if (object != nullptr)
{
object->engine = new MaatEngine((Arch::Type)arch, (env::OS)system);
object->engine = engine;
_init_MaatEngine_attributes(object);
}
}
Expand All @@ -447,7 +493,6 @@ PyObject* maat_MaatEngine(PyObject* self, PyObject* args){
return (PyObject*)object;
}


/* ------------------------------------
* Init function
* ------------------------------------ */
Expand All @@ -470,6 +515,8 @@ void init_engine(PyObject* module)
PyDict_SetItemString(stop_enum, "NONE", PyLong_FromLong((int)info::Stop::NONE));
PyObject* stop_class = create_class(PyUnicode_FromString("STOP"), PyTuple_New(0), stop_enum);
PyModule_AddObject(module, "STOP", stop_class);

register_type(module, (PyTypeObject*)get_Info_Type());
};

} // namespace py
Expand Down
Loading

0 comments on commit 2876019

Please sign in to comment.