-
Notifications
You must be signed in to change notification settings - Fork 9
Setup
This guide will get you up and running with Paper Machete (PM) on a fresh Ubuntu Linux 18.04 installation. We use Ubuntu as a standard to illustrate package management commands in this guide. Please adapt the commands to reflect your specific environment as needed. You can also use Docker.
(See Ubuntu commands below)
- Binary Ninja (commercial license required for Python API; otherwise, use pm_analysis.py as a plugin)
- Grakn Core
- python2.7 (required for updating old versions of Binary Ninja)
- python3 (required for Binary Ninja API and Grakn driver)
- Java JRE 8 (Grakn is highly dependent on Java JRE 8, other JRE versions will not work)
We have deployed PM on everything from a Linux VM with 2GB RAM and 1 core, all the way up to a bare metal 2x Intel Xeon E5-2697v4 with 1TB RAM and 72 total cores. Our goal is to keep PM usable for individual researchers on a modern laptop, all the way up to organizations that have large compute resources. You will earn reduced analysis and migration time with increased computing power. Grakn can run on default Java settings (heap of 768MB, 1GB machine) if the graph is small enough. Recommended production settings are at least 4GB machine with 3GB heap. (To change the heap, see "Performance Tweaks" below.) If you do not have at least 4GB of RAM to devote to PM, we highly recommend you do not analyze entire binaries! Instead, we suggest you analyze a few functions of interest within the target binary. Both pm_analysis.py and paper_machete.py allow you to specify a list of functions to analyze. This will make your graph much smaller and require less overhead from Grakn.
Install python2.7, Java JRE 8, and any Python requirements for PaperMachete, including the Grakn Python driver:
$ sudo apt update
$ sudo apt install openjdk-8-jre-headless python2.7 python3-pip git
# PM can be installed anywhere,
# but you must run paper_machete.py from the installation directory
$ sudo mkdir /opt/papermachete
$ sudo chown $(whoami) /opt/papermachete
$ git clone https://github.com/cetfor/PaperMachete /opt/papermachete
$ cd /opt/papermachete
$ pip3 install -r requirements.txt
Install Binary Ninja:
# Binary Ninja can be installed anywhere,
# but you must have binaryninja in your python2.7 path
$ sudo unzip /path/to/BinaryNinja.zip -d /opt/
$ /opt/binaryninja/binaryninja
# Follow the prompt to provide your license
Install Binary Ninja Python API:
$ mkdir -p ~/.local/lib/python2.7/site-packages/
$ echo "/opt/binaryninja/python" > ~/.local/lib/python2.7/site-packages/binaryninja.pth
Install Grakn Core:
$ wget https://github.com/graknlabs/grakn/releases/download/v1.4.2/grakn-core-1.4.2.zip -O /tmp/grakn.zip
# Grakn can be installed anywhere,
# but you must have grakn and graql in your PATH
$ sudo unzip /tmp/grakn.zip -d /opt/ && rm /tmp/grakn.zip
$ sudo chown -R $(whoami) /opt/grakn-core-1.4.2/
$ sudo ln -s /opt/grakn-core-1.4.2/grakn /usr/local/bin/
$ sudo ln -s /opt/grakn-core-1.4.2/graql /usr/local/bin/
Sanity Check:
$ python2.7
Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import binaryninja
>>> # look no errors!
>>> exit()
$ python3
Python 3.6.6 (default, Sep 12 2018, 18:26:19)
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import grakn
>>> # look no errors!
>>> exit()
$ grakn server start
... <truncated>
Starting Storage.....SUCCESS
Starting Engine.....SUCCESS
Place some binaries you want to analyze in /opt/papermachete/analysis/ and start PM:
$ cd /opt/papermachete/
$ python2.7 paper_machete.py
____ __ __ _ _
| _ \ __ _ _ __ ___ _ __ | \/ | __ _ ___| |__ ___| |_ ___ ________
| |_) / _` | '_ \/ _ \ '__| | |\/| |/ _` |/ __| '_ \ / _ \ __/ _ \ /_______/
| __/ (_| | |_)| __/ | | | | | (_| | (__| | | | __/ || __/ \_______\
|_| \__,_| .__/\___|_| |_| |_|\__,_|\___|_| |_|\___|\__\___| /_______/
|_| @==|;;;;;;>
================================================================================
[1] Analyze a binary file
[2] Migrate a JSON file into Grakn
[3] Run all CWE queries
[4] Clean and restart Grakn
[5] Quit
Follow the prompts.
You can access the Grakn knowledge graph by browsing to localhost:4567.
We have created a Dockerfile for your convenience.
To build the Docker image, make sure to supply your own BinaryNinja.zip (Linux sources) and commercial license.txt (see below).
The built Docker image contains all of the prerequisites, but not Paper Machete itself, which is useful for development. The container expects the Paper Machete repo to be mounted as a shared volume at /opt/papermachete, so you can modify queries, contents of analysis/, or even PM itself without rebuilding the Docker image.
$ git clone https://github.com/cetfor/PaperMachete
$ cd PaperMachete
$ cp /path/to/BinaryNinja.zip binaryninja/
$ cp /path/to/license.txt binaryninja/
$ docker build . -t papermachete
$ docker run -it -v "$(pwd)":/opt/papermachete -p 4567:4567 papermachete
You should see the Grakn banner followed by the PM prompts.
You can access the Grakn knowledge graph by browsing to localhost:4567 on the host.
You can change the Java heap limit globally by running export _JAVA_OPTIONS="-Xmx3G". If you are using a Docker container, make sure to increase the container memory.
You could instead set Java's heap limit by building Grakn from source and modifying the grakn-dist/src/grakn and/or grakn-dist/src/graql bash/batch script(s):
- java -cp "${CLASSPATH}" ... <truncated> ...
+ java -Xmx3G -cp "${CLASSPATH}" ... <truncated> ...