Skip to content

Home

Jump to bottom
Gaulthier Gain edited this page Dec 11, 2020 · 3 revisions

Unikraft tools

Welcome to the Unikraft tools wiki!

The Unikraft tools are a set of tools to automatically build images of operating systems targeting applications. The toolchain will include the following tools:

  1. Decomposition tool to assist developers in breaking existing monolithic software into smaller components.
  2. Dependency analysis tool to analyse existing, unmodified applications to determine which set of libraries and OS primitives are absolutely necessary for correct execution.
  3. Automatic build tool to match the requirements derived by the dependency analysis tools to the available libraries constructed by the OS decomposition tools. This one is composed of two components: a static analysis and a dynamic analysis.
  4. Verification tool to ensure that the functionality of the resulting, specialized OS+application matches that of the application running on a standard OS. The tool will also take care of ensuring software quality.
  5. Performance optimization tool to analyse the running specialized OS+application and to use this information as input to the automatic build tools so that they can generate even more optimized images.

For now only the Dependency analysis tool and the Automatic build tool are available.

1. Installation

The toolchain is written in golang/go. Please install go and read the golang wiki to set up your environment.

Once go is installed, please run the Makefile (via make deps) to get dependencies and build the toolchain. You can also download yourself the dependencies by running the following commands:

go get -v github.com/fatih/color
go get -v github.com/akamensky/argparse
go get -v github.com/awalterschulze/gographviz
go get -v github.com/sergi/go-diff/...
go get -v github.com/AlecAivazis/survey

Once you have the dependencies, run the make command to build the toolchain.

2. Architecture

The project contains the high-level architecture:

Package Description
configFiles Contains sample configuration of specific application.
testFiles Contains sample test of a specific application for the dynamic analysis. The format used is a json which contains three different fields typeTest, timeMsCommand and listCommands. The first field represents the type of test which can be: exec (test via a classic execution like a script), stdin (test via stdin) or telnet (test via telnet). The timeMsCommand field defines the time of execution in ms. Finally, the listCommands field contains the different commands which will be executed during the dynamic analysis.
srcs Contains the go source of the toolchain.

3. Getting started

As stated previously, the toolchain contains 5 different tools. You can either completely run the toolchain or specify which tool to use. The default behaviour executes all the tools (except the crawler one). To execute a specific tool, use one of the following arguments:

  • --dep: runs only the dependency analysis tool.
  • --build: runs only the automatic build tool.
  • --verif: runs only the dependency analysis tool (prototype).
  • --perf: runs only the dependency analysis tool (Not implemented).

Outside the toolchain:

  • --crawler: run only the crawler tool. It creates a graph which represents dependencies of dependencies.

3.1 Dependency analysis Tool

The objective of the dependency analysis tool is to gather, for the target applications, which software in the operating system they actually use. Such software will include shared libraries, other applications, core kernel components, kernel modules, and so forth.

The dependency analysis tool relied on two sub-modules: static and dynamic analysers and requires a binary application as input. For the dynamic analysis, different tests should be used to explore execution paths of a particular application. In order to do it, the dependency tool provides two different procedures: either passing a test file as argument which contains internal commands to test the current application (e.g., SQL queries for a database, ...) or either to perform manually tests by specifying a duration to test.

The dependency analysis tool uses the following arguments:

  • -p or --program: the name (or path) of the application to analyse. This argument is required.

  • --saveOutput: save results as txt file(s) and graph(s) as PNG file(s). By default, this argument is set to false

  • --fullDeps: computes dependencies of dependency. By default, this argument is set to false

  • -o or --options: extra options/parameters needed by a given application (e.g, configuration file, pid file, ...). [dynamic analysis]

  • -w or --waitTime: the time (in secondes) for external tests. By default, this argument is set to 60 secondes. [dynamic analysis]

  • -t or --testFile: the path to the test file (.json). [dynamic analysis]

  • -c or --configFile: the path to the config file (.txt). [dynamic analysis]

Note1: Several test and config files can be respectively found in the tests and config folders within the dependency analysis folder.

Note2: To execute tests on stdin, add the [STDIN] flag within your test files (view /dependtool/tests/test_sqlite.txt).

3.2 Automatic Build Tool

Given a list of dependencies obtained with the dependency analysis tool, the tool will automatically build an OS image (kernel and filesystem) that can run, out of the box, the target application(s). The tool is also divided into two components: a controller written in golang and the Unikraft build system.

Please read the Unikraft's documentation to get further information: http://www.unikraft.org

Important: The tool checks if you already set the unikraft workspace. If no, it will do it and set the unikraft workspace. If you already have unikraft, you must have the following structure to use the automatic build tool:

my-workspace/
├── apps/
├── libs/
└── unikraft/

The automatic build tool uses the following arguments:

  • -p or --program: the name (or path) of the application to analyse. This argument is required.
  • -u or --unikraft: the path of the root folder that contains Unikraft (see below). This argument is required.
  • -s or --sources: the path of the application to build as unikernel. This argument is required.
  • -m or --makefile: specify a makefile configuration to add to the makefile.uk.

3.3 Verification Tool

The verification tool ensures that the newly built app (unikernel) is equivalent to the initial one. For now, the tool uses fuzz-testing (from dependency analysis) and performs a diff on their output.

The verification tool uses the following arguments:

  • -p or --program: the name (or path) of the application to analyse. This argument is required.
  • -u or --unikraft: the path of the root folder that contains Unikraft (see below). This argument is required.
  • -t or --testFile: the path to the test file (.txt).
  • -c or --configFile: the path to the config file (.txt).

Toolchain Examples:

./tools --dep -p /home/unikraft/mini_httpd/mini_httpd -w 10
./tools --dep -p /home/unikraft/mysql/mysqld -w 120 -o "--pid-file=/run/mysqld/mysqld.pid"
./tools --dep -p /usr/sbin/nginx --fullDeps --saveOutput -t /home/unikraft/tests/test_http.json
./tools --dep -p redis-server -t /home/unikraft/tests/test_redis.json

./tools -p /usr/sbin/nginx -w 10 -u /home/unikraft/unikraft/ -s /home/unikraft/mini_httpd/mini_httpd_src/
./tools -p /home/unikraft/temp/sqlite_src/sqlite3 -t /home/unikraft/tests/test_sqlite.json -s /home/unikraft/temp/sqlite_src/ -m /home/unikraft/temp/sqlite_src/makefile

./tools --verif -p sqlite3 -t /home/unikraft/tests/test_sqlite.json -u /home/unikraft/

Unikraft Crawler tool

The Unikraft Crawler allows to create a graph of dependencies of unikraft micro-libraries. It works as follow:

  1. It needs as input either a list of libraries folder (specified within a text file - see below) or the path to a particular folder to explore.
  2. Then it crawls each library's Config.uk (including the multiple ones in the main Unikraft repo).
  3. Finally, it generates a dependencies graph whenever a Config.uk has a select keyword.

To crawler needs the following arguments:

  • --output or -o: the path to the output file (.dot).
  • --libs or -l: the path to a text file that contains the list of libraries to crawl (as absolute path).
  • --repository or -r: the path to the repository to crawl (as absolute path).
  • --full or -f: specifies if other Config fields must be considered. By default, this argument is set to false

Crawler Tool Examples:

./tools --crawler -l /home/unikraft/temp/libs.txt -o /home/unikraft/temp
./tools --crawler -r /home/unikraft/temp/unikraft -o /home/unikraft/temp
Clone this wiki locally