OSWatcher

Tracking the evolution of operating systems over time

Table of Contents

• Overview
• Requirements
• Install
• Usage
• Troubleshooting
• Maintainers
• Contributing
• License

Overview

OSWatcher is an ambitious project that aims to track the evolution of operating systems by making diffs between recognizable characteristics.

The core of OSWatcher is to build a reference database about every OS releases, that is to be populated by an extractor in charge of capturing the various information that can be extracted from an installed operating system, both online and offline, in a reproducible way.

Offline:

• filesystem hierarchy
• setuid binaries
• executable properties
• library graph dependencies
• statistics around perl/sh/python scripts
• syscall tables
• kernel configuration
• cronjobs
• /etc configuration

Online:

• IDLE memory consumption
• default processes running
• mapped libraries
• listening ports and associated services
• DNS requests sent
• unix sockets
• dbus traffic
• iptables rules
• loaded drivers

Requirements

• python >= 3.7
• virtualenv
• libguestfs
• Docker (optional)

Install

1. Clone repo and submodules

git clone https://github.com/Wenzel/oswatcher.git
cd oswatcher
git submodule update --init

2. Install system dependencies

On Ubuntu 18.04

sudo apt-get install virtualenv python3-virtualenv libguestfs0 libguestfs-dev python3-guestfs python3-dev pkg-config libvirt-dev

3. Create a Python3 virtualenv

virtualenv --system-site-packages -p python3 venv
source venv/bin/activate
pip install .

Note: We have to use --system-site-packages because libguestfs is not available on pip.

Hooks configuration

Open hooks.json and edit /path/to/repo to an empty git repository (outside of oswatcher's git repo).

        {
            "name": "hooks.filesystem.GitFilesystemHook",
            "configuration":
            {
                "repo": "/home/user/test/git_fs"
            }
        }

VM setup

OSWatcher works on VMs stored in libvirt, either via qemu:///session or qemu:///system.

Note: qemu:///session is recommended as it requires less permission and should work without further configuration.

Usage: Filesystem capture in Git

Start the capture tool on a VM and specify the hooks configuration to start capturing the VM's filesystem in the previously configured git repository.

(venv) $ python -m oswatcher <vm_name> hooks.json

Example:

Advanced Usage

Neo4j

Some of OSWatcher's plugins are using neo4j as a database.

• system.OperatingSystemHook
• filesystem.Neo4jFilesystemHook
• security.SecurityHook

Follow the instructions in the db directory to run a Neo4j inside a docker container.

Access Neo4j web interface at http://localhost:7474

Note: the system.OperatingSystemHook requires to specify a release_date in JSON format, so that the capture tool can insert this information in the database as well.

-> In the VM XML <description> field, add the following content:

{"release_date": "2012-04-01"}

You can use edit virsh edit <domain> or virt-manager tool which should be easier.

Web frontend (WIP)

A web frontend developed on top of VueJS is in development in the web folder.

Filesystem view

Process list view

Syscall table view

Troubleshooting

libguestfs

If libguestfs fails to initialize, you can use the libguestfs-test-tool to quickly understand the root cause of the failure.

Maintainers

@Wenzel

Contributing

PRs accepted.

Small note: If editing the Readme, please conform to the standard-readme specification.

License

GNU General Public License v3.0