Added bpftrace backend with tests, Added docker

Feature/bpftrace backend

README.md

@@ -1,36 +1,41 @@
 # secimport
 
+
 <p align="center">
  <a href="https://github.com/avilum/secimport"><img src="https://user-images.githubusercontent.com/19243302/177835749-6aec7200-718e-431a-9ab5-c83c6f68565e.png" alt="secimport"></a>
 </p>
 <p align="center">
-Secure import for python modules using dtrace under the hood.<br>
+An easy way to constrain python modules in your code using backends like bpftrace (eBPF) and dtrace.<br>
 <a href="https://infosecwriteups.com/sandboxing-python-modules-in-your-code-1e590d71fc26?source=friends_link&sk=5e9a2fa4d4921af0ec94f175f7ee49f9">Medium Article</a>
 </p>
 
 `secimport` can be used to:
 - Confine/Restrict specific python modules inside your production environment.
-  - Open Source, 3rd party from unstrusted sources.
-  - Audit the flow of your python application at user-space/os/kernel level.
-- Run an entire python application under unified configuration
-  - Like `seccomp` but not limited to Linux kernels. Cross platform.
-
+  - Restrict 3rd party or open source modules in your code.
+- Audit the flow of your python application at user-space/os/kernel level.
+- Kill the process upon violoation of a profile.
 
 # Quick Start
 `secimport` can be used out of the box in the following ways:
-1. Inside your code using `module = secimport.secure_import('module_name', ...)`.
-    - Replacing the regular `import` statement with `secure_import`
-    - Only modules that were imported with `secure_import` will be traced.
-2. As a sandbox, by specifying the modules and their policies.
-    - Use this repository to:
-        - Generate a YAML policy from your code
-        - Compile that YAML to dscript.
-    - Use `dtrace` command to run your main python application, with your tailor-made sandbox.
-        - No need for `secure_import`, you can keep using regular `import`s
+1. Modify your imports
+    - Inside your code using `module = secimport.secure_import('module_name', ...)`.
+      - Replacing the regular `import` statement with `secure_import`
+      - Only modules that were imported with `secure_import` will be traced.
+2. As a sandbox that runs your main code.
+      1. Generate a YAML policy from your code, by specifying the modules and the policy you want for each module you use.
+      2. Convert that YAML policy to dscript/bpftrace sandbox code.
+      3. Use `dtrace` or `ebpf` to run your main python application, with your tailor-made sandbox.
+          - No need for `secure_import`, you can keep using regular `import`s and not change your code at all.
 
 For the full list of examples, see <a href="docs/EXAMPLES.md">EXAMPLES.md</a>.
 
-# Pickle Example
+# Docker
+A working environment is not easy to create.<br>
+The easiest way to evaluate secimport, is by using our <a href="docker/README.md">Docker for MacOS and Linux</a> that includes secimport, bpftrace backend and eBPF libraries.<br>
+dtrace backend is not available in docker, and can be tried directly on the compatible hosts (MacOS, Solaris, Unix, some Linux distributions)
+
+# Use Cases
+
 ### How pickle can be exploited in your 3rd party packages:
 ```python
 >>> import pickle
@@ -72,7 +77,7 @@ $ less /tmp/.secimport/sandbox_pickle.log
 :
 ```
 
-## YAML Template Example
+## YAML Policy Example
 For a full tutorial, see <a href="docs/YAML_PROFILES.md">YAML Profiles Usage</a>
 ```shell
 # An example yaml template for a sandbox.
@@ -102,7 +107,7 @@ modules:
 
 ```
 
-## Python Processing Example
+## Blocking New Processes Example
 ```python
 Python 3.10.0 (default, May  2 2022, 21:43:20) [Clang 13.0.0 (clang-1300.0.27.3)] on darwin
 Type "help", "copyright", "credits" or "license" for more information.
@@ -129,8 +134,12 @@ Type "help", "copyright", "credits" or "license" for more information.
 # Damn! That's cool.
 ```
 
-- The dtrace profile for the module is saved under:
-  -  `/tmp/.secimport/sandbox_subprocess.d`:
+When using secure_import, the following files are created:
+- The dtrace/bpftrace sandbox code for the module is saved under:
+  -  `/tmp/.secimport/sandbox_subprocess.d`
+      - when using dtrace
+  -  `/tmp/.secimport/sandbox_subprocess.bt`:
+      - when using bpftrace
 - The log file for this module is under
   -  `/tmp/.secimport/sandbox_subprocess.log`:
         ```shell
@@ -216,8 +225,9 @@ Not related for python, but for the sake of explanation (Equivilant Demo soon).
 - <a href="docs/TRACING_PROCESSES.md">Tracing Guides</a>
 - <a href="docs/FAQ.md">F.A.Q</a>
 - <a href="docs/INSTALL.md">Installation</a>
-- <a href="docs/MAC_OS_USERS.md">Mac OS Users</a> - Disabling SIP for dtrace
+- <a href="docs/MAC_OS_USERS.md">Mac OS Users</a> - Disabling SIP (System Intergity Protection)
 - https://www.brendangregg.com/DTrace/DTrace-cheatsheet.pdf
+- https://www.brendangregg.com/blog/2018-10-08/dtrace-for-linux-2018.html
 <br><br>
 
 ## TODO:
@@ -226,6 +236,14 @@ Not related for python, but for the sake of explanation (Equivilant Demo soon).
   - ✔️ Use secimport to compile that yml
   - ✔️ Create a single dcript policy
   - ✔️ Run an application with that policy using dtrace, without using `secure_import`
-- Node support (dtrace hooks)
-- Go support (dtrace hooks)
-- Use current_module_str together with thread ID
+- ✔️ <b>Add eBPF basic support using bpftrace</b>
+  - ✔️ bpftrace backend tests
+- <b>Extandible Language Template</b>
+  - Increase extandability for new languages tracing with bpftace/dtrace.
+    - Adding a new integration will be easy, in a single directory, using templates for filters, actions, etc.
+- <b>Node support</b> (bpftrace/dtrace hooks)
+  - Implement a template for Node's call stack and event loop
+- <b>Go support</b> (bpftrace/dtrace hooks)
+   - Implement a template for golang's call stack
+- Multi Process support: Use current_module_str together with thread ID to distinguish between events in different processes
+- Update all linux syscalls in the templates (filesystem, networking, processing) to improve the sandbox blocking of unknowns.

docker/README.md

@@ -0,0 +1,30 @@
+# Try secimport with bpftrace
+
+## How to Use
+
+1. Install Docker: https://docs.docker.com/get-docker
+2. ./build.sh
+    - will build a docker image with
+        - python with dtrace static USDT instrumentations
+        - bpftrace
+        - secimport code
+    - ~1GB in size
+3. ./run.sh
+    - Runs temporary example sandbox using bpftrace
+    - Then, it will execute os.system('ps').
+      -  the process should be killed.
+    - Once the process is killed, it prints the logs of the sandbox.
+
+
+## FAQ
+
+### How it runs on macOS?
+- The Docker for mac runs Linux on a hypervisor called hyperkit, and docker runs inside it, so you can use Linux features.
+
+### Can we trace a macOS host with this docker?
+- Not at the moment. The bpftrace runs inside a Linux VM. 
+- For macOS, there is dtrace.
+
+=====================
+
+Based on the great example repo: https://github.com/mmisono/try-bpftrace-in-mac

docker/build.sh

@@ -0,0 +1,26 @@
+#!/bin/bash
+
+if [[ "$PWD" =~ docker$ ]]
+then
+    echo "Building secimport docker container...";
+else
+    echo "Please run this script from the secimport/docker directory.";
+    exit 1;
+fi
+
+# linukit kernel version
+KERNEL_VERSION=`docker run --rm -it alpine uname -r | cut -d'-' -f1`
+BPFTRACE_VERSION=${BPFTRACE_VERSION:-v0.16.0}
+PYTHON_VERSION=${PYTHON_VERSION:-"3.10.0"}
+
+pushd docker
+
+docker build \
+    --build-arg KERNEL_VERSION=${KERNEL_VERSION} \
+    --build-arg BPFTRACE_VERSION=${BPFTRACE_VERSION} \
+    --build-arg PYTHON_VERSION=${PYTHON_VERSION} \
+    -t secimport:${KERNEL_VERSION} .
+
+popd
+
+echo "You can now use the ./run.sh script to try secimport."

docker/docker/Dockerfile

@@ -0,0 +1,38 @@
+ARG KERNEL_VERSION
+
+FROM linuxkit/kernel:${KERNEL_VERSION} as ksrc
+FROM ubuntu:20.04 AS build
+
+ARG BPFTRACE_VERSION
+ARG PYTHON_VERSION
+
+WORKDIR /kernel
+COPY --from=ksrc /kernel-dev.tar .
+RUN tar xf kernel-dev.tar
+
+WORKDIR /workspace
+ARG DEBIAN_FRONTEND=noninteractive
+
+# TODO: add openssl (longer build time, but pip will work for our interpreter)
+RUN echo "Installing prerequisites" && \
+    apt-get update && apt-get install sudo build-essential libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev curl wget auditd vim tmux git binutils unzip gcc systemtap-sdt-dev cmake zlib1g-dev -y
+RUN echo "Installing python with dtrace" && \
+    curl -o Python-${PYTHON_VERSION}.tgz https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz && tar -xzf Python-${PYTHON_VERSION}.tgz && \
+    cd Python-${PYTHON_VERSION} && ./configure --with-dtrace --prefix=/usr/local/openssl --prefix=$(pwd) --with-ensurepip=install && make && make install
+RUN echo "Installing bpftrace" && \ 
+    wget https://github.com/iovisor/bpftrace/releases/download/${BPFTRACE_VERSION}/bpftrace && \
+    chmod +x bpftrace && \
+    mv bpftrace /bin && \
+    wget https://github.com/iovisor/bpftrace/archive/${BPFTRACE_VERSION}.zip && \
+    unzip ${BPFTRACE_VERSION}.zip && \
+    cp -r bpftrace*/tools /workspace/bpftrace/ && \
+    echo "Done building bpftrace" && \
+    mv /kernel/usr/src/linux-headers* /kernel/usr/src/linux-headers
+
+ENV BPFTRACE_KERNEL_SOURCE=/kernel/usr/src/linux-headers
+COPY setup.sh .
+COPY sandbox.bt .
+COPY run_sandbox.sh .
+RUN chmod 755 sandbox.bt run_sandbox.sh
+
+ENTRYPOINT ["/bin/sh", "/workspace/setup.sh"]

docker/docker/run_sandbox.sh

@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+
+# "--unsafe" is required to run system command for remediation.
+#            If process termination on violoation is not needed,
+#            You can remove this argument.
+
+echo "Starting secimport sandbox with bpftrace backend, the sandbox should kill the python process..."
+bpftrace -c "/workspace/Python-3.10.0/python -c __import__('os').system('ps')" -o sandbox.log sandbox.bt --unsafe || echo "The process was killed, as expected."
+echo "The sandbox bpftrace code is at sandbox.bt"
+echo "The sandbox log is at sandbox.log"
+# tail -n 20 sandbox.log
+# less +G sandbox.log

docker/docker/sandbox.bt

@@ -0,0 +1,32 @@
+#!/usr/bin/env bpftrace
+
+BEGIN {
+    printf("STARTED\n")
+}
+
+
+usdt:/workspace/Python-3.10.0/python:function__entry {
+        @["depth"]++;
+        @entrypoints[str(arg0)] = @["depth"];
+        @globals["previous_module"] = @globals["current_module"];
+        @globals["current_module"] = str(arg0);
+        printf("%s, %s, depth=%d\n", str(arg0), str(arg1), @["depth"]) ;
+}
+
+usdt:/workspace/Python-3.10.0/python:function__return {
+        @["depth"]--;
+}
+
+tracepoint:raw_syscalls:sys_enter /comm == "python"/ {
+        if(args->id == 59){
+            printf("KILLING PROCESS %s - EXECUTED execve;\n", str(pid));
+                system("pkill -9 args"); // optional
+                printf("Killed process %s", str(pid));
+                exit(); // optional
+        }
+        printf("%s SYSCALL %ld depth=%d previous=%s current=%s \n", probe, args->id, @["depth"], @globals["previous_module"], @globals["current_module"] );
+}
+
+END {
+    clear(@entrypoints);
+}

docker/docker/setup.sh

@@ -0,0 +1,7 @@
+#!/bin/sh
+mount -t debugfs none /sys/kernel/debug/
+sysctl -w kernel.kptr_restrict=0 >/dev/null 2>&1
+sysctl -w kernel.perf_event_paranoid=2 >/dev/null 2>&1
+cd /workspace/
+/bin/bash
+# /bin/bash ./run_sandbox.sh

docker/run.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+
+if [[ "$PWD" =~ docker$ ]]
+then
+    echo "Running secimport docker container...";
+else
+    echo "Please run this script from the secimport/docker directory.";
+    exit 1;
+fi
+
+KERNEL_VERSION=`docker run --rm -it alpine uname -r | cut -d'-' -f1`
+
+cd .. # back to repo root dir
+docker run --rm --name=secimport --privileged -v "$(pwd)/src/secimport":"/workspace/secimport/" -it secimport:${KERNEL_VERSION}

docs/TRACING_PROCESSES.md

@@ -10,12 +10,14 @@ There are several ways to create a secimport profile for your modules.
     -  `sudo dtrace -s src/secimport/templates/default.allowlist.template.d -c "python -m http.server"`
     - CTRL+C
     - Create a secure import based on that log.
-  - Using simple `dtrace`
+  - Using `bpftrace`
+    - See https://github.com/iovisor/bpftrace/tree/master/tools
+  - Using `dtrace`
     - Tracing the syscalls of a process with pid `12345`
       - `dtrace -n 'syscall::: /pid == ($1)/ {@[pid,execname,probefunc]=count()}' 12345`
     - Tracing the syscalls of a docker container with pid `12345`
       - `dtrace -n 'syscall::: /progenyof($1)/ {@[pid,execname,probefunc]=count()}' 12345`
-  - Using `strace`
+  - Using an `strace` script I contributed to FireJail
     -  A script to list all your application's syscalls using `strace`.<br> I contributed it to `firejail` a few years ago:
       - https://github.com/netblue30/firejail/blob/master/contrib/syscalls.sh
       - ```

examples/run_bpftrace_example.sh

@@ -0,0 +1,15 @@
+#!/usr/bin/env bash
+
+# "--unsafe" is required to run system command for remediation.
+#            If process termination on violoation is not needed,
+#            You can remove this argument.
+
+echo "Starting secimport sandbox with python shell..."
+bpftrace -c "/workspace/Python-3.10.0/python -c __import__('os').system('ps')" -o sandbox.log sandbox.bt --unsafe
+
+# The process is killed becused we ran os.system inside our sandbox.
+# Watch the logs:
+less +G sandbox.log
+# OR: 
+# tail -n 20 sandbox.log
+echo "The sandbox log is at ./sandbox.log"

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "secimport"
-version = "0.4.2"
+version = "0.5.0"
 description = "A sandbox/supervisor for python modules."
 authors = ["Avi Lumelsky"]
 license = "MIT"