Skip to content

Commit

Permalink
Merge branch 'feature/python_3_11_8'
Browse files Browse the repository at this point in the history
  • Loading branch information
Avi Lumelsky committed May 19, 2024
2 parents 589db13 + 3bff442 commit 92f9b18
Show file tree
Hide file tree
Showing 3 changed files with 74 additions and 108 deletions.
178 changes: 71 additions & 107 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,98 +1,67 @@
# secimport
[![Upload Python Package](https://github.com/avilum/secimport/actions/workflows/python-publish.yml/badge.svg)](https://github.com/avilum/secimport/actions/workflows/python-publish.yml)
![](https://img.shields.io/badge/Test_Coverage-90%-blue)

A Tailor-Made Sandbox for Your Python Applications.<br>
secimport is eBPF-based sandbox toolkit for Python, that enforces specific syscalls per module in yoyr code.<br>
1. It traces your python code
2. It creates a security profile for your application<br>
3. Use the profile by applying it to running processes, or to run a new python process with supervision.

<b>secimport for python is like seccomp-bpf for linux but per module in your code.</b>

<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [secimport](#secimport)
- [The Tailor-Made Sandbox for Your Application](#the-tailor-made-sandbox-for-your-application)
- [The problem](#the-problem)
- [The solution](#the-solution)
- [Installation](#installation)
- [With Docker](#with-docker)
- [Background](#background)
- [Technical Blogs](#technical-blogs)
- [The problem](#the-problem)
- [How it works](#how-it-works)
- [Who is it for?](#who-is-it-for)
- [Getting Started](#getting-started)
- [Example Sandboxes](#example-sandboxes)
- [Using Docker](#using-docker)
- [Without Docker](#without-docker)
- [Usage](#usage)
- [Stop on violation](#stop-on-violation)
- [Kill on violation](#kill-on-violation)
- [Dynamic profiling - trace, build sandbox, run.](#dynamic-profiling---trace-build-sandbox-run)
- [nsjail support (seccomp)](#nsjail-support-seccomp)
- [Command Line Usage](#command-line-usage)
- [Stop on violation](#stop-on-violation)
- [Kill on violation](#kill-on-violation)
- [Quickstart: trace, build, run.](#quickstart-trace-build-run)
- [Advanced](#advanced)
- [Python API](#python-api)
- [Docker](#docker)
- [Examples](#examples)
- [Contributing](#contributing)
- [Roadmap](#roadmap)
- [seccomp-bpf support using nsjail](#seccomp-bpf-support-using-nsjail)
- [Changelog](#changelog)
- [Contributing](#contributing)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

# secimport
[![Upload Python Package](https://github.com/avilum/secimport/actions/workflows/python-publish.yml/badge.svg)](https://github.com/avilum/secimport/actions/workflows/python-publish.yml)
![](https://img.shields.io/badge/Test_Coverage-90%-blue)
# Background
## Technical Blogs
- <a href="https://infosecwriteups.com/sandboxing-python-modules-in-your-code-1e590d71fc26?source=friends_link&sk=5e9a2fa4d4921af0ec94f175f7ee49f9">secimport + Dtrace</a>
- <a href="https://infosecwriteups.com/securing-pytorch-models-with-ebpf-7f75732b842d?source=friends_link&sk=14d8db403aaf66724a8a69b4dea24e12">secimprt + eBPF + PyTorch</a>
- <a href="https://avi-lumelsky.medium.com/secure-fastapi-with-ebpf-724d4aef8d9e?source=friends_link&sk=b01a6b97ef09003b53cd52c479017b03">secimport + eBPF + FastAPI </a>


## The Tailor-Made Sandbox for Your Application
secimport is production-oriented sandbox toolkit.<br>
It traces your code, and runs an executable that allows only the same syscalls, per module.

Technical Blogs (Free Reading):
1. <a href="https://infosecwriteups.com/sandboxing-python-modules-in-your-code-1e590d71fc26?source=friends_link&sk=5e9a2fa4d4921af0ec94f175f7ee49f9">secimport + Dtrace</a>
2. <a href="https://infosecwriteups.com/securing-pytorch-models-with-ebpf-7f75732b842d?source=friends_link&sk=14d8db403aaf66724a8a69b4dea24e12">secimprt + eBPF + PyTorch</a>
3. <a href="https://avi-lumelsky.medium.com/secure-fastapi-with-ebpf-724d4aef8d9e?source=friends_link&sk=b01a6b97ef09003b53cd52c479017b03">secimport + eBPF + FastAPI </a>

### The problem
## The problem
Traditional tools like seccomp or AppArmor enforce syscalls for the entire process.<br>Something like `allowed_syscalls=["read","openat","bind","write"]`, which is great, but not enough for python's attack surface.<br>

### The solution
`secimport` is able to trace which syscalls each module in your code uses (by package name).<br>
After tracing, secimport creates a JSON/YAML policy for your code. It compiles it into a high-performance eBPF instrumentation script (smaller then 512 bytes overall), that looks like <a>this</a>.
```
modules:
requests:
destructive: true # when true, secimport will kill on vilation instead of logging.
syscall_allowlist:
- fchmod
- getentropy
- getpgrp
- getrlimit
...
```
You can also use JSON instaead of YAML, and even confine builtin python modules.<br>
An example policy that uses logging, multiprocessing, os and filesystem will look approximately like this:
```
...
"/workspace/Python-3.11.8/Lib/logging/__init__.py": [
" clock_gettime",
" getpid",
" write"
],
"/workspace/Python-3.11.8/Lib/multiprocessing/process.py": [
" getcwd",
" getpid",
" getrandom"
],
"/workspace/Python-3.11.8/Lib/multiprocessing/util.py": [
" prlimit64"
],
"/workspace/Python-3.11.8/Lib/os.py": [
" read"
],
"/workspace/Python-3.11.8/Lib/platform.py": [
" uname"
],
"/workspace/Python-3.11.8/Lib/posixpath.py": [
" close",
" fstat",
" getcwd",
" getdents64",
" openat"
],
"/workspace/Python-3.11.8/Lib/random.py": [
" getrandom"
],
...
```
It was created by tracing the code. secimport automatically finds these per-module syscalls for you.

Finally, you convert this policy into an sandbox (eBPF instrumentation script) to run the python process in production. Running the sandbox will enfore python to obey any given policy.

## How it works
1. `secimport` is able to trace which syscalls each module in your code uses (by package name).<br>
2. After tracing, secimport creates a JSON/YAML policy for your code. It compiles it into a high-performance eBPF instrumentation script (smaller then 512 bytes overall), that looks like <a>this</a>.
```
modules:
requests:
destructive: true # when true, secimport will kill on vilation instead of logging.
syscall_allowlist:
- fchmod
- getentropy
- getpgrp
- getrlimit
...
```
You can also use JSON instaead of YAML, and even confine builtin python modules.<br>
3. Finally, you convert this policy into an sandbox (eBPF instrumentation script) to run the python process in production. Running the sandbox will enfore python to obey any given policy.
## Who is it for?
`secimport` is great for...
- Preventing Code Execution: reduce the risk of supply chain attacks.
- Trace the syscalls flow of your application at the user-space/os/kernel level and per module.
Expand All @@ -108,14 +77,14 @@ Finally, you convert this policy into an sandbox (eBPF instrumentation script) t
- Has negligible performance impact and is production-ready thanks to eBPF. Check out the [Performance](https://github.com/avilum/secimport/wiki/Performance-Benchmarks) benchmarks.
- Trace which syscalls are called by each module in your code.
- secimport uses USDT (Userland Statically Defined Tracing) together with kernel probes in the runtime using eBPF or dtrace instrumentation scripts.
## Installation
Tested on Ubuntu, Debian, Rocky (Linux x86/AMD/ARM) and MacOS in (x86/M1). If you run on MacOS you will need to <a href="https://github.com/avilum/secimport/blob/master/docs/MAC_OS_USERS.md">disable SIP for dtrace. </a>
## Examples
The [Sandbox Examples](https://github.com/avilum/secimport/wiki/Sandbox-Examples) page contains basic and advanced real-world examples.
# Getting Started
## Example Sandboxes
## With Docker
The [Sandbox Examples](https://github.com/avilum/secimport/wiki/Sandbox-Examples) page contains basic and advanced real-world examples.
## Using Docker
The quickest way to evaluate `secimport` is to use our [Docker container](docker/README.md), which includes `bpftrace` (`ebpf`) and other plug-and-play examples.
For quicker evaluation, we recommend using the <a href="https://github.com/avilum/secimport/tree/master/docker">Secimport Docker Image</a> instead of self-installing.<br>
- Build and run the Docker container with a custom kernel that matches your existing OS kernel version:
```
Expand All @@ -124,6 +93,8 @@ For quicker evaluation, we recommend using the <a href="https://github.com/avilu
A temporary container will be created, and you will be logged in as the root user.
## Without Docker
Tested on Ubuntu, Debian, Rocky (Linux x86/AMD/ARM) and MacOS in (x86/M1). If you run on MacOS you will need to <a href="https://github.com/avilum/secimport/blob/master/docs/MAC_OS_USERS.md">disable SIP for dtrace. </a>
1. Install python with USDT probes by <a href="https://github.com/avilum/secimport/wiki/Installation#python-interpreter-requirements">configuring it with '--dtrace'</a>
2. Install one of the backends: <a href="https://github.com/avilum/secimport/wiki/Installation">eBPF or DTrace</a>.
3. Install secimport
Expand All @@ -138,7 +109,7 @@ For quicker evaluation, we recommend using the <a href="https://github.com/avilu
```
## Usage
# Command Line Usage
To sandbox your program using the CLI, start a bpftrace program that logs all the syscalls for all the modules in your application into a file with the secimport trace command. Once you have covered the logic you would like to sandbox, hit CTRL+C or CTRL+D, or wait for the program to finish. Then, build a sandbox from the trace using the secimport build command, and run the sandbox with the secimport run command.
```shell
Expand Down Expand Up @@ -201,7 +172,7 @@ COMMANDS
Traces a running process by pid. It might require sudo privilleges on some hosts.
```

## Stop on violation
### Stop on violation
```
root@1bc0531d91d0:/workspace# secimport run --stop_on_violation=true
>>> secimport run
Expand All @@ -218,7 +189,7 @@ Type "help", "copyright", "credits" or "license" for more information.
PROCESS 85918 STOPPED.
```

## Kill on violation
### Kill on violation
```
root@ee4bc99bb011:/workspace# secimport run --kill_on_violation
>>> secimport run
Expand All @@ -237,7 +208,7 @@ Type "help", "copyright", "credits" or "license" for more information.
SANDBOX EXITED;
```

## Dynamic profiling - trace, build sandbox, run.
# Quickstart: trace, build, run.
```shell
root@1fa3d6f09989:/workspace# secimport interactive

Expand Down Expand Up @@ -289,27 +260,20 @@ Type "help", "copyright", "credits" or "license" for more information.

For more detailed usage instructions, see the [Command-Line Usage](https://github.com/avilum/secimport/wiki/Command-Line-Usage) page.

## nsjail support (seccomp)
# Advanced

## Python API
See the [Python Imports](examples/python_imports/) example for more details.<br>
You can use secimport directly from python, instead of the CLI.<br> you can replace `import` with `secimport.secure_import` for selected modules, and have them supervised in real time.<br>

## seccomp-bpf support using nsjail
Beside the sandbox that secimport builds, <br>
The `secimport build` command creates an <a href="https://github.com/google/nsjail">nsjail</a> sandbox with seccomp profile for your traced code.<br> `nsjail` enables namespace sandboxing with seccomp on linux<br>
`secimport` automatically generates seccomp profiles to use with `nsjail` as executable bash script.
It can be used to limit the syscalls of the entire python process, as another layer of defence.

## Python API

Instead of CLI, you can also use `secimport` by replacing "`import`" with "`secimport.secure_import`" for selected modules. See the [Python Imports](examples/python_imports/) example for more details.

## Docker
The quickest way to evaluate `secimport` is to use our [Docker container](docker/README.md), which includes `bpftrace` (`ebpf`) and other plug-and-play examples.
## Changelog
See the [Changelog](https://github.com/avilum/secimport/blob/master/docs/CHANGELOG.md) for development progress and existing features.

## Contributing

For information on how to contribute to `secimport`, see the [Contributing](https://github.com/avilum/secimport/blob/master/docs/CONTRIBUTING.md) guide.

## Roadmap

See the [Roadmap](https://github.com/avilum/secimport/blob/master/docs/ROADMAP.md) for the planned features and development milestones.

## Changelog

See the [Changelog](https://github.com/avilum/secimport/blob/master/docs/CHANGELOG.md) for development progress and existing features.
3 changes: 2 additions & 1 deletion docker/docker/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@ ARG KERNEL_VERSION

#FROM --platform=linux/amd64 linuxkit/kernel:${KERNEL_VERSION} as ksrc
FROM docker/for-desktop-kernel:5.10.25-6594e668feec68f102a58011bb42bd5dc07a7a9b as ksrc
FROM --platform=linux/amd64 ubuntu:latest AS build
# FROM --platform=linux/amd64 ubuntu:latest AS build
FROM ubuntu:latest AS build

ARG BPFTRACE_VERSION
ARG PYTHON_VERSION
Expand Down
1 change: 1 addition & 0 deletions docs/FAQ.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
- [Is it safe to use in production?](#is-it-safe-to-use-in-production)
- [Should I use dtrace or bpftrace backend?](#should-i-use-dtrace-or-bpftrace-backend)
- [What are the tradeoffs? How does it change they way I code?](#what-are-the-tradeoffs-how-does-it-change-they-way-i-code)
- [On Ubuntu, I get the error message `ERROR: Could not resolve symbol: /proc/self/exe:BEGIN_trigger`](#on-ubuntu-i-get-the-error-message-error-could-not-resolve-symbol-procselfexebegin_trigger)
- [What are the performance impacts?](#what-are-the-performance-impacts)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->
Expand Down

0 comments on commit 92f9b18

Please sign in to comment.