10. January 2023

[dimakuv]

Agenda

(please write your proposed agenda items in comments under this discussion)

• Frieder Paape and others: Using Gramine to run Geth in SGX, experiences and discussion.

Using Gramine to run Geth in SGX

Frieder Paape: presented Gramine Buildpacks 1 year ago. Today would like to present Geth (go-ethereum) running in Gramine. Also would like to introduce Patrick and Phillipp, both working on Gramine-related problems.

Geth + Gramine -- solves pre-confirmation privacy (your transaction is private to you until it is actually committed). Without privacy, somebody could see your transaction and steal your profit. But there's more to it -- if you're able to have a public pool of transactions that are not yet committed, then there is a consensus risk. (Note from Dmitrii: I am not sure I captured these correctly.)

There is also another project -- adding SGX as a 2nd factor for zero-knowledge rollup state transition proofs. (Note from Dmitrii: I didn't capture the name of that other project.)

Enabling Geth in Gramine:

• unsupported flock() syscall, but circumvented with a patch to Geth.
• without file encryption: 50% perf overhead in Gramine-SGX.
• trying encrypted file mounts: 10-100x slower (800GB database with random access).
• finally opted for purely in-memory approach (with a swap file since EPC was less that 800GB -- around 256GB) -- back to 50% overhead.

Tried other SGX frameworks:

• Ego:
  • pro: good docs
  • pro: convenient workaround for missing syscalls
  • con: weak perf
  • con: no encrypted files
• Occlum
  • pro: good perf
  • con: incomplete, unstable
  • con: weak documentation

Future work:

• flock() implementation in Gramine.
• Reproducible Builds.
• Oblivious RAM:
  • Michal: Cannot be done at the level of Gramine. Gramine only intercepts syscalls, but not memory accesses.
  • Mona: there is an Opaque project for SGX (only for Spark).
  • Chia-Che: there is a CoSMIX project (recompile and instrument all memory accesses).
• Encrypted files performance:
  • Patrik is looking into this.
  • Michal: our encrypted-files implementation is theoretically slow for GBs of data, so you could tweak the parameters but should not expect significant (10x) improvement.

Introducing Patrik and Phillipp:

• Patrik: no questions currently, maybe in the coming weeks.
• Phillipp: can we piggy-back on already-implemented fcntl() and POSIX locks and use them as the building blocks for flock()? Looking at some other projects, it seems possible to use flock() via fcntl(). And how to go about implementing syscalls in general?
  • Michal: How to implement syscalls: you read the Linux kernel sources, the man page, and try to implement similarly but using Gramine interfaces and structs, and also relying on the PAL interfaces.
  • Dmitrii: If there is already a real success story of implementing flock() on top of fcntl(), then we can analyze it and implement in the same way. But be aware -- if it's implemented by some project, doesn't mean it's correct/stable.

Syscall plugins -- any possibility to have such feature in Gramine?

• Michal: we can't have a "syscall" plugin in Gramine, but we could have "dummy always-success" syscall mentions in Gramine manifest.
• Chia-Che: we can always tell users to use function interception (LD_PRELOAD a lib that intercepts Glibc-wrapper syscall functions). Con: doesn't work for Google Go apps, since they don't use Glibc.

Other topics

We didn't have time to discuss other topics, so moving all @woju's topics to next meeting.

[woju]

• tracking vulnerabilities in upstream projects (like CVE-2022-46392, which we've narrowly missed)

An issue was discovered in Mbed TLS before 2.28.2 and 3.x before 3.3.0. An adversary with access to precise enough information about memory accesses (typically, an untrusted operating system attacking a secure enclave) can recover an RSA private key after observing the victim performing a single private-key operation, if the window size (MBEDTLS_MPI_WINDOW_SIZE) used for the exponentiation is 3 or smaller.

• deprecation of gramine-sgx-get-token (Do gramine-sgx-get-token automatically #1093, Remove gramine-sgx-get-token from upstream/DCAP flavour #1094)

[kailun-qin]

An issue was discovered in Mbed TLS before 2.28.2 and 3.x before 3.3.0. An adversary with access to precise enough information about memory accesses (typically, an untrusted operating system attacking a secure enclave) can recover an RSA private key after observing the victim performing a single private-key operation, if the window size (MBEDTLS_MPI_WINDOW_SIZE) used for the exponentiation is 3 or smaller.

For this security issue, I actually tracked and checked it before making our mbedTLS upgradation from v3.2.1 to v3.3.0: #1088. Since the maximum window size used for modular exponentiation is 6 by default (which we use): https://github.com/Mbed-TLS/mbedtls/blob/v3.2.1/include/mbedtls/bignum.h#L64-L75, it doesn't seem to impact Gramine.

[kailun-qin]

tracking vulnerabilities in upstream projects (like GHSA-2cww-m3rc-2vgr, which we've narrowly missed)

We should and probably already have some internal setup and tracking for these (like via BDBA security scans etc.). cc @dimakuv @aneessahib

[aneessahib]

Yes we have BDBA set up. Its usually run on the final packages before release. (and at a quarterly cadence on currently shipping packages)

[woju]

• gramine-sgx-sign plugin architecture