How do I configure my Gramine manifest so that shmget() works?

[fwoodruff-ab]

I am looking to run Postgres inside Gramine. Postgres allocates shared memory upfront. I am configuring pg_hba.conf so that the shared memory does not leak anything we are concerned about.
Therefore I need shmget()/shmctl().

Almost all code that uses shm* instructions within Postgres is in this file:
https://github.com/postgres/postgres/blob/master/src/backend/port/sysv_shmem.c

I have the following manifest file.

libos.entrypoint = "/postgres/entrypoint.sh"

loader.entrypoint = "file:/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/libsysdb.so"

loader.env.LD_LIBRARY_PATH = "/gramine/meson_build_output/lib/x86_64-linux-gnu/gramine/runtime/glibc:/usr/lib/x86_64-linux-gnu:{{"{{library_paths}}"}}"

loader.env.PATH = "{{env_path}}"
loader.log_level = "error"

fs.root.type = "chroot"
fs.root.uri = "file:/"

fs.start_dir = "/postgres"
sgx.debug = false

sys.enable_sigterm_injection = true
sys.experimental__enable_flock = true

sys.stack.size = "16M"
sys.enable_extra_runtime_domain_names_conf = true
sgx.nonpie_binary = true
sgx.enclave_size = "4G"
sgx.edmm_enable = false
sgx.max_threads = 32

sys.insecure__allow_eventfd = true

sgx.trusted_files = [
  "file:/etc/host.conf",
  "file:/etc/hosts",
  "file:/etc/nsswitch.conf",
  "file:/etc/resolv.conf",
  "file:/postgres/entrypoint.sh",
  "file:/usr/local/bin/",
  "file:/usr/local/pgsql/data/",
  "file:/var/log/postgresql/",
]

fs.mounts = [
  { path = "/var/lib/postgresql/data", uri = "file:/var/lib/postgresql/data" },
  { type = "tmpfs", path = "/dev/shm" },
  { type = "tmpfs", path = "/tmp" },
  { type = "untrusted_shm", path = "/dev/shm", uri = "dev:/dev/shm", mode = "0777" }
]

sgx.allowed_files = [
  "file:/var/lib/postgresql/data",
  "file:/docker-entrypoint-initdb.d/init.sql",
  "file:/"
]

loader.env.TZ = ":/etc/localtime"
loader.env.SGX_AESM_ADDR="1"

loader.uid = 1000
loader.gid = 1000

loader.env.POSTGRES_USER = "user"
loader.env.POSTGRES_PASSWORD = "pass"
loader.env.POSTGRES_DB = "our_db"
loader.env.PGDATA = "/usr/local/pgsql/data"

sys.insecure__shared_memory = "passthrough"

sgx.allowed_files = [ "file:/" ] is staying in there until I have something that actually runs. It is harder to test security assumptions about code you can't run.

What changes should I make to enable the various shared memory system calls?

[dimakuv]

@fwoodruff-ab I have bad news for you.

Gramine currently doesn't support System-V shared memory (https://man7.org/linux/man-pages/man7/sysvipc.7.html). Please check this docs: https://gramine.readthedocs.io/en/stable/devel/features.html#shared-memory

What Gramine supports is the POSIX shared memory (https://man7.org/linux/man-pages/man7/shm_overview.7.html), with the untrusted_shm FS mount type (as you correctly use in your manifest).

I'm afraid Postgres is adamant on requiring System-V shared memory, they have an interesting top-level comment: https://github.com/postgres/postgres/blob/c50d4f4028e5518511b9bfc3a17860a90dc88357/src/backend/port/sysv_shmem.c#L43-L55

So Postgres requires both Sys-V shared memory and POSIX shared memory to be implemented in the OS. Gramine unfortunately implements only POSIX shared memory.

Technically we could implement Sys-V shared memory, but it would be a major feature request and would take some time...

Alternatively, you/we can bug Postgres developers to remove Sys-V requirement (or make it an optional feature).

[fwoodruff-ab]

Hi @dimakuv, I am always super impressed by how quickly you reply to everything! You have saved us a few times.

Since we're here, loader.uid = 1000 is needed because Gramine overrides the Dockerfile's user ID. It would reduce friction substantially if one could just run GSC on a Docker file with a verbose and overly-permissive manifest file and the shielded container just works (with loud warning messages) without making assumptions about the executable running. It's easier to remove the various security-hole stabiliser-wheels on something that runs in the first place, than diagnose cryptic error messages in something that doesn't work. At the same time I agree that Postgres is being overzealous, and if System V is difficult to implement for Gramine, that's not your fault. Engineering is hard.

In the first instance, I'm going to try to implement the System V calls in terms of the POSIX calls, just enough to make the problem go away. Failing that, I've come across this SGX postgres database. Do you have any experience with it? Should we anticipate any headaches adding RA-TLS functionality inside of it?

[dimakuv]

I'm going to try to implement the System V calls in terms of the POSIX calls, just enough to make the problem go away.

Yes, that would be my suggestion too.

Failing that, I've come across this SGX postgres database. Do you have any experience with it? Should we anticipate any headaches adding RA-TLS functionality inside of it?

I don't know this project, sorry. I can't comment on it.