Enclave hangs up if Dockerfile CMD has a relative path

[abhinit]

I have a dockerfile with a relative path in the CMD:

CMD ["python3" , "./ubuntu-python-server/server.py"]

or

WORKDIR /home
CMD ["python3" , "ubuntu-python-server/server.py"]

An enclave created using enclave-run command is created and terminated immediately due to (a possible) missing socket connection. /run/nitro_enclaves/ has no .sock file.

The complete log is as follows:

[nitro-cli:28204][INFO][2022-06-22T06:28:57.279Z][src/main.rs:72] Start Nitro CLI
[nitro-cli:28204][INFO][2022-06-22T06:28:57.279Z][src/main.rs:115] Sent command: Run
[enc-xxxxxxx:28206][INFO][2022-06-22T06:28:57.280Z][src/enclave_proc/mod.rs:571] Enclave process PID: 28206
[enc-xxxxxxx:28206][INFO][2022-06-22T06:28:57.280Z][src/enclave_proc/mod.rs:479] Received command: Run
[enc-xxxxxxx:28206][INFO][2022-06-22T06:28:57.280Z][src/enclave_proc/mod.rs:272] Run args = RunEnclavesArgs { eif_path: "./d3.eif", enclave_cid: Some(17), memory_mib: 3072, cpu_ids: None, debug_mode: Some(true), attach_console: false, cpu_count: Some(2), enclave_name: Some("d3_error") }
[enc-xxxxxxx:28206][INFO][2022-06-22T06:28:57.280Z][src/enclave_proc/resource_manager.rs:371] Allocating memory regions to hold 3221225472 bytes.
[enc-xxxxxxx:28206][INFO][2022-06-22T06:28:57.281Z][src/enclave_proc/resource_manager.rs:453] Allocated 3 region(s): 3 page(s) of 1024 MB
[enc-xxxxxxx:28206][INFO][2022-06-22T06:28:58.019Z][src/enclave_proc/resource_manager.rs:693] Finished initializing memory.
[enc-xxxxxxx:28206][INFO][2022-06-22T06:29:02.956Z][src/enclave_proc/mod.rs:281] Enclave ID = i-0dca5a2cb0a6e6ffc-enc1818a1985367667
[enc-1818a1985367667:28206][WARN][2022-06-22T06:29:03.556Z][src/enclave_proc/mod.rs:207] Received hang-up event from the enclave. Enclave process will shut down.
[enc-1818a1985367667:28206][INFO][2022-06-22T06:29:03.556Z][src/enclave_proc/mod.rs:541] Enclave process 28206 exited event loop.
[enc-1818a1985367667:28206][INFO][2022-06-22T06:29:03.558Z][src/enclave_proc/resource_manager.rs:762] Enclave terminated.
[nitro-cli:28211][INFO][2022-06-22T06:29:15.579Z][src/main.rs:72] Start Nitro CLI
[nitro-cli:28211][INFO][2022-06-22T06:29:15.579Z][src/main.rs:211] Sent command: Describe

It succeeds if I use an absolute path in the dockerfile CMD:

CMD ["python3" , "/home/ubuntu-python-server/server.py"]

Recreating the error:

Dockerfile:

# Fetch ubuntu
FROM ubuntu:bionic

WORKDIR /home

COPY server.py /home/server.py

# Get packages
RUN apt-get update
RUN apt-get install python3 -y
RUN apt-get install -f -y

CMD ["python3" , "./server.py"]

server.py:

# // Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# // SPDX-License-Identifier: MIT-0

import time

def main():
    count = 1
    while True:
        print(f"[{count:4d}] Hello from the enclave side!")
        count += 1
        time.sleep(5)

if __name__ == '__main__':
    main()

Build image docker build ./ -t d3_error
Build enclave image nitro-cli build-enclave --docker-uri d3_error:latest --output-file ./d3_error.eif
Run enclave: nitro-cli run-enclave --cpu-count 2 --memory 1024 --eif-path ./d3_error.eif --debug-mode --enclave-cid 17
Describe enclaves nitro-cli describe-enclaves returns []

Just to add, docker run succeeds docker run -i -t --name d3_error_c d3_error:latest

[eugkoira]

We set a working directory for CMD execution to a rootfs root folder (https://github.com/aws/aws-nitro-enclaves-sdk-bootstrap/blob/main/init/init.c#L428). That basically means root of the enclave file system. In this case your relative path should be valid from /.

WORKDIR dockerfile directive is unfortunately not yet handled when building enclaves. But it sounds like a good proposal, we will add it to our TODO list!

[shamiek]

@abhinit What was the OS (which release of Ubuntu/Amazon Linux? or something else?) of the instance on which you were building the enclave docker image? I see that bionic is used as the base for the enclave.
The issue I am facing is that I can only run an enclave image (eif) on Ubuntu 18.04 ec2 instance when I use Alpine linux as the base for the enclave docker file. I was wondering if I could use Ubuntu bionic as the base for the enclave docker file as you did (FROM ubuntu:bionic).

[abhinit]

@shamiek I used ubuntu bionic. You can find the docker template here. But I moved to Amazon Linux 2 later to support KMS (and other) libraries.

[shamiek]

Thanks a ton. This is helpful. It is promising to know that bionic ec2 parent instances can run nitro docker images based on bionic. I will double check the nitro installation/configuration on my bionic ec2 instance which for some reason (maybe kernel version or something in my Dockerfile like you experienced with WORKDIR) can only ever run nitro docker images based on Alpine linux. I will then run your example.

[swakv]

Hello, I am facing the same issue, I am using the Amazon Linux 2 and mentioning the full path in the Dockerfile, yet it still does not seem to work

[Luke-Rogerson]

We set a working directory for CMD execution to a rootfs root folder (https://github.com/aws/aws-nitro-enclaves-sdk-bootstrap/blob/main/init/init.c#L428). That basically means root of the enclave file system. In this case your relative path should be valid from /.

WORKDIR dockerfile directive is unfortunately not yet handled when building enclaves. But it sounds like a good proposal, we will add it to our TODO list!

I believe the fact that the WORKDIR directive is unsupported is not documented anywhere, correct? I just spent a while debugging why my Enclave kept crashing with a generic E11 socket error and it turned out it was because of this. Probably worth making clear what Dockerfile directives are not supported.