Couldn't run SGX-LKL examples

[kanakraju73]

I'm getting exactly the same error as this while running run-https-server and run-wolfssl-server examples with SGX-LKL.

I did run the EXTERNAL_IFACE=eth0 make -C sgxlkl up-sgxlkl-network before running the examples.

The last few lines of https-server reads:

make[1]: Leaving directory '/home/bhamidi/clones/sgx-ra-tls/sgxlkl/sgx-lkl'
LD_PRELOAD=/ldpreload.so SGXLKL_TAP=sgxlkl_tap0 RATLS_AESMD_IP=10.0.1.254 SGXLKL_HEAP=268435456 SGXLKL_KEY=/home/bhamidi/clones/sgx-ra-tls/sgxlkl/sgx-lkl/build/config/enclave_debug.key sgx-lkl/build/sgx-lkl-run sgx-lkl/apps/https-server/sgxlkl-miniroot-fs.img /usr/bin/python /https-server.py
Makefile:118: recipe for target 'run-https-server' failed
make: Leaving directory '/home/bhamidi/clones/sgx-ra-tls/sgxlkl'
Assertion failed: rc != -1 (../nonsdk-ra-attester.c: open_socket: 262)
Segmentation fault
make: *** [run-https-server] Error 139

The detailed logs for both the example are attached.
run-wolfssl-server
run-https-server

Note: I'm able to run SGX-SDK and graphene examples without any problems.

Where am I going wrong?

[thomasknauth]

Can you run ifconfig on the host and paste the output here. Thanks.

[kanakraju73]

bhamidi@wp-04:~$ ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
ether 02:42:a6:fc:3a:ca txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 141.12.90.185 netmask 255.255.252.0 broadcast 141.12.91.255
inet6 fe80::bb23:7497:21e3:c3d6 prefixlen 64 scopeid 0x20
ether b8:85:84:ba:72:4c txqueuelen 1000 (Ethernet)
RX packets 6624827 bytes 1149487211 (1.1 GB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 16867705 bytes 23906175817 (23.9 GB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 20 memory 0x7f000000-7f020000

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 1000 (Local Loopback)
RX packets 18364 bytes 1650208 (1.6 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 18364 bytes 1650208 (1.6 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

sgxlkl_tap0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 10.0.1.254 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::9035:ceff:fe84:215f prefixlen 64 scopeid 0x20
ether 92:35:ce:84:21:5f txqueuelen 1000 (Ethernet)
RX packets 70 bytes 5044 (5.0 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 111 bytes 12695 (12.6 KB)
TX errors 0 dropped 4 overruns 0 carrier 0 collisions 0

vmnet1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.16.66.1 netmask 255.255.255.0 broadcast 172.16.66.255
inet6 fe80::250:56ff:fec0:1 prefixlen 64 scopeid 0x20
ether 00:50:56:c0:00:01 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 4905 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vmnet8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.16.144.1 netmask 255.255.255.0 broadcast 172.16.144.255
inet6 fe80::250:56ff:fec0:8 prefixlen 64 scopeid 0x20
ether 00:50:56:c0:00:08 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 4907 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

[thomasknauth]

I am guessing that eno1 is your "external interface". Hence, you could try to run EXTERNAL_IFACE=eno1 make -C sgxlkl up-sgxlkl-network and see if that solves your problem.

[kanakraju73]

I did try that. Unfortunately, that didn't work either.

The app stays blank for some time before it fails.

LD_PRELOAD=/ldpreload.so SGXLKL_TAP=sgxlkl_tap0 RATLS_AESMD_IP=10.0.1.254 SGXLKL_HEAP=268435456 SGXLKL_KEY=/home/bhamidi/clones/sgx-ra-tls/sgxlkl/sgx-lkl/build/config/enclave_debug.key sgx-lkl/build/sgx-lkl-run sgx-lkl/apps/https-server/sgxlkl-miniroot-fs.img /usr/bin/python /https-server.py
ERROR: ld.so: object '/ldpreload.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
Assertion failed: rc != -1 (../nonsdk-ra-attester.c: open_socket: 262)
Segmentation fault
Makefile:118: recipe for target 'run-https-server' failed
make: *** [run-https-server] Error 139
make: Leaving directory '/home/bhamidi/clones/sgx-ra-tls/sgxlkl'

client:

bhamidi@wp-04:~/clones/sgx-ra-tls$ echo -n hello | ./openssl-client -p 11111 -h 10.0.1.1
errored; unable to connect.
139999029147328:error:0200206F:system library:connect:Connection refused:bss_conn.c:246:host=10.0.1.1:11111:https
139999029147328:error:20073067:BIO routines:CONN_STATE:connect error:bss_conn.c:249:
139999029147328:error:140E0197:SSL routines:SSL_shutdown:shutdown while in init:ssl_lib.c:1066:


bhamidi@wp-04:~/clones/sgx-ra-tls$ echo -n hello | ./openssl-client -p 4443 -h 10.0.1.1
errored; unable to connect.
139685708576448:error:0200206F:system library:connect:Connection refused:bss_conn.c:246:host=10.0.1.1:4443:https
139685708576448:error:20073067:BIO routines:CONN_STATE:connect error:bss_conn.c:249:
139685708576448:error:140E0197:SSL routines:SSL_shutdown:shutdown while in init:ssl_lib.c:1066:

[thomasknauth]

Hard to tell from a distance. Likely something in your environment. The code running in SGX-LKL cannot access the AESMD socket on the host for whatever reason. Maybe a firewall?

[kanakraju73]

I've disabled the firewall.
Now I got the following error.

[    0.054328] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    0.054998] Warning: unable to open an initial console.
[    0.055062] This architecture does not have kernel memory protection.
[    0.058658] EXT4-fs (vda): recovery complete
[    0.058683] EXT4-fs (vda): mounted filesystem with ordered data mode. Opts: 
Assertion failed: res == CURLE_OK (ias-ra.c: http_get: 67)
Segmentation fault
Makefile:122: recipe for target 'run-wolfssl-server' failed
make: *** [run-wolfssl-server] Error 139
make: Leaving directory '/home/bhamidi/Desktop/sgx-ra-tls/sgxlkl'

[thomasknauth]

You can add/enable some debug output in ias-ra.c to see what exactly the HTTP response is. The assertion fails because curl return an error code for one of its functions. Would be helpful to find out the error code and/or the HTTP response (printf header and body should be enough. Don't forget to flush the stream). Likely, it's some certificate issue.

[thomasknauth]

Since I just experience this myself. The error you are running into above can also happen if SGXLKL cannot talk to IAS, i.e., the response is empty. In my case, if forgot to specify EXTERNAL_IFACE when starting to run the workload, e.g., EXTERNAL_IFACE=... make check.

[kanakraju73]

Thanks for the reply. As of now, I'm not receiving any response. I'll dig a little more and come up with the findings.