Skip to content

ARM CCA emulation on QEMU (New Community Version)

Jump to bottom
Wu Yongzheng edited this page Jul 23, 2023 · 2 revisions

Note the old version which was created by Huawei from scratch is at https://github.com/Huawei/Huawei_CCA_RMM/wiki

Introduction

This project brings ARM's Confidential Computing Architecture (CCA) to the QEMU platform. The current ARM CCA software stack from Trusted Firmware only supports the FVP platform. By using QEMU, it allows experimenting and debugging the whole software and hardware stack. We are in the process of contributing this code to QEMU and Trusted Firmware. Before the code is merged to the upstream, feature and fixes can be submitted to us through pull requests or email to [email protected]

Disclaimer: This is not an actual product of Huawei. Use it for research purpose only. Do NOT use it for production.

How to build

Prepare the compiler:

export PATH=/path/to/gcc-arm-10.3-2021.07-x86_64-aarch64-none-elf/bin:$PATH

The working directory contains 4 subdirectories:

# build qemu
mkdir qemu/build
cd qemu/build
../configure --target-list=aarch64-softmmu --disable-docs --enable-debug
make -j 40
cd ../../

# build qemu
env CROSS_COMPILE=aarch64-none-elf- cmake -DRMM_CONFIG=qemu_defcfg -S tf-rmm -B tf-rmm/build -DCMAKE_BUILD_TYPE=Debug -DLOG_LEVEL=50
env CROSS_COMPILE=aarch64-none-elf- cmake --build tf-rmm/build -j 20

# build tf-a
make -C tf-a-tests -j 20 CROSS_COMPILE=aarch64-none-elf- PLAT=qemu DEBUG=1 LOG_LEVEL=50 TESTS=realm-payload pack_realm
make -C trusted-firmware-a -j 20 CROSS_COMPILE=aarch64-none-elf- ARCH=aarch64 PLAT=qemu ENABLE_RME=1 DEBUG=1 RMM=../tf-rmm/build/Debug/rmm.img all BL33=../tf-a-tests/build/qemu/debug/tftf.bin all fip

# make flash image for qemu
rm -f flash.bin
dd if=trusted-firmware-a/build/qemu/debug/bl1.bin of=flash.bin bs=4096 conv=notrunc
dd if=trusted-firmware-a/build/qemu/debug/fip.bin of=flash.bin seek=64 bs=4096 conv=notrunc

How to run

./qemu/build/qemu-system-aarch64 -nographic -serial telnet::54340,server \
  -smp clusters=2,cores=4 -machine virt,secure=on,rmm=on,virtualization=on,gic-version=3 \
  -m 2048 -cpu max,lpa2=off -d unimp,guest_errors -D qemu.log \
  -bios flash.bin

In another terminal, run telnet localhost 54340 to get test output like this:

NOTICE:  Booting Trusted Firmware
NOTICE:  BL1: v2.8(debug):v2.8-726-g04f59c4a6-dirty
NOTICE:  BL1: Built : 23:37:30, Apr 14 2023
INFO:    BL1: RAM 0xe0ee000 - 0xe0f6000
INFO:    BL1: Loading BL2
INFO:    Loading image id=1 at address 0xe06b000
INFO:    Image id=1 loaded: 0xe06b000 - 0xe0742b9
NOTICE:  BL1: Booting BL2
INFO:    Entry point address = 0xe06b000
INFO:    SPSR = 0x3cd
INFO:    [GPT] Boot Configuration
INFO:      PPS/T:     0x1/36
INFO:      PGS/P:     0x0/12
INFO:      L0GPTSZ/S: 0x0/30
INFO:      PAS count: 0x4
INFO:      L0 base:   0xe001000
INFO:    [GPT] PAS[0]: base 0x0, size 0xf000000, GPI 0xf, type 0x1
INFO:    [GPT] PAS[1]: base 0xffff000, size 0x30001000, GPI 0xf, type 0x1
INFO:    [GPT] PAS[2]: base 0x40000000, size 0xc0000000, GPI 0x9, type 0x1
INFO:    [GPT] PAS[3]: base 0xf000000, size 0xfff000, GPI 0xb, type 0x1
INFO:    Enabling Granule Protection Checks
NOTICE:  BL2: v2.8(debug):v2.8-726-g04f59c4a6-dirty
NOTICE:  BL2: Built : 23:37:30, Apr 14 2023
INFO:    BL2: Doing platform setup
INFO:    BL2: Loading image id 3
INFO:    Loading image id=3 at address 0xe0a0000
INFO:    Image id=3 loaded: 0xe0a0000 - 0xe0ae374
INFO:    BL2: Loading image id 34
INFO:    Loading image id=34 at address 0xf000000
INFO:    Image id=34 loaded: 0xf000000 - 0xf201ce8
INFO:    BL2: Loading image id 5
INFO:    Loading image id=5 at address 0x60000000
INFO:    Image id=5 loaded: 0x60000000 - 0x60a06188
NOTICE:  BL2: Booting BL31
INFO:    Entry point address = 0xe0a0000
INFO:    SPSR = 0x3cd
NOTICE:  BL31: v2.8(debug):v2.8-726-g04f59c4a6-dirty
NOTICE:  BL31: Built : 23:37:30, Apr 14 2023
INFO:    GICv3 without legacy support detected.
INFO:    ARM GICv3 driver initialized in EL3
INFO:    Maximum SPI INTID supported: 287
INFO:    BL31: Initializing runtime services
INFO:    RMM setup done.
INFO:    BL31: Initializing RMM
INFO:    RMM init start.
mmap:
...
NOTICE:  Booting trusted firmware test framework
NOTICE:  Built : 23:37:25, Apr 14 2023
NOTICE:  v2.8(qemu,debug):v2.8-59-g79fc919-dirty

NOTICE:  Running at NS-EL2
INFO:    GICv3 mode detected
NOTICE:  Platform topology:
NOTICE:    2 cluster(s)
NOTICE:    8 CPU(s) (total)

NOTICE:    Cluster #0   [4 CPUs]
NOTICE:      CPU #0   [MPID: 0x0]
NOTICE:      CPU #1   [MPID: 0x1]
NOTICE:      CPU #2   [MPID: 0x2]
NOTICE:      CPU #3   [MPID: 0x3]
NOTICE:    Cluster #1   [4 CPUs]
NOTICE:      CPU #4   [MPID: 0x100]
NOTICE:      CPU #5   [MPID: 0x101]
NOTICE:      CPU #6   [MPID: 0x102]
NOTICE:      CPU #7   [MPID: 0x103]
NOTICE:
INFO:    Registered IRQ handler 0x600010f0 for IRQ #2
VERBOSE: Enabled IRQ #7
INFO:    Always starting a new test session (NEW_TEST_SESSION == 1)
NOTICE:  Starting a new test session
INFO:    Initialising NVM
VERBOSE: Enabled IRQ #0
INFO:    Going into suspend state
INFO:    Resumed from suspend state
VERBOSE: Disabled IRQ #0
INFO:    Original PSCI power state format detected
--
Running test suite 'Realm payload at EL1'
Description: Test Realm EL1 framework capabilities

> Executing 'Realm EL1 creation and execution test'
VERBOSE: Entering the test (1 CPUs in the test now)
SMC_RMM_VERSION               41a710d63af1 60b7acd93ab50c2a 4431b7821c06dac8 6058ed856e509fe 56f32f4377a4044d > 380000
VERBOSE: RMM version is: 56.0
SMC_RMM_FEATURES                     0 46cdbe206d7d4b3 43cd374755e83917 fc41113113c398 88e495435702e2f > RMI_SUCCESS 33400030
Measurement (SHA256): 0xad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7
...
Realm: waiting for PMU vIRQ...
Realm: PMU vIRQ not received in 3000ms
SMC_RSI_HOST_CALL             60b07000        0        0        0        0
VERBOSE: host_realm_rec_enter() run->exit.exit_reason=5 run->exit.esr=0x0 EC_BITS=0 ISS_DFSC_MASK=0x0
ERROR:   host_enter_realm_execute(6) RMI_EXIT_HOST_CALL host_call_result=2
VERBOSE: Exiting the test  (1 CPUs in the test now)
INFO:    Powering off
SMC_RMM_REC_DESTROY           60c07000 41c68531505275b3 55ad63f76a7e1912 7ca0235c7571acef 7a10bb25606eb8be > RMI_SUCCESS
SMC_RMM_REALM_DESTROY         60c00000 7f9b7d41393c77da 30e0a6916cd792a8 3cdd8b6b74ec3d04 419a0d9468f19131 > RMI_SUCCESS
ERROR:   host_test_realm_pmuv3() enter=0 destroy=1
VERBOSE: Exiting the test  (0 CPUs in the test now)
  TEST COMPLETE                                                 Failed

******************************* Summary *******************************
> Test suite 'Realm payload at EL1'
                                                                Failed
=================================
Tests Skipped : 0
Tests Passed  : 6
Tests Failed  : 4
Tests Crashed : 0
Total tests   : 10
=================================
NOTICE:  Exiting tests.

How to debug

Add -s -S command line option in qemu, like this:

./qemu/build/qemu-system-aarch64 -s -S -nographic -serial telnet::54340,server \
  -smp clusters=2,cores=4 -machine virt,secure=on,rmm=on,virtualization=on,gic-version=3 \
  -m 2048 -cpu max,lpa2=off -d unimp,guest_errors -D qemu.log -bios flash.bin

In another terminal, run telnet localhost 54340. In a third terminal, run aarch64-none-linux-gnu-gdb. Then you can start debugging like this:

target remote localhost:1234
add-symbol-file trusted-firmware-a/build/qemu/debug/bl31/bl31.elf
add-symbol-file tf-rmm/build/Debug/rmm.elf 0xf000000
add-symbol-file tf-a-tests/build/qemu/debug/tftf/tftf.elf
break runtime/rmi/rtt.c:1111
continue
Clone this wiki locally