Using EL3 Registers

[instrangeland]

Hey y'all! My lab and I are doing tests on the caches using direct access (see https://developer.arm.com/documentation/100798/0401/L1-memory-system/Direct-access-to-internal-memory) on the RPi5, which are all EL3 registers. I know there are some ways to do some operations in EL3 and then switch back using the SMC, but I feel like it might be easier to just stay in EL3 by editing the armstub (apparently it still does support arbitrary armstubs, see https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/plat/rpi5.rst)... Yes the example armstub they provide only does EL2, but I'm sure I could hunt down or cobble together one that doesn't switch from EL3.

Is there any big issues with circle that would occur from not switching from EL3? I'm not using most of circle's advanced features (usb, networking, screen, etc.)

[rsta2]

I'm afraid the whole interrupt system won't work in Circle, if you don't switch to EL1 in the startup phase.

[instrangeland]

Is this just because the SCR_EL3 register is set to have EL3 not take IRQs?

[rsta2]

Is this just because the SCR_EL3 register is set to have EL3 not take IRQs?

No, it's the Circle model in AArch64 to use EL1t for TASK_LEVEL and EL1h for IRQ_LEVEL and FIQ_LEVEL. This affects (at least) the system initialization, exception stubs, and the critical sections handling in lib/synchronize.cpp.

[rsta2]

I figured out how to add custom user functions to the Trusted Firmware module, used for the Raspberry Pi 5. It's not difficult and maybe this could help you. A short description follows:

First you have to clone the Trusted Firmware project from GitHub:

git clone https://github.com/raspberrypi/arm-trusted-firmware.git
cd arm-trusted-firmware/
git checkout bcm2712

Now add the following file to the sub-directory plat/rpi/rpi5/ and name it rpi5_oem_test_svc.c:

#include <common/runtime_svc.h>
#include <common/debug.h>

/* SMC function IDs for OEM Service queries */
#define OEM_SVC_CALL_TEST	0x8300ff00

/*
 * This function is responsible for handling all OEM calls from the NS world
 */
static uintptr_t oem_smc_handler(uint32_t smc_fid,
				u_register_t x1,
				u_register_t x2,
				u_register_t x3,
				u_register_t x4,
				void *cookie,
				void *handle,
				u_register_t flags)
{
	uint32_t ns;

	/* Determine which security state this SMC originated from */
	ns = is_caller_non_secure(flags);
	if (!ns)
		SMC_RET1(handle, SMC_UNK);

	switch (smc_fid) {
	case OEM_SVC_CALL_TEST:
		/* TODO: Some operation at EL3 here. */

		/* Return some value. */
		SMC_RET1(handle, 0xFEDCBA9876543210UL);

	default:
		ERROR("%s: unhandled SMC (0x%x)\n", __func__, smc_fid);
		SMC_RET1(handle, SMC_UNK);
	}
}

/* Define a runtime service descriptor for fast SMC calls */
DECLARE_RT_SVC(
	rpi5_oem_test_svc,
	OEN_OEM_START,
	OEN_OEM_END,
	SMC_TYPE_FAST,
	NULL,
	oem_smc_handler
);

This declares a runtime SMC service with the OEM ID (3) and the handler function oem_smc_handler(). This example has only one service OEM_SVC_CALL_TEST with the function ID 0x8300ff00. This ID has to be loaded into the x0 register, before the smc #0 instruction to call this service.

Now add this file to the project by appending this line to the file platform.mk in the same sub-directory:

BL31_SOURCES += plat/rpi/rpi5/rpi5_oem_test_svc.c

You can build the Trusted Firmware now from project root using:

CROSS_COMPILE=aarch64-none-elf- make PLAT=rpi5

Copy the file build/rpi5/release/bl31.bin to your SD card and add this to the file config.txt:

armstub=bl31.bin

Now when you add the following to your Circle application, it should show the value returned from the runtime service:

	u64 ulResult = 0;

	asm volatile
	(
		"mov	x0, %1\n"
		"smc	#0\n"
		"mov	%0, x0\n"

		: "=r" (ulResult)
		: "r" (0x8300FF00UL)			// function code OEM_TEST
	);

	m_Logger.Write ("kernel", LogNotice, "Result is 0x%lX", ulResult);

You can find further related information in the file docs/getting_started/rt-svc-writers-guide.rst in the project.

[instrangeland]

Thank you! I appreciate it. I'm exploring few options (we're trying to be reading the entirety of cache to watch for bit upsets while under a radiation beam, so it'll need to be high throughput...) This is definitely one of them. I'll definitely test this and see if I can get the speed I need (I might just need to modify the linker script to make room for a decent sized buffer or something). Thank you again!