Approach to implement Bao onto Renesas RZ/T2M hardware.

[pmittireddy]

Hi ,
I request your assistance to port Bao onto Renesas RZ/T2M hardware in creating 2 isolated VMs (Baremetal + RTOS).

[josecm]

Hey @pmittireddy. We have ran Bao on that board before and I can get you the fork we have used to do so.

However, there is a major issue with that platform. Despite being a dual-core system and, in principle, that would allow Bao to run a dual-guest configuration, the truth is one should look at that platform as two single-core systems consolidated. This is because each core is completely separate from each other and has its own separate GIC. Therefore Bao is not fully equipped to take advantage of this board at the moment. We have ran Bao on only one of the cores and used a WIP version of Bao that allows multiple guests to run on a single CPU.

To solve this (because this kind of configuration seems to be present in other R52-based systems) we are looking at ways to configure Bao as a multikernel and deploy multiple Bao instances aware of each other in the same system. However this is still really an idea. The most straightforward way to use Bao (as it is) in this platform, would be to deploy two Bao instances (one in each core) with different platform description configurations that would partition the available memory a priori between the two. This would limit some Bao functionality. For example, communication between the guests would not work. It would be possible to make some ad hoc modifications for it to work.

At the moment we are not actively working on this. But let me know if you are interested in exploring this yourself. We will be more than happy to provide any assistance.

[pmittireddy]

Hi
Actually running bao onto a single R52 core is more relevant to my project. Kindly show a pointer to implementation that is done ( allowing multiple guests to run on a single CPU)

[pmittireddy]

Hi @josecm and bao-hypervisor team, I am currently working on a project that involves adding a runtime monitoring service that takes predicted actions in case of fault occurrences. The idea is to ensure fault-tolerance by utilizing the Bao hypervisor, creating two virtual machines (VMs) where the actual RTOS software will run on one VM, and a bare-metal monitor will run in another VM. The hardware platform I am targeting for this implementation is having ARM Cortex-R52. Given your experience, I would appreciate your guidance on how to approach this project, particularly if you have worked with this hardware before. Additionally, if you have any relevant implementation code or documentation, I would be grateful if you could share them, as they would greatly assist in the development process . I also appreciate the offer to share the fork of the Bao that allows multiple guests to run on a single CPU on renesas RZ/T2M, as that aligns with my current project objectives. Thank you for your time and support.

[AfonsoSantos96]

Hi, @pmittireddy! Sorry for not getting back to you sooner.

The work implemented on the RZ/T2M platform is on this fork:
Bao scheduler on Renesas RZ/T2M

A config file was added to the configs folder to help you compile Bao with 2 VMs.
In that config file, each VM has one UART associated with it, and the scheduler uses the generic timer to trigger the context switch.
The platform has two different UARTs. The ports and how to connect to them are described in the rzt2m_desc.c file.

Please note that this implementation on the Renesas board (which we later stopped using due to the reasons that @josecm previously told) still needs more optimizations on the platform drivers. As such, the UART using the USB port (UART0) has a glitch in that it only starts writing when multiple prints are done. For this reason, we usually used the UART3 (through an FTDI), which is functioning perfectly. Also, the baud rate is fixed at 19200.
If we fix both of these things on the driver in the future, we will update the fork.

If you have difficulty getting Bao to run multiple VMs, feel free to ask for help!

[pmittireddy]

Hi @AfonsoSantos96 , Thanks for sharing the work implemented on RZ/T2M and the detailed update! I cloned the fork repository [Bao scheduler on Renesas RZ/T2M<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_AfonsoSantos96_bao-2Dhypervisor_tree_exp_sched&d=DwMFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=UszTRzBdebBAgkRzX3RelxP_-cdTF-xEtDtESyol-kw&m=eEvBXJ3D0rjvbNJ76sFma8K9n7jJJ3qMMB8J022ctrNIn1vrJRoYA8-8HOnSesUJ&s=t5Rj7fspITx09kp6YHvD71I2RBWstmkJGR_izMMg3JM&e=>] i used this make command : make PLATFORM=rzt2m CONFIG=rzt2m I was getting the error: Makefile:50: /home/test/bao-hypervisor/ci/ci.mk: No such file or directory make: *** No rule to make target '/home/test/bao-hypervisor/ci/ci.mk'. Stop. Could you please assist me in resolving this issue? Thanks again for the support!

[josecm]

You just need to update the subrepo. That snapshot of Bao still makes you clone the CI subrepo even if you don't use it. Just run:

git submodule update --init

[pmittireddy]

Hi @AfonsoSantos96 and @josecm, I really appreciate the help you have extended so far. Thanks!
Hope you can help me tackle some more issues as well…
Right now, my objective is to run two baremetal guest VMs in the bao implementation that you had pointed to, for prototyping.
But I am facing issues with compiling the bao-baremetal-guest repository for the RZ/T2M platform. I copied the platform description (rzt2m directory) and the UART driver (rzt2m_uart directory) into the 'platform' and 'drivers' directories of the bao-baremetal-guest repository (master branch), respectively. But I am facing a lot of 'include' errors, missing file errors and missing macros among others, while trying to compile the baremetal guest.
Is there something more to be done than just this? Or is it that the master branch of the bao-baremetal-guest isn't compatible for the bao implementation for RZ/T2M?
2. In a general sense, can the guest VMs and the hypervisor be built independently? Or is it that the guest VM codebase has to be nested inside the hypervisor codebase before building? If yes, it would be great if you could also share the typical directory structure of the bao hypervisor project for this scenario.
I understand that this is a very lengthy question, but your answers would prove to be very helpful and insightful.
Thanks again for the support

[josecm]

it that the master branch of the bao-baremetal-guest isn't compatible for the bao implementation for RZ/T2M?

It should be pretty straightforward to add support for that platform on the baremetal guest. @AfonsoSantos96 do you have any fork of that repo with rzt2m support?

In a general sense, can the guest VMs and the hypervisor be built independently?

Guest must be built independently of the hypervisor. Bao has no notion of the guests. For Bao the guests are just a binary blob that is included in the configuration.

[AfonsoSantos96]

Hi @pmittireddy!

Sorry for the delay in responding. I've been sick these past few days, preventing me from reaching out earlier.
Nevertheless, I've been working on providing a fully functional baremetal. However, I've been having difficulties in getting EL1 IRQs to work correctly (IRQs within the guest). Despite that, the uart and generic timer IRQs to the EL2 (used by the scheduler) are functioning correctly.

I'll keep working on fixing this problem and will eventually notify you about the commit I'll make to fix it.
Meanwhile, here is the RZT2M baremetal guest so you can test it on the platform.
For the baremetal to configure the uart, it has to access the System Control peripheral region (SYSC_NS). So please add on the configs/rzt2m/config.c file the following device instance on the .devs struct:

{   
    .pa = 0x80281000,
    .va = 0x80281000,
    .size = 0x1A14,
},

In a general sense, can the guest VMs and the hypervisor be built independently?

Yes, you can build both independently. As "independently", I understand that you are saying that the VM image is not included in the binary of Bao. However, by doing this, you have to send the image of the guest to the platform memory. Please check lines 42 and 43 of config.c of configs/rzt2m.
There is the macro function to be used in that case, the VM_IMAGE_LOADED. This macro replaces the .image structure on the lines 44 to 52.

Thank you for your patience, and for further assistance, feel free to ask!