Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to run ubuntu guest + freeRTOS guest on bao? #27

Open
wusnga8dn opened this issue May 7, 2023 · 8 comments
Open

How to run ubuntu guest + freeRTOS guest on bao? #27

wusnga8dn opened this issue May 7, 2023 · 8 comments
Assignees
@ninolomata
Labels
feature-request New feature or request

Comments

@wusnga8dn
Copy link

Hello, I want to run the combination of ubuntu + freeRTOS guest on bao, may I ask if you have done related configuration work, and how should I do it?
@D3boker1
Copy link
Contributor

D3boker1 commented May 7, 2023

Hello!

Yes, we have done a similar configuration using Raspberry Pi OS (previously called Raspbian). The steps required should be quite similar if you are using a custom Linux image, as in the demos. The process involves providing the necessary configuration required by the guest OS.

For instance, if you need to use the Ethernet port on the guest OS, you'll need to give the guest access to the Ethernet device in its configuration. Similarly, if you plan to connect a monitor to the board, you'll need to give the guest access to the HDMI device.

A similar approach must be taken for both operating systems (Linux and freeRTOS), as you can see in our demos. For example, please refer to the Linux+freeRTOS configuration for the Raspberry Pi 4B available here

@wusnga8dn
Copy link
Author

你好！

是的，我们已经使用 Raspberry Pi OS（以前称为 Raspbian）完成了类似的配置。如果您使用的是自定义 Linux 映像，则所需的步骤应该与演示中的步骤非常相似。该过程涉及提供来宾操作系统所需的必要配置。

例如，如果您需要在来宾操作系统上使用以太网端口，则需要在其配置中授予来宾访问以太网设备的权限。同样，如果您计划将显示器连接到开发板上，则需要授予来宾访问 HDMI 设备的权限。

正如您在我们的演示中所见，必须对两种操作系统（Linux 和 freeRTOS）采取类似的方法。例如，请参考此处提供的 Raspberry Pi 4B 的 Linux+freeRTOS 配置

I'm very sorry that I don't understand such a configuration. If it is convenient, can you send your configuration file and configuration process? (Similar to the operation of bao-demo step-by-step guide) My basic configuration is similar to the example you described, and I need hdmi to display the Raspberry Pi and configure Ethernet. thank you very much for your help

@D3boker1
Copy link
Contributor

D3boker1 commented May 8, 2023

The configuration is straightforward, and we highly recommend that you take a look at the configuration example provided in the Bao repository at https://github.com/bao-project/bao-hypervisor/blob/main/configs/example/config.c
This should give you the basic knowledge to adapt one of our configuration files to meet your needs. To do so, you will need mainly to add your entries to devs and dev_num. I am attaching a configuration file that includes a Linux guest and a FreeRTOS guest for the RPI4b. This configuration file provides more extensive coverage than the demos.

The Linux guest is using:

  • 2 Cores
  • UART
  • HDMI
  • USB
  • 1 GPIO Port

The FreeRTOS is using:

  • 2 Cores
  • 1 GPIO Port
  • PWM0
  • UART2
  • GPIO Clock for the PWM
  • I2C1

The guests have shared memory at address 0x70000000 for communication.

#include <config.h>

VM_IMAGE(linux_image, "path/to/Linux/image.bin");
VM_IMAGE(freertos_image, "path/to/freeRTOS/image.bin");

struct config config = {
    
    CONFIG_HEADER
    
    .shmemlist_size = 1,
    .shmemlist = (struct shmem[]) {
        [0] = { .size = 0x00010000 }
    },
    
    .vmlist_size = 2,
    .vmlist = {
        { /** Linux */
            .image = {
                .base_addr = 0x10000000,
                .load_addr = VM_IMAGE_OFFSET(linux_image),
                .size = VM_IMAGE_SIZE(linux_image)
            },

            .entry = 0x10000000,

            .platform = {
                .cpu_num = 2,
                
                .region_num = 1,
                .regions =  (struct mem_region[]) {
                    {
                        .base = 0x10000000,
                        .size = 0x30000000,
                        .place_phys = true,
                        .phys = 0x10000000
                    }
                },
                
                .ipc_num = 1,
                .ipcs = (struct ipc[]) {
                    {
                        .base = 0x70000000,
                        .size = 0x00010000,
                        .shmem_id = 0
                    }
                },
                
                .dev_num = 11,
                .devs =  (struct dev_region[]) {
                    {   
                        /* uarta */
                        .pa = 0xfe215000,
                        .va = 0xfe215000,
                        .size = 0x1000,
                        .interrupt_num = 1,
                        .interrupts = (uint64_t[]) {125}                        
                    },
                    {
                        .pa = 0xfd580000,
                        .va = 0xfd580000,
                        .size = 0x10000,
                        .interrupt_num = 2,
                        .interrupts = (uint64_t[]) {189, 190}  
                    },
                    {   
                        /* Arch timer interrupt */
                        .interrupt_num = 1,
                        .interrupts = 
                            (uint64_t[]) {27}                         
                    },
                    /** HDMI */
                    {
                        /*HDMI 0 and 1, ddc 0 and 1, DVP*/
                        .va = 0XFEF00000,
                        .pa = 0XFEF00000,
                        .size = 0x10000,
                        .interrupt_num = 1,
                        .interrupts = (uint64_t[]) {128}
                    },
                    {
                        /*dma*/
                        .va = 0xFE007000,
                        .pa = 0xFE007000,
                        .size = 0X1000,
                        .interrupt_num = 9,
                        .interrupts = (uint64_t[]) {112, 113, 114, 115, 116, 117, 118, 119, 120}
                    },
                    {
                        /*Mailbox Can go out. HDMI its not using it*/
                        .va = 0XFE00B000,
                        .pa = 0XFE00B000,
                        .size = 0x1000,
                        .interrupt_num = 1,
                        .interrupts = (uint64_t[]) {65}
                    },
                    {
                        /* GPIO */
                        .va = 0XFE200000,
                        .pa = 0XFE200000,
                        .size = 0x1000
                    },
                    {   
                        /** cprman */  
                        .pa = 0xfe101000,
                        .va = 0xfe101000,
                        .size = 0x2000,                     
                    },
                    {
                        /* pcie */
                        .pa = 0xfd500000,
                        .va = 0xfd500000,
                        .size = 0x10000,
                        .interrupt_num = 1,
                        .interrupts = (uint64_t[]) {180}                          
                    },
                    {
                        /* pcie  address space */
                        .pa = 0x600000000,
                        .va = 0x600000000,
                        .size = 0x200000000,                    
                    },
                    {
                        /* usb */
                        .pa = 0xfe980000,
                        .va = 0xfe980000,
                        .size = 0x10000,
                        .interrupt_num = 2,
                        .interrupts = (uint64_t[]) {72, 105}                          
                    },
                },

                .arch = {
                    .gic = {
                        .gicd_addr = 0xff841000,
                        .gicc_addr = 0xff842000,
                    }
                }
            },
        },
        { /** FreeRTOS*/
            .image = {
                .base_addr = 0x20000000,
                .load_addr = VM_IMAGE_OFFSET(freertos_image),
                .size = VM_IMAGE_SIZE(freertos_image)
            },

            .entry = 0x20001788,
            .cpu_affinity = 0b0001,

            .platform = {
                .cpu_num = 1,
                
                .region_num = 1,
                .regions =  (struct mem_region[]) {
                    {
                        .base = 0x20000000,
                        .size = 0x4000000 
                    }
                },
                
                .ipc_num = 1,
                .ipcs = (struct ipc[]) {
                    {
                        .base = 0x70000000,
                        .size = 0x00010000,
                        .shmem_id = 0
                    }
                },

                .dev_num = 6,
                .devs =  (struct dev_region[]) {
                    {   
                        // UART2 
                        .pa = 0xFE201000,
                        .va = 0xFE201000,
                        .size = 0x1000,
                        .interrupt_num = 1,
                        .interrupts = (uint64_t[]) {153}                        
                    },
                    {
                        /*GPIO*/
                        .pa = 0xFE200000,
                        .va = 0xFE200000,
                        .size = 0x1000,
                        .interrupt_num = 1,
                        .interrupts = (uint64_t[]) {145}
                    },
                    {
                        /*PWM0*/
                        .pa = 0xFE20C000,
                        .va = 0xFE20C000,
                        .size = 0x1000,
                    },
                    {
                        /*GPIO Clock: this is a GPIO clock for the PWM device*/
                        .pa = 0xFE101000,
                        .va = 0xFE101000,
                        .size = 0x1000,
                    },
                    {
                        /*I2C device 1*/
                        .pa = 0xFE804000,
                        .va = 0xFE804000,
                        .size = 0x1000,
                    },
                    {   
                        /* Arch timer interrupt */
                        .interrupt_num = 1,
                        .interrupts = 
                            (uint64_t[]) {27}                         
                    }
                },
                
                .arch = {
                    .gic = {
                        .gicd_addr = 0xFF841000,
                        .gicc_addr = 0xFF842000,
                    }
                }
            },
        }
    },
};

@wusnga8dn
Copy link
Author

The configuration is straightforward, and we highly recommend that you take a look at the configuration example provided in the Bao repository at https://github.com/bao-project/bao-hypervisor/blob/main/configs/example/config.c This should give you the basic knowledge to adapt one of our configuration files to meet your needs. To do so, you will need mainly to add your entries to devs and dev_num. I am attaching a configuration file that includes a Linux guest and a FreeRTOS guest for the RPI4b. This configuration file provides more extensive coverage than the demos.

The Linux guest is using:

  • 2 Cores
  • UART
  • HDMI
  • USB
  • 1 GPIO Port

The FreeRTOS is using:

  • 2 Cores
  • 1 GPIO Port
  • PWM0
  • UART2
  • GPIO Clock for the PWM
  • I2C1

The guests have shared memory at address 0x70000000 for communication.

#include <config.h>

VM_IMAGE(linux_image, "path/to/Linux/image.bin");
VM_IMAGE(freertos_image, "path/to/freeRTOS/image.bin");

struct config config = {
    
    CONFIG_HEADER
    
    .shmemlist_size = 1,
    .shmemlist = (struct shmem[]) {
        [0] = { .size = 0x00010000 }
    },
    
    .vmlist_size = 2,
    .vmlist = {
        { /** Linux */
            .image = {
                .base_addr = 0x10000000,
                .load_addr = VM_IMAGE_OFFSET(linux_image),
                .size = VM_IMAGE_SIZE(linux_image)
            },

            .entry = 0x10000000,

            .platform = {
                .cpu_num = 2,
                
                .region_num = 1,
                .regions =  (struct mem_region[]) {
                    {
                        .base = 0x10000000,
                        .size = 0x30000000,
                        .place_phys = true,
                        .phys = 0x10000000
                    }
                },
                
                .ipc_num = 1,
                .ipcs = (struct ipc[]) {
                    {
                        .base = 0x70000000,
                        .size = 0x00010000,
                        .shmem_id = 0
                    }
                },
                
                .dev_num = 11,
                .devs =  (struct dev_region[]) {
                    {   
                        /* uarta */
                        .pa = 0xfe215000,
                        .va = 0xfe215000,
                        .size = 0x1000,
                        .interrupt_num = 1,
                        .interrupts = (uint64_t[]) {125}                        
                    },
                    {
                        .pa = 0xfd580000,
                        .va = 0xfd580000,
                        .size = 0x10000,
                        .interrupt_num = 2,
                        .interrupts = (uint64_t[]) {189, 190}  
                    },
                    {   
                        /* Arch timer interrupt */
                        .interrupt_num = 1,
                        .interrupts = 
                            (uint64_t[]) {27}                         
                    },
                    /** HDMI */
                    {
                        /*HDMI 0 and 1, ddc 0 and 1, DVP*/
                        .va = 0XFEF00000,
                        .pa = 0XFEF00000,
                        .size = 0x10000,
                        .interrupt_num = 1,
                        .interrupts = (uint64_t[]) {128}
                    },
                    {
                        /*dma*/
                        .va = 0xFE007000,
                        .pa = 0xFE007000,
                        .size = 0X1000,
                        .interrupt_num = 9,
                        .interrupts = (uint64_t[]) {112, 113, 114, 115, 116, 117, 118, 119, 120}
                    },
                    {
                        /*Mailbox Can go out. HDMI its not using it*/
                        .va = 0XFE00B000,
                        .pa = 0XFE00B000,
                        .size = 0x1000,
                        .interrupt_num = 1,
                        .interrupts = (uint64_t[]) {65}
                    },
                    {
                        /* GPIO */
                        .va = 0XFE200000,
                        .pa = 0XFE200000,
                        .size = 0x1000
                    },
                    {   
                        /** cprman */  
                        .pa = 0xfe101000,
                        .va = 0xfe101000,
                        .size = 0x2000,                     
                    },
                    {
                        /* pcie */
                        .pa = 0xfd500000,
                        .va = 0xfd500000,
                        .size = 0x10000,
                        .interrupt_num = 1,
                        .interrupts = (uint64_t[]) {180}                          
                    },
                    {
                        /* pcie  address space */
                        .pa = 0x600000000,
                        .va = 0x600000000,
                        .size = 0x200000000,                    
                    },
                    {
                        /* usb */
                        .pa = 0xfe980000,
                        .va = 0xfe980000,
                        .size = 0x10000,
                        .interrupt_num = 2,
                        .interrupts = (uint64_t[]) {72, 105}                          
                    },
                },

                .arch = {
                    .gic = {
                        .gicd_addr = 0xff841000,
                        .gicc_addr = 0xff842000,
                    }
                }
            },
        },
        { /** FreeRTOS*/
            .image = {
                .base_addr = 0x20000000,
                .load_addr = VM_IMAGE_OFFSET(freertos_image),
                .size = VM_IMAGE_SIZE(freertos_image)
            },

            .entry = 0x20001788,
            .cpu_affinity = 0b0001,

            .platform = {
                .cpu_num = 1,
                
                .region_num = 1,
                .regions =  (struct mem_region[]) {
                    {
                        .base = 0x20000000,
                        .size = 0x4000000 
                    }
                },
                
                .ipc_num = 1,
                .ipcs = (struct ipc[]) {
                    {
                        .base = 0x70000000,
                        .size = 0x00010000,
                        .shmem_id = 0
                    }
                },

                .dev_num = 6,
                .devs =  (struct dev_region[]) {
                    {   
                        // UART2 
                        .pa = 0xFE201000,
                        .va = 0xFE201000,
                        .size = 0x1000,
                        .interrupt_num = 1,
                        .interrupts = (uint64_t[]) {153}                        
                    },
                    {
                        /*GPIO*/
                        .pa = 0xFE200000,
                        .va = 0xFE200000,
                        .size = 0x1000,
                        .interrupt_num = 1,
                        .interrupts = (uint64_t[]) {145}
                    },
                    {
                        /*PWM0*/
                        .pa = 0xFE20C000,
                        .va = 0xFE20C000,
                        .size = 0x1000,
                    },
                    {
                        /*GPIO Clock: this is a GPIO clock for the PWM device*/
                        .pa = 0xFE101000,
                        .va = 0xFE101000,
                        .size = 0x1000,
                    },
                    {
                        /*I2C device 1*/
                        .pa = 0xFE804000,
                        .va = 0xFE804000,
                        .size = 0x1000,
                    },
                    {   
                        /* Arch timer interrupt */
                        .interrupt_num = 1,
                        .interrupts = 
                            (uint64_t[]) {27}                         
                    }
                },
                
                .arch = {
                    .gic = {
                        .gicd_addr = 0xFF841000,
                        .gicc_addr = 0xFF842000,
                    }
                }
            },
        }
    },
};

Thanks a lot for your help, here I have two questions:
First of all, I am going to directly try the configuration file you gave and make a debian linux image. How should I change it in the following steps?
Second, I am going to modify this part of the content after testing the configuration file you gave me, but I don’t know much about the settings of the physical address and virtual address inside. How is this part calculated? Is there any documentation available? refer to?
Linux Guest

  1. Download the Linux kernel source
    Setup linux environment variables. Start by the demo resource directory:

export BAO_DEMOS_LINUX=$BAO_DEMOS/guests/linux
Setup repo and version. Specifically for the NXP iMX platforms use:

export BAO_DEMOS_LINUX_REPO=https://source.codeaurora.org/external/imx/linux-imx
export BAO_DEMOS_LINUX_VERSION=rel_imx_5.4.24_2.1.0
For all other platforms clone the latest mainline Linux release:

export BAO_DEMOS_LINUX_REPO=https://github.com/torvalds/linux.git
export BAO_DEMOS_LINUX_VERSION=v6.1
Setup an environment variable pointing to Linux's source code:

export BAO_DEMOS_LINUX_SRC=$BAO_DEMOS_WRKDIR_SRC/linux-$BAO_DEMOS_LINUX_VERSION
And make a shallow clone of the target repo:

git clone $BAO_DEMOS_LINUX_REPO $BAO_DEMOS_LINUX_SRC
--depth 1 --branch $BAO_DEMOS_LINUX_VERSION
cd $BAO_DEMOS_LINUX_SRC
git apply $BAO_DEMOS_LINUX/patches/$BAO_DEMOS_LINUX_VERSION/*.patch
Finally, setup and environment variable pointing to the target architecture and platform specific config to be used by buildroot:

export BAO_DEMOS_LINUX_CFG_FRAG=$(ls $BAO_DEMOS_LINUX/configs/base.config
$BAO_DEMOS_LINUX/configs/$ARCH.config
$BAO_DEMOS_LINUX/configs/$PLATFORM.config 2> /dev/null)
Use Buildroot to build Linux with a built-in initramfs
Setup buildroot environment variables:

export BAO_DEMOS_BUILDROOT=$BAO_DEMOS_WRKDIR_SRC/
buildroot-$ARCH-$BAO_DEMOS_LINUX_VERSION
export BAO_DEMOS_BUILDROOT_DEFCFG=$BAO_DEMOS_LINUX/buildroot/$ARCH.config
export LINUX_OVERRIDE_SRCDIR=$BAO_DEMOS_LINUX_SRC
Clone the latest buildroot at the latest stable version:

git clone https://github.com/buildroot/buildroot.git $BAO_DEMOS_BUILDROOT
--depth 1 --branch 2022.11
cd $BAO_DEMOS_BUILDROOT
Use our provided buildroot defconfig, which itself points to the a Linux kernel defconfig and patches (mainly for the inter-vm communication drivers) and build:

make defconfig BR2_DEFCONFIG=$BAO_DEMOS_BUILDROOT_DEFCFG
make linux-reconfigure all
mv $BAO_DEMOS_BUILDROOT/output/images/Image
$BAO_DEMOS_BUILDROOT/output/images/Image-$PLATFORM
Build the device tree and wrap it with the kernel image
NOTE

If your target demo features multiple Linux virtual machines, you will have to repeat this last step for each of these, which should correspond to different .dts files in $BAO_DEMOS/$DEMO/devicetrees.

The device tree(s) for your target demo are available in $BAO_DEMOS/$DEMO/devicetrees/$PLATFORM. For a device tree file named linux.dts define a virtual machine variable:

export BAO_DEMO_LINUX_VM=linux
Then build:

dtc $BAO_DEMOS/demos/$DEMO/devicetrees/$PLATFORM/$BAO_DEMO_LINUX_VM.dts >
$BAO_DEMOS_WRKDIR_IMGS/$BAO_DEMO_LINUX_VM.dtb
Wrap the kernel image and device tree blob in a single binary:

make -C $BAO_DEMOS_LINUX/lloader
ARCH=$ARCH
IMAGE=$BAO_DEMOS_BUILDROOT/output/images/Image-$PLATFORM
DTB=$BAO_DEMOS_WRKDIR_IMGS/$BAO_DEMO_LINUX_VM.dtb
TARGET=$BAO_DEMOS_WRKDIR_IMGS/$BAO_DEMO_LINUX_VM

@D3boker1
Copy link
Contributor

Hi @wusnga8dn,

I believe it would be better if you took a step back and understood the configuration file for bao's guests. Note that our demos exist to provide users with a "plug and play" experience, allowing you to test Bao on your platform and ensure everything runs smoothly.

That being said, I noticed you are attempting to replace the Rasberry Linux kernel URL in buildroot. If you already have a Linux image (the Raspberry Pi OS), why are you trying to use buildroot?

Therefore, it would be best to perform the steps separately. Here's what I suggest:

  1. Get the Linux image (the RPIOS).
  2. Obtain the freeRTOS image.
  3. Adapt the configuration file I provided above by changing the path to your files, adding or removing devices as needed for the guests, and consulting the BCM2711 datasheet to obtain the devices addresses.

@wusnga8dn
Copy link
Author

2. freeRTOS

Thank you very much for your help, but I don't understand a little about the steps you mentioned. Here are my questions:

  1. How should I make the linux image of the Raspberry Pi system so that it can be successfully loaded by bao like the linux in the demo? I am currently trying to make a kernel image through the steps of compiling the kernel provided on the Raspberry Pi official website, and use it to replace the linux image in the demo. Is this possible?
  2. For the freeRTOS image, what I think of is to make it according to the method given in the demo. I hope to change the steps in the demo as little as possible to obtain the Linux + freeRTOS image I want

@josecm josecm added the feature-request New feature or request label Jun 28, 2023
@josecm
Copy link
Member

josecm commented Jun 28, 2023

@wusnga8dn We'll be working on adding a demo featuring a full Ubuntu image to some of the platforms. For that reason, I've marked this as a feature request.

@josecm josecm mentioned this issue Jun 28, 2023
Closed
@josecm
Copy link
Member

josecm commented Sep 15, 2023 •
edited
Loading

@ninolomata is looking into this. Hoping to have some news soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ninolomata ninolomata

Labels
feature-request New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@josecm @ninolomata @D3boker1 @wusnga8dn