Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(doc): add guest configuration to bao-classic config documentation #78

Open
wants to merge 70 commits into
base: wip/bao-classic_config
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
746823c
feat(doc): add guest configuration introduction
Diogo21Costa Dec 7, 2023
407c8dd
feat(doc): add guest image configuration
Diogo21Costa Dec 7, 2023
c92f56b
feat(doc): add VM configuration - introduction
Diogo21Costa Jul 29, 2023
413b37b
feat(doc): add VM configuration - number of vCPUs
Diogo21Costa Jul 29, 2023
b6db67a
feat(doc): add VM configuration - memory mapping
Diogo21Costa Jul 29, 2023
da5b673
feat(doc): add VM configuration - IPC
Diogo21Costa Jul 29, 2023
b78ee89
feat(doc): add VM configuration - devices
Diogo21Costa Jul 29, 2023
c22ba39
feat(doc): add VM configuration - memory management
Diogo21Costa Dec 7, 2023
3d43ffb
feat(doc): add VM configuration - architectural-specific configurations
Diogo21Costa Jul 29, 2023
1300f5a
feat(doc): add cpu affinity
Diogo21Costa Jul 29, 2023
7415b8d
feat(doc): add cache coloring
Diogo21Costa Jul 29, 2023
77c4429
fix(doc): include format tweaks to use the new column limit of 100
Diogo21Costa Dec 7, 2023
223e053
fix(img): update cpu-affinity
Diogo21Costa Jan 5, 2024
b5096f0
fix(heading): update section naming
Diogo21Costa Dec 23, 2023
88b3e28
fix(vm_config): fix introduction text
Diogo21Costa Jan 5, 2024
fedaf73
fix(vm_config): update vm entry description
Diogo21Costa Dec 23, 2023
9d82a52
fix(vm_config): fix struct vm_image declaration
Diogo21Costa Jan 5, 2024
8e218ad
fix(guest_image): update size field description
Diogo21Costa Jan 5, 2024
4af39fd
fix(typo): improve Virtual Machine Configuration section
Diogo21Costa Jan 5, 2024
31f18ac
update(doc): refactor Virtual Machine Configuration description
Diogo21Costa Jan 5, 2024
0155757
update(VM Config): improve struct description
Diogo21Costa Jan 5, 2024
4aa86ec
update(doc): improve Virtual Machine Configuration description
Diogo21Costa Jan 5, 2024
d6c2e0a
update(n_cpus): add warning to number of vCPUs description
Diogo21Costa Jan 5, 2024
580d626
fix(vm_config): rename memory regions subsection
Diogo21Costa Jan 5, 2024
5f9b9ea
fix(vm_config): improve region_num description
Diogo21Costa Jan 5, 2024
9bf7c0c
update(mem_reg): include small entry
Diogo21Costa Jan 5, 2024
1ca6f2e
update(mem_reg): include important notes
Diogo21Costa Jan 5, 2024
1c373fe
update(mem_reg): include details on the use of place_phys
Diogo21Costa Jan 5, 2024
4b057cd
update(guest_img): update separately_loaded description
Diogo21Costa Jan 17, 2024
3264cd3
fix(vm_config): fix format
Diogo21Costa Jan 17, 2024
af0601c
fix(vm_config): fix format
Diogo21Costa Jan 17, 2024
ba5a87f
fix(guest_img): fix format
Diogo21Costa Jan 17, 2024
42c87ce
fix(guest_img): remove outdated image
Diogo21Costa Jan 17, 2024
987878a
update(guest_img): empashise the importance of using macros to configure
Diogo21Costa Jan 17, 2024
dd49512
update(guest_img): include macros use-cases examples
Diogo21Costa Jan 17, 2024
e9e1c43
update(mem_manag): fix struct declaration
Diogo21Costa Jan 17, 2024
431be7f
update(arch): fix typo
Diogo21Costa Jan 17, 2024
e501eb4
update(cache_col): update image
Diogo21Costa Jan 17, 2024
2bbd78f
update(cache_col): improve coloring description
Diogo21Costa Jan 19, 2024
404de5a
update(ipc): add note regarding to shmem size and base
Diogo21Costa Jan 19, 2024
854fe14
update(ipc): add overview of IPC workflow
Diogo21Costa Jan 19, 2024
ae5a5bd
update(ipc): include note regarding to base and size
Diogo21Costa Jan 19, 2024
c77efcb
fix(vm_config): improve readability
Diogo21Costa Mar 12, 2024
7ea8acb
fix(guest_image): improve readability
Diogo21Costa Mar 12, 2024
a899626
fix(vm_config): improve readability
Diogo21Costa Mar 12, 2024
e671c5b
ref(arch): remove mem. management section and split arch specific config
Diogo21Costa Mar 12, 2024
3594c13
fix(format): include format tweaks
Diogo21Costa Mar 12, 2024
df43831
fix(cpu_affinity): include improved description
Diogo21Costa Mar 12, 2024
7bfac37
fix(doc): improve readability
Diogo21Costa May 22, 2024
8a6069e
fix(vm_config): update image struct definition
Diogo21Costa Sep 4, 2024
00a28ff
fix(doc): improve readability
Diogo21Costa Sep 4, 2024
69735d9
fix(doc): remove content from memory regions section
Diogo21Costa Sep 4, 2024
e564d6e
fix(doc): redefine IPC nomenclature
Diogo21Costa Sep 4, 2024
696ae37
fix(doc): improve IPC section readability
Diogo21Costa Sep 4, 2024
a83bb96
fix(doc): fix typo
Diogo21Costa Sep 4, 2024
831e066
fix(doc): improve readability by introducing GA and PA terms
Diogo21Costa Sep 4, 2024
3558ec9
fix(doc): update device configuration note
Diogo21Costa Sep 4, 2024
d2efdcf
fix(doc): remove empty subsection
Diogo21Costa Sep 4, 2024
a139c99
fix(doc): update MPU VA and PA configuration description
Diogo21Costa Sep 4, 2024
8b6eb3b
feat(doc): add shared memory configuration
Diogo21Costa Jul 29, 2023
33652e5
fix(doc): improve shared memory documentation
Diogo21Costa Sep 17, 2024
b4d14d8
fix(doc): improve shared memory documentation
Diogo21Costa Sep 17, 2024
f6b8cde
fix(doc): improve shared memory documentation
Diogo21Costa Sep 17, 2024
07452d7
fix(doc): improve shared memory documentation
Diogo21Costa Sep 17, 2024
3f2a6f1
fix(doc): improve shared memory documentation
Diogo21Costa Sep 17, 2024
12a303c
fix(doc): fix guest image documentation
Diogo21Costa Sep 19, 2024
9ffd91e
fix(doc): fix VM config documentation
Diogo21Costa Sep 19, 2024
9744a25
fix(doc): fix shared memory documentation
Diogo21Costa Sep 19, 2024
b778726
fix(doc): fix devices documentation
Diogo21Costa Sep 19, 2024
0c360b5
fix(doc): add note regarding to MMIO mapping
Diogo21Costa Dec 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
244 changes: 244 additions & 0 deletions source/bao_hyp/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -68,36 +68,280 @@ relative to the config source file.
Guests Configuration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Guests Configuration
VM Configuration

maybe it is more appropriate...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in c4e19ed96a14cf7fe0af614d016e1dec60a51fb3.

--------------------

Within this configuration file, it is possible to define various aspects of your virtual
environment, including hardware resources, devices, and storage options. One of the most important
features of the configuration file is the ability to define VMs, or guests (VM 1, VM 2, ... VM N).
This allows to create and manage multiple VMs, each with its own operating system and hardware
resources.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this intro is too convoluted. Let's just summarize what can be done more straight to the point. For example:

Suggested change
Within this configuration file, it is possible to define various aspects of your virtual
environment, including hardware resources, devices, and storage options. One of the most important
features of the configuration file is the ability to define VMs, or guests (VM 1, VM 2, ... VM N).
This allows to create and manage multiple VMs, each with its own operating system and hardware
resources.
Bao's configuration file allows you to partition the platforms' hardware resources, such as CPU cores, memory, or devices, by assigning them one or more VMs. It also allows you to configure the guest image to run on that VM. Remember that in Bao resources are exclusively assigned to each VM. The as well as communication channels between two or more VMs in the more or shared memory.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in 93be85f1070b26f5c6757f2f814d989021e213b4.


.. figure:: img/guest-config.svg
:align: center
:name: guest-config-fig


The configuration of different guests is done by populating a struct called *vmconfig*, as follows:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe here we could add a reference the vmlist referenced in the first part of the document, saying that each entry in that list is a vm_config struct...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in 923bbca5719fda58025bfc192b7461efa1d1a1c4.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The configuration of different guests is done by populating a struct called *vmconfig*, as follows:
The configuration of different VMs is done by populating a struct called *vmconfig*, as follows:

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in 923bbca5719fda58025bfc192b7461efa1d1a1c4.


.. code-block:: c

struct vm_config {
struct image;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not correct. I know that the structure used for the image is an anonymous structure, but maybe we can change this is to:

Suggested change
struct image;
struct vm_image image;

(both here and in the code)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in fadc7a2bb90adc5f6d6fd4eebad36337f2748c39.

vaddr_t entry;
cpumap_t cpu_affinity;
colormap_t colors;
struct vm_platform platform;
};

For each VM, the following parameters must be specified:

- **image** [mandatory] - corresponds to the guest image (see details in `Guest Image`_)
- **entry** [mandatory] - defines the entry point address in VM's address space;
- **platform description** [mandatory] - corresponds to the guest platform (see details in `Virtual
Machine Configuration`_);
- **cpu_affinity** [optional] - corresponds to the selection of physical CPUs assigned to the
virtual platform (see details in `CPU Affinity`_);
- **colors** [optional] - enables the cache coloring feature (see details in `Coloring`_).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These definitions need to be more technical. E.g.,:

Suggested change
- **image** [mandatory] - corresponds to the guest image (see details in `Guest Image`_)
- **entry** [mandatory] - defines the entry point address in VM's address space;
- **platform description** [mandatory] - corresponds to the guest platform (see details in `Virtual
Machine Configuration`_);
- **cpu_affinity** [optional] - corresponds to the selection of physical CPUs assigned to the
virtual platform (see details in `CPU Affinity`_);
- **colors** [optional] - enables the cache coloring feature (see details in `Coloring`_).
- **image** [mandatory] - a structure containing information about guest image loading (see details in `Guest Image`_)
- **entry** [mandatory] - defines the entry point address in VM's address space;
- **platform description** [mandatory] - a description of the VM platform, defining its resource assignments and requirements (see details in `Virtual
Machine Configuration`_);
- **cpu_affinity** [optional] - defines the affinity of the VM's vCPUs to the physical CPUs assigned to the
virtual platform (see details in `CPU Affinity`_);
- **colors** [optional] - assignment of shared LLC cache colors (or partitions) to this VM (see details in `Coloring`_).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Fixed in 7ab7bf436542d6fd6626211577837d6ecc5452d1.


Guest Image
***********
.. _Guest Image:

The guest ``image`` comprises a structure that describes the image configuration running on the
guest side. It encompasses the following options:
Comment on lines +120 to +121
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More precisely, the structure contains information relevant to image loading and mapping.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also thinking that we should restructure this section. Start by explaining in depth the two options we have regarding loading images (builtin vs separately loaded). Then say that we have to macros for the two options. And then go to the internals of the struct to explain everything in detail.

Also this sentence should state that the use of VM_IMAGE is optional. And maybe we should move the details on VM_IMAGE from that initial section to this section and there only say the basic and refer this section.


- **image** [mandatory] - definition of the ``image`` to run on a given VM. The ``image``
danielRep marked this conversation as resolved.
Show resolved Hide resolved
corresponds to the following structure:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this here? I think this information is already in the previous section.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While this information may seem redundant, providing context about the image within the vm_config struct can be valuable for readers. Additionally, incorporating a pointer or link to review any necessary information could assist individuals engaging in a non-linear read. For instance, if someone specifically seeks the VM configuration section and bypasses the image definition, having this contextualization might be helpful. Should we remove this and keep it simpler (users still have the table of contents on the Read The Docs page)?


.. code-block:: c

struct image {
vaddr_t base_addr;
paddr_t load_addr;
size_t size;
bool separately_loaded;
bool inplace;
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
struct image {
vaddr_t base_addr;
paddr_t load_addr;
size_t size;
bool separately_loaded;
bool inplace;
};
struct vm_image {
vaddr_t base_addr;
paddr_t load_addr;
size_t size;
bool separately_loaded;
bool inplace;
} image;

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in fadc7a2bb90adc5f6d6fd4eebad36337f2748c39.


where:

- **base_addr** [mandatory] - corresponds to the ``image`` load address in the VM's address space;
- **load_addr** [mandatory] - corresponds to the ``image`` load address in the hypervisor address
space. This value can be defined using the macro VM_IMAGE_OFFSET(img_name);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **load_addr** [mandatory] - corresponds to the ``image`` load address in the hypervisor address
space. This value can be defined using the macro VM_IMAGE_OFFSET(img_name);
- **load_addr** [mandatory] - corresponds to the ``image`` load address in physical memory/physical address space. This value can be defined using the macro VM_IMAGE_OFFSET(img_name);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, we need to explain that, if separately loaded is set to false, the hypervisor initially interprets this as the offset of the builtin image guest in its own image (hence, VM_IMAGE_OFFSET). At runtime, it fixes the value to be later interpreted as a physical address, by adding to it the address the hypervisor was loaded.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in 4b057cd.

- **size** [mandatory] - corresponds to the image size. This value can be defined using the macro
VM_IMAGE_SIZE(img_name);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **size** [mandatory] - corresponds to the image size. This value can be defined using the macro
VM_IMAGE_SIZE(img_name);
- **size** [mandatory] - corresponds to the image size. For builtin images declared using `VM_IMAGE`, this value can be defined using the macro VM_IMAGE_SIZE(img_name);

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in dde0f11fddf139eee0aa33223d587b912c60aecd.

- **separately_loaded** [optional] - informs the hypervisor if the VM image is to be loaded
separately by a bootloader; By default, separately_loaded is set as false;
- **inplace** [optional]- use the image inplace and don’t copy the image. By default, inplace is
set as false;

.. figure:: img/vm-image.svg
:align: center
:name: vm-image-fig

To ease the process of configuring the image running on each VM, the configuration of Bao allows
the use of two different macros:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here you should stress that the user should use these macros.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in 987878a.


1. **VM_IMAGE_BUILTIN** - This macro simplifies image configuration by requiring only the
``img_name`` and the image ``base_addr``. This macro specifies both the base address and image
size.

2. **VM_IMAGE_LOADED** - This macro requires additional configurations. It requires the definition
of image ``base_addr``, the image ``load_addr``, and the image ``size``.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An example usage of the macros would be useful.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, explain why one might use one or the other.

BUILTIN makes it simple to configure the system as Bao knows where the image is by default and there is no need to separately configure and loaded the guest images using a bootloader or adjust the size of the guest images change.

LOADED is best and should be used for MPU systems as it is difficult to manually allocate the space for the guest image if it is embedded in Bao's binary. Otherwise, Bao will need to copy the image, which might wast space.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, addressed in dd49512.


Virtual Machine Configuration
*****************************

The VM configuration enables users to define the characteristics of each the virtualized platform.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The VM configuration enables users to define the characteristics of each the virtualized platform.
The VM configuration enables users to define the characteristics of each virtualized platform.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in 27f6cb74ebdae864bb7417b908a9de0f35853215.

It encompasses critical details that define the VM's run-time environment, performance, and overall
capabilities. By customizing this configuration, users can tailor the virtual platform to suit
specific workload requirements and application needs for their virtual machines. The configuration
encompasses the definition of:

- **Number of virtual CPUs** - see details in `Number of vCPUs`_;
- **Virtual memory regions** - see details in `Memory Mapping`_;`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Id prefer just "Number of CPUs" and "Memory Regions"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in 50d0eec13ce3fa0bf1c39a368e078985a0135d19.

- **Inter-Process Comunication (IPC)** - see details in `Inter-Process Communication (IPC)`_;
- **Devices** - see details in `Devices`_;
- **Memory Management** - see details in `Memory Management`_;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd remove this. It's too architecture specific (see below).

- **Architectural-Specific Configurations** - see details in `Architectural-Specific
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not put the definition of struct vm_platform as previously done for the vm config and vm image?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fully agree, refactored this in 3fec42951b45f15f0dad7da83f26a5fc75f07994.

Configurations`_;

1. Number of vCPUs
##################
.. _Number of vCPUs:

- **cpu_num** [mandatory] - defines the number of CPUs assigned to the VM;
danielRep marked this conversation as resolved.
Show resolved Hide resolved

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add a warning here: the sum of the number of cpus assigned across all vms in the vmlist must be less or equal, to the number of cpus available on the platform. The guest might just not boot without warning if this is not the case (maybe we should add this as an issue on Bao and just add the ERROR 🤔 )

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, fixed in a0259bc0ac7777e5bd1c04970f47d6ef1a52a9f2.


2. Memory Mapping
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
2. Memory Mapping
2. Memory Regions

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in b3489c1663dcc54035af65c2b1aba0165b852434.

#################
.. _Memory Mapping:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should add a small entry saying that Bao uses two-stage translation to control VM memory. Thus each VM memory region is defined by its virtual address (of the second stage). Also that Bao will allocate the physical memory for the region, but optionally, one can defined the physical memory region where this region will be mapped.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, addressed in 1b77d8a1ea233e229e576cb103ea2fc5e3cbd45a.


- **region_num** [mandatory] - defines the number of memory regions mapped to the VM. This
structure contemplates the following parameters:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to be more precise! Something like:

Suggested change
- **region_num** [mandatory] - defines the number of memory regions mapped to the VM. This
structure contemplates the following parameters:
- **region_num** [mandatory] - defines the number of memory regions in the VM, specifically, the number of `vm_mem_region` entries in the `vm_platform`'s `regions` list.
The `vm_mem_region` structure contemplates the following parameters:

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in f8011d97198f8b3d9f6a4683c582bd316dd9e5c1.


.. code-block:: c

struct vm_mem_region {
paddr_t base;
size_t size;
bool place_phys;
paddr_t phys;
};

where:

- **base** [mandatory] - corresponds to the base virtual address of the memory region;
- **size** [mandatory] - corresponds to the size of the memory region;
- **place_phys** [optional] - the memory region is mapped into the virtual memory, and it's
important to note that the virtual address (VA) might not necessarily be the same as the physical
address (PA). When "place_phys" is set to true, the virtual address corresponds to the physical
address. If ``place_phys`` equals to true, it allows to specify the physical address of the
memory region. By default, ``place_phys`` equals to false;
- **phys** [mandatory if ``place_phys`` is true] - it corresponds to the physical address where the
memory region should be mapped;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could add a few notes here:

  1. Explain that it is mandatory for base and size to be aligned to the architecture's smallest page size: in all architectures 4K for MMU systems and 64 bytes for MPU.
  2. For performance reasons, for MMU targets, memory base and size should be aligned as much as possible to the target architectures' huge pages (2 MiB is a good target for both Arm and RISCV). Also for phys if place_phys is enabled.
  3. For MMU systems, identity mappings can be achieved by setting phys to the same value as base.
  4. In MPU systems, place_phys and phys are ignored.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also we need to explain that by using place_phys/phys the user is essentially manually allocating memory. And this is a way to obtain physical mappings.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good points, thank you! Added in c5923099c8b4ab784b4bad4c305a0668063606e3 and in 9e61aa497068aead1e3aa1c1f492a34c39f0c275.

3. Inter-Process Communication (IPC)
####################################
.. _Inter-Process Communication (IPC):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to explain somewhere how the IPC works 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, included new content in 854fe14.

DavidMCerdeira marked this conversation as resolved.
Show resolved Hide resolved

- **ipc_num** [optional] - defines the number of IPCs assigned to the VM. By default, ``ipc_num``
equals to zero;
- **ipcs** [mandatory if ``ipc_num`` > 0] - corresponds to the specification of the IPC and is
configured through the following structure:

.. code-block:: c

struct ipc {
paddr_t base;
size_t size;
size_t shmem_id;
size_t interrupt_num;
irqid_t *interrupts;
};


where:

- **base** [mandatory] - corresponds to the virtual base address of the IPC memory region;
- **size** [mandatory] - corresponds to the size of the IPC memory region;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this must be less or equal to the size of the shared memory size.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, for MPU systems, base is ignored (the region address is the shared memory object address)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, addressed in 404de5a.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, it is mandatory for base and size to be aligned to the architecture's smallest page size: in all architectures 4K for MMU systems and 64 bytes for MPU.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanls, fixed in 404de5a.

- **shmem_id** [mandatory] - corresponds to the ID of the shared memory associated with the IPC;
- **interrupt_num** [mandatory] - defines the number of interrupts assigned to the IPC;
- **interrupts** [mandatory if *interrupt_num* > 0] - defines a list of interrupt IDs assigned to
the IPC - ``(irqid_t[]) {irq_1, ..., irq_n}``;

.. warning::
Specifying a number of interrupts in the ``interrupts`` buffer that differs from the
``interrupt_num`` may result in unforeseen behavior.

4. Devices
##########
.. _Devices:

- **dev_num** [mandatory] - corresponds to the number of devices assigned to the VM;
- **devs** [mandatory if *dev_num* > 0] - corresponds to the specification of the VM's devices and
is configured through the following structure:
danielRep marked this conversation as resolved.
Show resolved Hide resolved

.. code-block:: c

struct vm_dev_region {
paddr_t pa;
vaddr_t va;
DavidMCerdeira marked this conversation as resolved.
Show resolved Hide resolved
size_t size;
size_t interrupt_num;
irqid_t *interrupts;
streamid_t id; /* bus master id for iommu effects */
};

where:

- **pa** [mandatory] - corresponds to the base physical address of the device;
- **va** [mandatory] - corresponds to the base virtual address of the device;
- **size** [mandatory] - corresponds to the size of the device memory region;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, explain that it is mandatory for base and size to be aligned to the architecture's smallest page size: in all architectures 4K for MMU systems and 64 bytes for MPU.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, addressed in ae5a5bd.

- **interrupt_num** [optional] - corresponds to the number of interrupts generated by the device to
the VM. By default, ``interrupt_num`` equals to 0;
- **interrupts** [mandatory if *interrupt_num*>0] - defines a list of interrupt IDs generated by
the device - ``(irqid_t[]) {irq_1, ..., irq_n};``
- **id** [optional] - corresponds to the bus master id for iommu effects:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which should also add more explanation to this, possibly on a per-platform basis.


.. warning::
Specifying a number of interrupts in the ``interrupts`` buffer that differs from the
``interrupt_num`` may result in unforeseen behavior.

5. Memory Management
####################
.. _Memory Management:

**mmu** [optional] - In MPU-based platforms which might also support virtual memory (i.e. aarch64
cortex-r) the hypervisor sets up the VM using an MPU by default. If the user wants this VM to use
the MMU they must set the config ``mmu`` parameter to true;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can remove this and add it to the architecture specific part for armv8-r. We could eventually move this option to the config arch structure as well.


6. Architectural-Specific Configurations
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking this should be last and have two subsection: arm (maybe armv8-a and armv8-r) and risc-v

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed old section 5 and split the old section 6 (currently in section 5) in the two architectures (arm and risc-v). However, I saw this information was already on the porting guide, so I mirrored the information here.

########################################
.. _Architectural-Specific Configurations:

- **arch** [mandatory] - allows the definition of architecture dependent configurations and is
configured through the following structure:

.. code-block:: c

struct arch_vm_platform {

// Configuration of the Generic Interrupt Controller (GIC)
struct vgic_dscrp {
paddr_t gicd_addr;
paddr_t gicc_addr;
paddr_t gicr_addr;
size_t interrupt_num;
} gic;

// Configuration of the System Memory Management Unit (SMMU)
struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could add a name to the struct here also:

Suggested change
struct {
struct smmu_config {

(we have to do this in the code too)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in e9e1c43.

streamid_t global_mask;
size_t group_num;
struct smmu_group {
streamid_t mask;
streamid_t id;
} *groups;
} smmu;
};


where:

- **vgic_dscrp** [mandatory] - corresponds to the configuration of the Generic Interrupt Controller
(GIC);
- **vgic_dscrp** [optional] - corresponds to the configuration of the System Memory Management Unit
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **vgic_dscrp** [optional] - corresponds to the configuration of the System Memory Management Unit
- **smmu_config** [optional] - corresponds to the configuration of the System Memory Management Unit

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixed in 431be7f.

(SMMU);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These require a deeper explanation



CPU Affinity
************

The configuration file of the Bao hypervisor also enables the definition of core affinity, which
involves selecting the physical core where the guest should run.

.. figure:: img/cpu-affinity.svg
:align: center
:name: cpu-affinity-fig
danielRep marked this conversation as resolved.
Show resolved Hide resolved

This functionality is achieved through the following configuration parameter:
danielRep marked this conversation as resolved.
Show resolved Hide resolved

- **cpu_affinity** [optional] - corresponds to a bitmap signaling the preferred physical CPUs
assigned to the VM. If this value is mutually exclusive for all the VMs, the physical CPUs
assigned to each VM follow the bitmap. Otherwise (in case of bit overlap or lack of affinity
definition), the CPU assignment is defined by the hypervisor;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the cpu affinity will be as best as possible be followed, but it is not guaranteed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, added this information!


Coloring
********
Comment on lines 516 to 517
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Briefly explain what cache coloring is. A mechanism for partitioning the shared LLC based sets.

Also, mention that because it is based on carefully selecting the frames allocated to the VM, it does not take effect if the physical mapping feature is enabled for a region.

Also, it only works for virtual memory / MMU systems.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, addressed in 2bbd78f.


- **colors** [optional] - corresponds to a bitmap for the assigned cache colors of the VM. This
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This field can also be in shared memory definitions or the hypervisor config structure. We need to make this clear. Maybe we could have a generic cache coloring region (outside the VM config) to describe coloring and just point to it here.

value is truncated depending on the number of available colors calculated at run-time, i.e., its
platform-dependent. By default, the coloring mechanism is not active. For instance, the following
picture depicts a hypothetical setup with a 50/50 coloring scheme;

.. figure:: img/llc-colors.svg
:align: center
:name: llc-colors-fig
danielRep marked this conversation as resolved.
Show resolved Hide resolved

danielRep marked this conversation as resolved.
Show resolved Hide resolved
Shared Memory Configuration
---------------------------
DavidMCerdeira marked this conversation as resolved.
Show resolved Hide resolved

Expand Down
Loading
Loading