Update the guide to 23.11

README.md

@@ -1,6 +1,6 @@
 <!-- ANCHOR: cover -->
 
-# Getting Started with CheriBSD
+# Getting Started with CheriBSD 23.11
 
 Robert N. M. Watson (University of Cambridge)
 and
@@ -15,7 +15,7 @@ further information and support.
 <!--
 NOTE: A release version is also in SUMMARY.md.
 -->
-**The document describes CheriBSD as of the 22.12 release, unless explicitly
+**The document describes CheriBSD as of the 23.11 release, unless explicitly
 stated in sections referring to earlier or later releases.**
 
 *This document is a work-in-progress.  Feedback and contributions are

book.toml

@@ -3,7 +3,7 @@ authors = ["Robert N. M. Watson"]
 language = "en"
 multilingual = false
 src = "src"
-title = "Getting Started with CheriBSD"
+title = "Getting Started with CheriBSD 23.11"
 
 [output.html]
 git-repository-url = "https://github.com/CTSRD-CHERI/cheribsd-getting-started"

src/SUMMARY.md

@@ -1,6 +1,6 @@
 # Summary
 
-[Getting Started with CheriBSD](cover/README.md)
+[Getting Started with CheriBSD 23.11](cover/README.md)
 
 - [Introduction](introduction/README.md)
 - [Background](background/README.md)
@@ -24,12 +24,9 @@
   - [Limitations of packages](packages/limitations.md)
   - [Missing packages](packages/missing.md)
   - [Upgrading packages](packages/upgrading.md)
+  - [Debugging packages](packages/debugging.md)
   - [Useful commands](packages/commands.md)
-- [CheriABI "Hello World"](helloworld/README.md)
+- [Compiling "Hello World"](helloworld/README.md)
+- [Benchmarking guidance](benchmarking/README.md)
 - [Getting help](support/README.md)
 - [Resources](resources/README.md)
-
-<!--
-NOTE: A release version is also in README.md.
--->
-[Current release: 22.12]()

src/benchmarking/README.md

@@ -0,0 +1,104 @@
+# Benchmarking guidance
+
+CheriBSD is a research operating system designed to run on experimental
+hardware.
+In this section, we provide high-level guidance on configuring CheriBSD for
+software benchmarking.
+
+# Framing
+
+The expectation of this section is that a performance comparison is being
+made between baseline aarch64-compiled software (64-bit Arm) and memory-safe
+aarch64c software (CheriABI) on the Arm Morello board.
+
+Before proceeding to the remainder of this section, it is essential to first
+read [Early performance results from the prototype Morello
+microarchitecture](https://ctsrd-cheri.github.io/morello-early-performance-results/cover/index.html).
+That document provides detailed information on how to interpret performance
+measurements, including documenting known limitations of the prototype
+Morello microarchitecture.
+
+A key conclusion from that work is that CheriABI software being used for
+performance measurement should be compiled for the aarch64cb "Benchmark ABI",
+and not aarch64c.
+
+# Kernel configuration
+
+Depending on the version of CheriBSD you are using, the kernel may have a
+number of debugging fetures enabled.
+These features can substantially impact system performance, including
+inducing disproportionate performance overhead for specific system behaviours.
+For example, enabling kernel lock-order checking ("WITNESS") will introduce
+substantial overhead, and in particular will impact kernel-centric workloads
+that make more intensive use of locks, such as networking.
+During boot, CheriBSD will add the following lines to `/etc/motd`, which will
+be displayed at login, to warn you about the "INVARIANTS" and "WITNESS"
+debugging features:
+
+```
+WARNING: INVARIANTS kernel option defined, expect reduced performance
+WARNING: WITNESS kernel option defined, expect reduced performance
+```
+
+Performance measurements employing a hybrid kernel should use the
+`kernel.GENERIC-MORELLO-NODEBUG` configuration when kernel memory safety is
+not required.
+The following line can be added to `/boot/loader.conf`:
+
+```
+kernel="kernel.GENERIC-MORELLO-NODEBUG"
+```
+
+Measurements requiring a memory-safe kernel should use the
+`kernel.GENERIC-MORELLO-PURECAP-NODEBUG` configuration.
+The following can be added to `/boot/loader.conf`:
+
+```
+kernel="kernel.GENERIC-MORELLO-PURECAP-NODEBUG"
+```
+
+You will need to reboot in order for this change to take effect.
+
+# Heap temporal memory safety
+
+As of CheriBSD 23.11, userlevel heap temporal safety is enabled by default with jemalloc as the memory allocator.
+This support is experimental, and has not yet been through significant
+performance analysis and optimization.
+We recommend disabling temporal safety support for the full system during
+benchmarking not specifically intended to capture temporal safety
+performance.
+During boot, CheriBSD will add the following line to `/etc/motd`, which will
+be displayed at login, to warn you about the temporal safety feature:
+
+```
+WARNING: capability revocation enabled by default, this may affect performance
+```
+
+The following line can be added to `/boot/loader.conf`:
+
+```
+security.cheri.runtime_revocation_default=0
+```
+
+You will need to reboot in order for this change to take effect.
+
+# The Benchmark ABI
+
+As of CheriBSD 23.11, CheriBSD supports the Benchmark ABI, a modified form of
+code generation improving performance on the Arm Morello board.
+This is required due to limitations on bounds prediction in the current
+Morello prototype, which would be resolved in a production microarchitecture.
+The performance of the Benchmark ABI is more predictive of potential future
+CHERI microarchitectural performance than the Morello prototype running
+software compiled for CheriABI, making it preferable for benchmarking.
+However, this comes at the cost of reduce security, and so software compiled
+for the Benchmark ABI should not be used for security evaluation.
+Programs may be compiled to the Benchmark ABI using the
+`-mabi=purecap-benchmark` command-line argument to `cc`.
+
+More information on compiling with the Benchmark ABI can be found in
+[Compiling Hello World](../helloworld/).
+More information on what the Benchmark ABI is, and how to interpret
+performance results, can be found in [Early performance results from the
+prototype Morello
+microarchitecture](https://ctsrd-cheri.github.io/morello-early-performance-results/cover/index.html).

src/features/c18n.md

@@ -13,6 +13,108 @@ CHERI-enabled software compartmentalization models:
 - **Dynamic-linker-based compartmentalization** isolates shared libraries
   within a process using CHERI capabilities limiting the access of attackers
   who have achieved arbitrary code execution within a library.
-  Initial support for linker-based library compartmentalization is included
-  in the 22.12 release of CheriBSD.  See the compartmentalization(7) manual
-  page on an installed system for more information.
+  The linker-based library compartmentalization model has been included since
+  the 22.12 release of CheriBSD.
+  See the [c18n(3) man
+  page](https://man.cheribsd.org/cgi-bin/man.cgi/releng-23.11/c18n)
+  on an installed system for more information.
+
+## Library compartmentalization
+
+CheriBSD's library compartmentalization feature (c18n) executes each dynamic
+library within a compartmentalization-enabled process in its own protection
+domain.
+The non-default c18n-enabled run-time linker grants libraries capabilities
+only to resources (global variables, APIs) declared in their ELF linkage.
+Function calls that cross domain boundaries are interposed on by
+domain-crossing shims implemented by the run-time linker.
+
+The adversary model for these compartments is one of trusted code but
+untrustworthy execution: a library such as `libpng` or `libjpeg` is trusted
+until it begins dynamic execution -- and has potentially been exposed to
+malicious data.
+With library compartmentalization, an adversary who achieves arbitrary code
+execution within the library at run time will be able to reach only the
+resources (and further attack surfaces) declared statically through its
+linkage.
+The programmer must then harden that linkage, and any involved APIs, to make
+them suitable for adversarial engagement -- but the foundation of isolation,
+controlled access, and controlled domain transition is provided by the c18n
+implementation.
+
+In addition to a modified run-time linker, modest changes have been made to
+the aarch64c calling convention to avoid assumptions such as implicit stack
+sharing between callers and callees across library boundaries when passing
+variadic argument lists.
+This modified ABI is now used by all CheriABI binaries in CheriBSD, and so
+off-the-shelf aarch64c binaries and libraries can be used with library
+compartmentalization without recompilation to the modified ABI.
+More information on library compartmentalization can be found in the
+[c18n(3) man
+page](https://man.cheribsd.org/cgi-bin/man.cgi/releng-23.11/c18n):
+
+```
+man c18n
+```
+
+## Compiling applications for library compartmentalization
+
+To compile a main application to use library compartmentalization, add the
+following flags to compilation of the program binary:
+
+```
+-Wl,--dynamic-linker=/libexec/ld-elf-c18n.so.1
+```
+
+For example, compile `helloworld.c` using:
+
+```
+cc -Wall -g -o helloworld helloworld.c -Wl,--dynamic-linker=/libexec/ld-elf-c18n.so.1
+```
+
+You can confirm whether a binary uses the c18n run-time linker by inspecting
+it using the `file` command:
+
+```
+file helloworld
+```
+
+## Tracing compartment-boundary crossings
+
+The BSD ktrace(1) command is able to trace compartment-boundary crossings.
+To enable this feature, set the `LD_C18N_UTRACE_COMPARTMENT` environmental
+variable, which will cause the c18n run-time linker to emit records using
+the utrace(2) system call.
+Run the program under ktrace with the `-tu` argument to capture only those
+records (and not a full system-call trace):
+
+```
+env LD_C18N_UTRACE_COMPARTMENT=1 ktrace -tu ./helloworld
+```
+
+The resulting `ktrace.out` file can be viewed using the kdump(1) command:
+
+```
+kdump
+```
+
+It is important to understand, however, that simply isolating running code is
+almost always insufficient to achieve robust sandboxing.
+The competent adversary will now consider further rights and attack surfaces
+to explore in search of further vulnerabilities.
+While this increased work factor of finding additional vulnerabilities is an
+important part of compartmentalization, internal software APIs are rarely well
+suited to be security boundaries without performing additional hardening.
+With this in mind, you can:
+
+ * Inspect the source code, output from `objdump`, and output from
+   `chericat` to assess the robustness of this compartmentalization.
+   You can install `objdump` with `pkg64 install llvm-base` and `chericat` with `pkg64c install chericat`.
+ * Consider larger software architectural changes that will allow a library
+   to be used more robustly when running within a computerment.
+
+Library compartmentalization has the potential to significantly improve
+software integrity and confidentiality properties in the presence of a strong
+adversary.
+However, it is also limited by the abstraction being around the current
+library operational model.

src/features/desktop.md

@@ -1,8 +1,8 @@
 # CheriABI desktop environment (experimental)
 
-As of the 22.12 release, the installer has gained the option to install a
-desktop environment using the Mali Bifrost GPU on the Morello System-on-Chip.
-The option installs a basic desktop environment using KDE and Wayland
-compiled for CheriABI with the `cheri-desktop` package.  It also
-installs a hybrid ABI Firefox web browser via the `cheri-desktop-hybrid-extras`
-package.
+The installer has the option to install a desktop environment using the Mali
+Bifrost GPU on the Morello System-on-Chip.
+The option installs a basic desktop environment using KDE and Wayland compiled
+for CheriABI with the `cheri-desktop` package.
+It also installs a hybrid ABI Chromium web browser via the
+`cheri-desktop-hybrid-extras` package.

src/features/processes.md

@@ -1,21 +1,51 @@
 # Userlevel process environments
 
-The CheriBSD userspace likewise supports two different execution environments,
-hybrid processes and CheriABI (pure-capability) processes:
+CheriBSD 23.11 likewise supports three different userspace execution
+environments:
 
 - **Hybrid processes** provide strong binary compatibility with the non-CHERI
   version of the same architecture -- for example, aarch64 on Morello.
-- **CheriABI processes** implement strong referential and spatial memory
-  protection through the system-call interface, dynamic linker, language
-  runtime including heap memory allocators, and compiler-generated code.
+
+- **CheriABI processes** support pure-capability code execution.
+  This process environment implements strong referential, spatial, and
+  [heap temporal memory protection](temporal.md) through the system-call
+  interface, dynamic linker, language runtime including heap memory
+  allocators, and compiler-generated code.
   This protects against memory memory-safety vulnerabilities in both system
   services and applications.
+  See [Building for CheriABI](../helloworld/index.html#building-for-cheriabi)
+  for information on building for CheriABI.
   CheriABI is described in an [ASPLOS 2019
   paper](https://www.cl.cam.ac.uk/research/security/ctsrd/pdfs/201904-asplos-cheriabi.pdf).
 
-Both environments can be used over either of the hybrid or pure-capability
-kernels.
+- **Benchmark ABI processes** implement a code-generation model similar to
+  CheriABI, but with relaxed program-counter capability bounds that enable
+  improved performance prediction on the Arm Morello platform.
+  See [Building for the Benchmark
+  ABI](../helloworld/index.html#building-for-the-benchmark-abi).
+  The Benchmark ABI is described in a [technical report on the topic from
+  Arm and
+  Cambridge](https://ctsrd-cheri.github.io/morello-early-performance-results/cover/index.html).
+
+All of these environments can be used over either of the hybrid or
+pure-capability kernels.
+You can determine the process environment that a particular process is using
+via the `procstat(1)` command, whose default output includes an `EMUL`
+column indicating the process ABI:
+
+```
+% procstat -a
+  PID  PPID  PGID   SID  TSID THR LOGIN    WCHAN     EMUL           COMM
+...
+ 3197  3193  3193  3193     0   1 robert   select    FreeBSD ELF64C sshd
+ 3214  3170  3214  3170  3170   1 robert   select    FreeBSD ELF64  bash
+...
+```
+
+In this example, the `sshd` daemon is using CheriABI (`FreeBSD ELF64C`), and
+the `bash` command is using the Hybrid ABI (`FreeBSD ELF64`).
 
 Pre-compiled third-party software applications (packages) are provided for
-both ABIs, although CheriABI packages are currently considered experimental.
-This is discussed further in the [chapter on packages](../packages/).
+each of these ABIs, although CheriABI and Benchmark ABI packages are currently
+considered experimental.  This is discussed further in the [chapter on
+packages](../packages/).