Improve documentation on ABIs, temporal safety, and c18n for 23.11.

src/features/c18n.md

@@ -15,5 +15,106 @@ CHERI-enabled software compartmentalization models:
   who have achieved arbitrary code execution within a library.
   The linker-based library compartmentalization model has been included since
   the 22.12 release of CheriBSD.
-  See the compartmentalization(7) manual page on an installed system for more
-  information.
+  See the [c18n(3) man
+  page](https://man.cheribsd.org/cgi-bin/man.cgi/releng-23.11/c18n)
+  on an installed system for more information.
+
+## Library compartmentalization
+
+CheriBSD's library compartmentalization feature (c18n) executes each dynamic
+library within a compartmentalization-enabled process in its own protection
+domain.
+The non-default c18n-enabled run-time linker grants libraries capabilities
+only to resources (global variables, APIs) declared in their ELF linkage.
+Function calls that cross domain boundaries are interposed on by
+domain-crossing shims implemented by the run-time linker.
+
+The adversary model for these compartments is one of trusted code but
+untrustworthy execution: a library such as `libpng` or `libjpeg` is trusted
+until it begins dynamic execution -- and has potentially been exposed to
+malicious data.
+With library compartmentalization, an adversary who achieves arbitrary code
+execution within the library at run time will be able to reach only the
+resources (and further attack surfaces) declared statically through its
+linkage.
+The programmer must then harden that linkage, and any involved APIs, to make
+them suitable for adversarial engagement -- but the foundation of isolation,
+controlled access, and controlled domain transition is provided by the c18n
+implementation.
+
+In addition to a modified run-time linker, modest changes have been made to
+the aarch64c calling convention to avoid assumptions such as implicit stack
+sharing between callers and callees across library boundaries when passing
+variadic argument lists.
+This modified ABI is now used by all CheriABI binaries in CheriBSD, and so
+off-the-shelf aarch64c binaries and libraries can be used with library
+compartmentalization without recompilation to the modified ABI.
+More information on library compartmentalization can be found in the
+[c18n(3) man
+page](https://man.cheribsd.org/cgi-bin/man.cgi/releng-23.11/c18n):
+
+```
+man c18n
+```
+
+## Compiling applications for library compartmentalization
+
+To compile a main application to use library compartmentalization, add the
+following flags to compilation of the program binary:
+
+```
+-Wl,--dynamic-linker=/libexec/ld-elf-c18n.so.1
+```
+
+For example, compile `helloworld.c` using:
+
+```
+cc -Wall -g -o helloworld helloworld.c -Wl,--dynamic-linker=/libexec/ld-elf-c18n.so.1
+```
+
+You can confirm whether a binary uses the c18n run-time linker by inspecting
+it using the `file` command:
+
+```
+file helloworld
+```
+
+## Tracing compartment-boundary crossings
+
+The BSD ktrace(1) command is able to trace compartment-boundary crossings.
+To enable this feature, set the `LD_C18N_UTRACE_COMPARTMENT` environmental
+variable, which will cause the c18n run-time linker to emit records using
+the utrace(2) system call.
+Run the program under ktrace with the `-tu` argument to capture only those
+records (and not a full system-call trace):
+
+```
+env LD_C18N_UTRACE_COMPARTMENT=1 ktrace -tu ./helloworld
+```
+
+The resulting `ktrace.out` file can be viewed using the kdump(1) command:
+
+```
+kdump
+```
+
+It is important to understand, however, that simply isolating running code is
+almost always insufficient to achieve robust sandboxing.
+The competent adversary will now consider further rights and attack surfaces
+to explore in search of further vulnerabilities.
+While this increased work factor of finding additional vulnerabilities is an
+important part of compartmentalization, internal software APIs are rarely well
+suited to be security boundaries without performing additional hardening.
+With this in mind, you can:
+
+ * Inspect the source code, output from `objdump`, and output from
+   `chericat` to assess the robustness of this compartmentalization.
+   You can install `objdump` with `pkg64 install llvm-base` and `chericat` with `pkg64c install chericat`.
+ * Consider larger software architectural changes that will allow a library
+   to be used more robustly when running within a computerment.
+
+Library compartmentalization has the potential to significantly improve
+software integrity and confidentiality properties in the presence of a strong
+adversary.
+However, it is also limited by the abstraction being around the current
+library operational model.

src/features/processes.md

@@ -1,21 +1,51 @@
 # Userlevel process environments
 
-The CheriBSD userspace likewise supports two different execution environments,
-hybrid processes and CheriABI (pure-capability) processes:
+CheriBSD 23.11 likewise supports three different userspace execution
+environments:
 
 - **Hybrid processes** provide strong binary compatibility with the non-CHERI
   version of the same architecture -- for example, aarch64 on Morello.
-- **CheriABI processes** implement strong referential and spatial memory
-  protection through the system-call interface, dynamic linker, language
-  runtime including heap memory allocators, and compiler-generated code.
+
+- **CheriABI processes** support pure-capability code execution.
+  This process environment implements strong referential, spatial, and
+  [heap temporal memory protection](temporal.md) through the system-call
+  interface, dynamic linker, language runtime including heap memory
+  allocators, and compiler-generated code.
   This protects against memory memory-safety vulnerabilities in both system
   services and applications.
+  See [Building for CheriABI](../helloworld/index.html#building-for-cheriabi)
+  for information on building for CheriABI.
   CheriABI is described in an [ASPLOS 2019
   paper](https://www.cl.cam.ac.uk/research/security/ctsrd/pdfs/201904-asplos-cheriabi.pdf).
 
-Both environments can be used over either of the hybrid or pure-capability
-kernels.
+- **Benchmark ABI processes** implement a code-generation model similar to
+  CheriABI, but with relaxed program-counter capability bounds that enable
+  improved performance prediction on the Arm Morello platform.
+  See [Building for the Benchmark
+  ABI](../helloworld/index.html#building-for-the-benchmark-abi).
+  The Benchmark ABI is described in a [technical report on the topic from
+  Arm and
+  Cambridge](https://ctsrd-cheri.github.io/morello-early-performance-results/cover/index.html).
+
+All of these environments can be used over either of the hybrid or
+pure-capability kernels.
+You can determine the process environment that a particular process is using
+via the `procstat(1)` command, whose default output includes an `EMUL`
+column indicating the process ABI:
+
+```
+% procstat -a
+  PID  PPID  PGID   SID  TSID THR LOGIN    WCHAN     EMUL           COMM
+...
+ 3197  3193  3193  3193     0   1 robert   select    FreeBSD ELF64C sshd
+ 3214  3170  3214  3170  3170   1 robert   select    FreeBSD ELF64  bash
+...
+```
+
+In this example, the `sshd` daemon is using CheriABI (`FreeBSD ELF64C`), and
+the `bash` command is using the Hybrid ABI (`FreeBSD ELF64`).
 
 Pre-compiled third-party software applications (packages) are provided for
-both ABIs, although CheriABI packages are currently considered experimental.
-This is discussed further in the [chapter on packages](../packages/).
+each of these ABIs, although CheriABI and Benchmark ABI packages are currently
+considered experimental.  This is discussed further in the [chapter on
+packages](../packages/).

src/features/temporal.md

@@ -1,8 +1,94 @@
 # Userlevel heap temporal memory safety (experimental)
 
-CheriBSD implements, on an experimental development branch, support for the
-Cornucopia heap temporal safety algorithm, as well as successor algorithms
-based on load-side-barrier features present in the Morello prototype
-architecture and processor design.
-Cornucopia is described in an [IEEE SSP 2020
-paper](https://www.cl.cam.ac.uk/research/security/ctsrd/pdfs/2020oakland-cornucopia.pdf).
+CheriBSD 23.11 incorporates support for userlevel heap temporal memory
+safety based on a load-barrier extension to
+[Cornucopia](https://www.cl.cam.ac.uk/research/security/ctsrd/pdfs/2020oakland-cornucopia.pdf),
+which is inspired by garbage-collection techniques.
+A forthcoming paper at ASPLOS 2024 on the Cornucopia Reloaded technique will
+describe these differences in detail.
+
+This feature involves a collaboration between the kernel (which provides
+asynchronous capability revocation with VM acceleration) and the userlevel
+heap allocator (which quarantines freed memory until revocation of any
+pointers to it) to ensure that memory cannot be reallocated until there are
+no outstanding valid capabilities lasting from its previous allocation.
+The userlevel memory allocator and the kernel revoker share an `epoch counter`
+that counts the number of completed atomic revocation sweeps.
+Memory added to quarantine in one epoch cannot be removed from quarantine
+until at least one complete epoch has passed -- i.e., the epoch counter has
+been increased by 2.
+More information on temporal memory safety support can be found in the
+[mrs(3) man page](https://man.cheribsd.org/cgi-bin/man.cgi/dev/mrs):
+
+```
+man mrs
+```
+
+## Checking whether temporal safety is globally enabled
+
+Use the `sysctl(8)` command to inspect the value of the
+`security.cheri.runtime_revocation_default` system MIB entry:
+
+```
+sysctl security.cheri.runtime_revocation_default
+```
+
+This sysctl sets the default policy for revocation used by processes on
+startup.
+We recommend setting this in `/boot/loader.conf`, which is processed by the
+boot loader before any user processes start.
+
+## Controlling revocation by binary or process
+
+You can forcefully enable or disable revocations for a specific binary or
+process
+with
+[elfctl(1)](https://man.cheribsd.org/cgi-bin/man.cgi/dev/elfctl)
+or
+[proccontrol(1)](https://man.cheribsd.org/cgi-bin/man.cgi/dev/proccontrol)
+and ignore the default policy:
+
+```
+elfctl -e <+cherirevoke or +nocherirevoke> <binary>
+```
+
+```
+proccontrol -m cherirevoke -s <enable or disable> <program with arguments>
+```
+
+You can read more on these commands in the
+[mrs(3) man page](https://man.cheribsd.org/cgi-bin/man.cgi/dev/mrs).
+
+## Synchronous revocation
+
+Revocation normally occurs asynchronously, with a memory quarantine preventing
+memory reuse until revocation of any pointers to that memory.
+You can insert a specific call in your program to await synchronous
+revocation:
+
+```
+malloc_revoke();
+```
+
+Synchronous revocation on every call to `free()` can be configured using the
+`_RUNTIME_REVOCATION_EVERY_FREE_ENABLE` environmental variable.
+
+Note that synchronous revocation can incur extremely high expense.
+
+## Monitoring revocation in processes
+
+Use the `procstat cheri -v` command to inspect the CHERI memory-safety state
+of a target process.
+For example:
+
+```
+# procstat cheri -v 923 1012
+  PID COMM                C QUAR  RSTATE                              EPOCH
+  923 seatd               P  yes    none                                  0
+ 1012 Xorg                P  yes    none                               0xd2
+```
+
+Both processes in this example use a pure-capability process environment, have
+quarantining enabled, and are not currently revoking.
+seatd has never needed to perform a revocation pass, as it remains in epoch 0,
+whereas X.org has a non-zero epoch and has performed multiple passes.

src/helloworld/README.md

@@ -86,7 +86,7 @@ main(void)
 }
 ```
 
-## Building
+## Building for CheriABI
 
 To build a CheriABI Hello World program use:
 
@@ -99,6 +99,8 @@ root@cheribsd:~ # file helloworld
 helloworld: ELF 64-bit LSB pie executable, ARM aarch64, C64, CheriABI, version 1 (SYSV), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 14.0 (1400094), FreeBSD-style, with debug_info, not stripped
 ```
 
+## Building for the Benchmark ABI
+
 To target the [Benchmark ABI](../benchmarking/), add the argument
 `-mabi=purecap-benchmark` to the `cc` command line: