Store test content in a custom metadata section.

See also: swiftlang/swift#76698

Resolves #735.

Documentation/ABI/TestContent.md

@@ -0,0 +1,185 @@
+# Runtime-discoverable test content
+
+<!--
+This source file is part of the Swift.org open source project
+
+Copyright (c) 2024 Apple Inc. and the Swift project authors
+Licensed under Apache License v2.0 with Runtime Library Exception
+
+See https://swift.org/LICENSE.txt for license information
+See https://swift.org/CONTRIBUTORS.txt for Swift project authors
+-->
+
+This document describes the format and location of test content that the testing
+library emits at compile time and can discover at runtime.
+
+> [!WARNING]
+> The content of this document is subject to change pending efforts to define a
+> Swift-wide standard mechanism for runtime metadata emission and discovery.
+> Treat the information in this document as experimental.
+
+## Basic format
+
+Swift Testing uses the [ELF Note format](https://man7.org/linux/man-pages/man5/elf.5.html)
+to store individual records of test content. Records created and discoverable by
+the testing library are stored in dedicated platform-specific sections:
+
+| Platform | Binary Format | Section Name |
+|-|:-:|-|
+| macOS, iOS, watchOS, tvOS, visionOS | Mach-O | `__DATA_CONST,__swift5_tests` |
+| Linux, FreeBSD, Android | ELF | `PT_NOTE`[^1] |
+| WASI | Statically Linked | `swift5_tests` |
+| Windows | PE/COFF | `.sw5test`[^2] |
+
+[^1]: On platforms that use the ELF binary format natively, test content records
+      are stored in ELF program headers of type `PT_NOTE`. Take care not to
+      remove these program headers (for example, by invoking [`strip(1)`](https://www.man7.org/linux/man-pages/man1/strip.1.html).)
+[^2]: On Windows, the Swift compiler [emits](https://github.com/swiftlang/swift/blob/main/stdlib/public/runtime/SwiftRT-COFF.cpp)
+      leading and trailing padding into this section, both zeroed and of size
+      `sizeof(uintptr_t)`. Code that walks this section can safely skip over
+      this padding.
+
+### Record headers
+
+Regardless of platform, all test content records created and discoverable by the
+testing library have the following structure:
+
+```c
+struct SWTTestContentHeader {
+  int32_t n_namesz;
+  int32_t n_descsz;
+  int32_t n_type;
+  char n_name[n_namesz];
+  // ...
+};
+```
+
+This structure can be represented in Swift as a heterogenous tuple:
+
+```swift
+typealias SWTTestContentHeader = (
+  n_namesz: Int32,
+  n_descsz: Int32,
+  n_type: Int32,
+  n_name: (CChar, CChar, /* ... */),
+  // ...
+)
+```
+
+The size of `n_name` is dynamic and cannot be statically computed. The testing
+library always generates the name `"Swift Testing"` and specifies an `n_namesz`
+value of `20` (the string being null-padded to the correct length), but other
+content may be present in the same section whose header size differs. For more
+information about this structure such as its alignment requirements, see the
+documentation for the [ELF format](https://man7.org/linux/man-pages/man5/elf.5.html).
+
+Each record's _kind_ (stored in the `n_type` field) determines how the record
+will be interpreted at runtime:
+
+| Type Value | Interpretation |
+|-:|-|
+| `< 0` | Undefined (**do not use**) |
+| `0 ... 99` | Reserved |
+| `100` | Test or suite declaration |
+| `101` | Exit test |
+
+<!-- When adding cases to this enumeration, be sure to also update the
+corresponding enumeration in Test+Discovery.swift and
+TestContentGeneration.swift. -->
+
+### Record contents
+
+For all currently-defined record types, the header structure is immediately
+followed by the actual content of the record. A test content record currently
+contains an `accessor` function to load the corresponding Swift content and a
+`flags` field whose value depends on the type of record. The overall structure
+of a record therefore looks like:
+
+```c
+struct SWTTestContent {
+  SWTTestContentHeader header;
+  bool (* accessor)(void *outValue);
+  uint32_t flags;
+  uint32_t reserved;
+};
+```
+
+Or, in Swift as a tuple:
+
+```swift
+typealias SWTTestContent = (
+  header: SWTTestContentHeader,
+  accessor: @convention(c) (_ outValue: UnsafeMutableRawPointer) -> Bool,
+  flags: UInt32,
+  reserved: UInt32
+)
+```
+
+This structure may grow in the future as needed. Check the `header.n_descsz`
+field to determine if there are additional fields present. Do not assume that
+the size of this structure will remain fixed over time or that all discovered
+test content records are the same size.
+
+#### The accessor field
+
+The function `accessor` is a C function. When called, it initializes the memory
+at its argument `outValue` to an instance of some Swift type and returns `true`,
+or returns `false` if it could not generate the relevant content. On successful
+return, the caller is responsible for deinitializing the memory at `outValue`
+when done with it.
+
+The concrete Swift type of the value written to `outValue` depends on the type
+of record:
+
+| Type Value | Return Type |
+|-:|-|
+| `..< 0` | Undefined (**do not use**) |
+| `0 ... 99` | Reserved (**do not use**) |
+| `100` | `@Sendable () async -> Test`[^3] |
+| `101` | `ExitTest` (consumed by caller) |
+
+[^3]: This signature is not the signature of `accessor`, but of the Swift
+      function reference it writes to `outValue`. This level of indirection is
+      necessary because loading a test or suite declaration is an asynchronous
+      operation, but C functions cannot be `async`.
+
+#### The flags field
+
+For test or suite declarations (type `100`), the following flags are defined:
+
+| Bit | Description |
+|-:|-|
+| `1 << 0` | This record contains a suite declaration |
+| `1 << 1` | This record contains a parameterized test function declaration |
+
+For exit test declarations (type `101`), no flags are currently defined.
+
+#### The reserved field
+
+This field is reserved for future use. Always set it to `0`.
+
+## Third-party test content
+
+Testing tools may make use of the same storage and discovery mechanisms by
+emitting their own test content records into the test record content section.
+
+Third-party test content should use the same value for the `n_name` field
+(`"Swift Testing"`). The `n_type` field should be set to a unique value only
+used by that tool, or used by that tool in collaboration with other compatible
+tools. At runtime, Swift Testing ignores test content records with unrecognized
+`n_type` values. To reserve a new unique `n_type` value, open a [GitHub issue](https://github.com/swiftlang/swift-testing/issues/new/choose)
+against Swift Testing.
+
+The layout of third-party test content records must be compatible with that of
+`SWTTestContentHeader` as specified above. For the actual content of a test
+record, you do not need to use the same on-disk/in-memory layout as is specified
+by `SWTTestContent` above, but it is preferred. Third-party tools are ultimately
+responsible for ensuring the values they emit into the test content section are
+correctly aligned and have sufficient padding; failure to do so may render
+downstream test code unusable.
+
+<!--
+TODO: elaborate further, give examples
+TODO: standardize a mechanism for third parties to produce `Test` instances
+      since we don't have a public initializer for the `Test` type.
+-->

Documentation/Porting.md

@@ -113,13 +113,20 @@ Once the header is included, we can call `GetDateTime()` from `Clock.swift`:
 ## Runtime test discovery
 
 When porting to a new platform, you may need to provide a new implementation for
-`enumerateTypeMetadataSections()` in `Discovery.cpp`. Test discovery is
-dependent on Swift metadata discovery which is an inherently platform-specific
-operation.
+`enumerateTestContentSections()` in `Discovery.cpp`. Test discovery is dependent
+on Swift metadata discovery which is an inherently platform-specific operation.
 
-_Most_ platforms will be able to reuse the implementation used by Linux and
-Windows that calls an internal Swift runtime function to enumerate available
-metadata. If you are porting Swift Testing to Classic, this function won't be
+> [!NOTE]
+> You do not need to proviate an implementation for the similarly-named function
+> `enumerateTypeMetadataSections()` in Discovery+Old.cpp: it is present for
+> backwards compatibility with Swift 6.0 toolchains and will be removed in a
+> future release.
+
+_Most_ platforms in use today use the ELF image format and will be able to reuse
+the implementation used by Linux. That implementation calls `dl_iterate_phdr()`
+in the GNU C Library to enumerate available metadata.
+
+If you are porting Swift Testing to Classic, `dl_iterate_phdr()` won't be
 available, so you'll need to write a custom implementation instead. Assuming
 that the Swift compiler emits section information into the resource fork on
 Classic, you could use the [Resource Manager](https://developer.apple.com/library/archive/documentation/mac/pdf/MoreMacintoshToolbox.pdf)
@@ -132,28 +139,79 @@ to load that information:
  // ...
 +#elif defined(macintosh)
 +template <typename SectionEnumerator>
-+static void enumerateTypeMetadataSections(const SectionEnumerator& body) {
++static void enumerateTestContentSections(const SectionEnumerator& body) {
 +  ResFileRefNum refNum;
 +  if (noErr == GetTopResourceFile(&refNum)) {
 +    ResFileRefNum oldRefNum = refNum;
 +    do {
 +      UseResFile(refNum);
-+      Handle handle = Get1NamedResource('swft', "\p__swift5_types");
++      Handle handle = Get1NamedResource('swft', "\p__swift5_tests");
 +      if (handle && *handle) {
-+        size_t size = GetHandleSize(handle);
-+        body(*handle, size);
++        auto imageAddress = reinterpret_cast<const void *>(static_cast<uintptr_t>(refNum));
++        SWTSectionBounds sb = { imageAddress, *handle, GetHandleSize(handle) };
++        bool stop = false;
++        body(sb, &stop);
++        if (stop) {
++          break;
++        }
 +      }
 +    } while (noErr == GetNextResourceFile(refNum, &refNum));
 +    UseResFile(oldRefNum);
 +  }
 +}
  #else
- #warning Platform-specific implementation missing: Runtime test discovery unavailable
+ #warning Platform-specific implementation missing: Runtime test discovery unavailable (dynamic)
  template <typename SectionEnumerator>
- static void enumerateTypeMetadataSections(const SectionEnumerator& body) {}
+ static void enumerateTestContentSections(const SectionEnumerator& body) {}
  #endif
 ```
 
+You will also need to update the `makeTestContentRecordDecl()` function in the
+`TestingMacros` target to emit the correct `@_section` attribute for your
+platform. If your platform uses the ELF image format and supports the
+`dl_iterate_phdr()` function, add it to the existing `#elseif os(Linux) || ...`
+case. Otherwise, add a new case for your platform:
+
+```diff
+--- a/Sources/TestingMacros/Support/TestContentGeneration.swift
++++ b/Sources/TestingMacros/Support/TestContentGeneration.swift
+   // ...
++  #elseif os(Classic)
++  @_section(".rsrc,swft,__swift5_tests")
+   #endif
+```
+
+Keep in mind that this code is emitted by the `@Test` and `@Suite` macros
+directly into test authors' test targets, so you will not be able to use
+compiler conditionals defined in the Swift Testing package (including those that
+start with `"SWT_"`).
+
+## Runtime test discovery with static linkage
+
+If your platform does not support dynamic linking and loading, you will need to
+use static linkage instead. Define the `"SWT_NO_DYNAMIC_LINKING"` compiler
+conditional for your platform in both Package.swift and CompilerSettings.cmake,
+then define the `sectionBegin` and `sectionEnd` symbols in Discovery.cpp:
+
+```diff
+diff --git a/Sources/_TestingInternals/Discovery.cpp b/Sources/_TestingInternals/Discovery.cpp
+ // ...
++#elif defined(macintosh)
++extern "C" const char sectionBegin __asm__("...");
++extern "C" const char sectionEnd __asm__("...");
+ #else
+ #warning Platform-specific implementation missing: Runtime test discovery unavailable (static)
+ static const char sectionBegin = 0;
+ static const char& sectionEnd = sectionBegin;
+ #endif
+```
+
+These symbols must have unique addresses corresponding to the first byte of the
+test content section and the first byte _after_ the test content section,
+respectively. Their linker-level names will be platform-dependent: refer to the
+linker documentation for your platform to determine what names to place in the
+`__asm__` attribute applied to each.
+
 ## C++ stub implementations
 
 Some symbols defined in C and C++ headers, especially "complex" macros, cannot

Package.swift

@@ -128,6 +128,8 @@ extension Array where Element == PackageDescription.SwiftSetting {
       .enableExperimentalFeature("AccessLevelOnImport"),
       .enableUpcomingFeature("InternalImportsByDefault"),
 
+      .enableExperimentalFeature("SymbolLinkageMarkers"),
+
       .define("SWT_TARGET_OS_APPLE", .when(platforms: [.macOS, .iOS, .macCatalyst, .watchOS, .tvOS, .visionOS])),
 
       .define("SWT_NO_EXIT_TESTS", .when(platforms: [.iOS, .watchOS, .tvOS, .visionOS, .wasi, .android])),