Treat external content as "pre-rendered"

[d-ronnqvist]

Feature Name

Treat external content as "pre-rendered"

Description

When DocC resolves content from an external source via an "external reference resolver" it should treat that content as "pre-rendered" instead of trying to render it in the local context.

This is also tracked as rdar://78718811

Motivation

Treating external content as "pre rendered" would enable:

• Resolving DocC content from one build in another DocC build
• Resolving content with references (for example to media, downloads, or other external pages)
• Resolving content with formatted abstracts

It would also remove some code duplication among the different representations of external content—as output from DocC or as input to DocC.

Importance

Resolving DocC content from one build in another DocC build

DocC supports writing information about documentation elements that can be linked to (for example pages and on-page landmarks). This information contains rendered representation of content.

In the current state DocC can't use content from a previous DocC build. It could be partially achieved in the current implementation by attempting to transform the content into an "unrendered state" so that it can be rendered again in the local context. This transformation is lossy and wouldn't work for content with references (see more below).

There's also a risk that the re-rendering the external content would result in a different result than the original content.

Resolving content with references (for example to media, downloads, or other external pages)

For external content with references it is already known what the rendered representations of those references are but with the current implementation they would need to be "unrendered" and re-resolved and re-rendered in the local context. Even if the external references were recursively resolved and added to the local context, re-resolving them in the local context could result in different results than the original external references. For example, there could be another page in the local context that's a closer match or the reference could be ambiguous in the local context and fail to resolve. What's worse is that adding the external references to the local context could result in links to actual local content failing to resolve because of ambiguous results.

If the already rendered representations of the references could be passed through DocC unmodified they would be known to be the same as their originals and it wouldn't mix local and external content and assets.

Resolving content with formatted abstracts

For external content with rendered representations of formatted abstracts, this content needs to be "un-rendered", re-parsed as markdown, and re-rendered with the current implementation. This could be achieved for basic formatting (for example styled text) with the current implementation but it would support all formatting if the already rendered content could be passed through DocC unmodified.

Reducing code duplication

If DocC used the same format for the external content that it outputs and the external content that it uses as input then we could remove the previous input format. It would also mean that new information that's added to the output format would automatically be supported in the input format.

Alternatives Considered

Rendering external content in the local context (the current implementation) is the main alternative. However, this has the downsides that are mentioned above. We have gotten by with this solution for a few years but it's reaching its limits and continuing to build upon the current implementation is resulting in a few unintuitive workarounds.

Changing the way DocC treats external content would be a breaking change to the public API for SwiftDocC and a complete change of the process-to-process data format that DocC uses to ingest content from external resolvers. However, it's a niche API and I don't think anyone else is using it.