Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SwiftDocC: use "portable" paths for file names #668

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

compnerd
Copy link
Member

Windows restricts a CharacterSet from use in the file name. Replace that set with _. This requires an associated change in the DocC renderer to perform the substitution when converting the URL to a file path.

compnerd added a commit to compnerd/swift-docc-render that referenced this pull request Jul 22, 2023
Some file systems have restrictions on character sets which are valid
file name characters.  Add a filter for the Windows file system
character set restrictions.  We replace them with `_` to match the
behaviour in the DocC bundle generation after swiftlang/swift-docc#668.
compnerd added a commit to compnerd/swift-docc-render that referenced this pull request Jul 22, 2023
Some file systems have restrictions on character sets which are valid
file name characters.  Add a filter for the Windows file system
character set restrictions.  We replace them with `_` to match the
behaviour in the DocC bundle generation after swiftlang/swift-docc#668.
@compnerd
Copy link
Member Author

This has the associated change in the docc-render application to permit the continued use of the characters in the URL.

@compnerd
Copy link
Member Author

@swift-ci please test

@compnerd compnerd requested a review from mportiz08 July 26, 2023 15:18
let components = name.components(separatedBy: ["<", ">", ":", "\"", "/", "\\", "|", "?", "*"])
if components.count > 1 {
isURLModified = true
name = components.joined(separator: "_")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should always append a hashed identifier to the name here to ensure that the file names don't collide.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that appending the hashed identifier seems reasonable. Definitely would avoid any possible conflicts. Would we do that for any case where isURLModified is true?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code already mostly does that when modifying the URL, although if it's a specific path component that's longer than what a file name is allowed to be, then it shortens that path component and appends the hash to that component instead of later in the URL.

"/tutorials/Technology-file:-with---whitespace,-and_punctuation/Volume_Section-Title:-with---various!-whitespace,-and/punctuation",
"/tutorials/Technology-file:-with---whitespace,-and_punctuation/Volume_Section-Title:-with---various!-whitespace,-and/punctuation/Chapter_Title:-with---various!-whitespace,-and/punctuation"
"/tutorials/Technology-file_-with---whitespace,-and_punctuation/Volume_Section-Title:-with---various!-whitespace,-and/punctuation",
"/tutorials/Technology-file_-with---whitespace,-and_punctuation/Volume_Section-Title:-with---various!-whitespace,-and/punctuation/Chapter_Title:-with---various!-whitespace,-and/punctuation"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it expected that the : towards the end of this string ("Chapter_Title:") isn't replaced with _?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that isn't expected, but how did the test pass 🤯. : is a ADS (alternate data stream, think of it as macOS file forks) separator, the file would just be truncated to the : and the rest would be the name of the stream.

@@ -276,7 +276,7 @@ class ExternalLinkableTests: XCTestCase {
let summary = node.externallyLinkableElementSummaries(context: context, renderNode: renderNode)[0]

XCTAssertEqual(summary.title, "globalFunction(_:considering:)")
XCTAssertEqual(summary.relativePresentationURL.absoluteString, "/documentation/mykit/globalfunction(_:considering:)")
XCTAssertEqual(summary.relativePresentationURL.absoluteString, "/documentation/mykit/globalfunction(__considering_)")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this value should change. It represents the relative URL for this page. As far as I understood the URL would still contain the characters that aren't allowed in the file names.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, the URLs can contain the view, but the file paths would be replaced. I believe that the replativePresentationURL is still referencing the file path?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this is meant to refer to the web URL relative to where this documentation is hosted.

This API is used to bridge with other documentation systems which aren't expected to have access to the doccarchive directly. Those systems are expected to use the summary information to create links that navigate the reader to the hosted documentation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can also refer to a landmark on a page which wouldn't have its own file in the documentation archive.

@@ -96,7 +96,7 @@ final class RenderIndexTests: XCTestCase {
"type": "groupMarker"
},
{
"path": "/documentation/mixedlanguageframework/bar/mystringfunction(_:)",
"path": "/documentation/mixedlanguageframework/bar/mystringfunction(__)",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ethan-kusters do you remember if this value represents the path of the web URL or the file path within the documentation archive? I thought that this is the path that the browser would load to access this page.

compnerd added a commit to compnerd/swift-docc-render that referenced this pull request Jul 29, 2023
Some file systems have restrictions on character sets which are valid
file name characters.  Add a filter for the Windows file system
character set restrictions.  We replace them with `_` to match the
behaviour in the DocC bundle generation after swiftlang/swift-docc#668.
compnerd added a commit to compnerd/swift-docc-render that referenced this pull request Aug 10, 2023
Some file systems have restrictions on character sets which are valid
file name characters.  Add a filter for the Windows file system
character set restrictions.  We replace them with `_` to match the
behaviour in the DocC bundle generation after swiftlang/swift-docc#668.

Conditionally enable the new portable paths based upon a setting in
`theme-settings.json` to provide a means for compatibility.
@d-ronnqvist d-ronnqvist added the needs forum discussion Needs to be discussed in the Swift Forums label Dec 21, 2023
Windows restricts a CharacterSet from use in the file name.  Replace
that set with `_`.  This requires an associated change in the DocC
renderer to perform the substitution when converting the URL to a file
path.
@compnerd
Copy link
Member Author

compnerd commented May 9, 2024

@d-ronnqvist this would be nice to get in for 6.0

@d-ronnqvist
Copy link
Contributor

This is not ready to be merged. I added the "needs forum discussion" tag because this previous Swift Forums thread or the Workgroup meeting where this topic was discussed didn't reach a specific conclusion for how to best solve this problem.

It seems like the solution would be to add an opt-in feature flag, communicate about the change and the possible disruption in the Swift Forums, and then reevaluate after one minor release if we should change the default behavior top opt-out instead of opt-in. However, it might be good to revive that thread just to confirm that everyone is happy with that solution. If so, what's missing from this PR is adding that flag and making the behavior conditional.

@richardtop
Copy link

Interested in this being implemented/merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs forum discussion Needs to be discussed in the Swift Forums
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants