Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

swift: split build & Darwin support. #189977

Merged
merged 27 commits into from
Jan 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
d694535
apple_sdk_11_0: install swift modules
stephank Aug 13, 2022
6a20451
apple_sdk_11_0: Foundation depends on Combine
stephank Aug 16, 2022
e23a977
apple_sdk_11_0: Fix missing CoreVideo include
stephank Sep 10, 2022
6062fd0
apple_sdk_11_0: Add SwiftUI dependencies
stephank Sep 9, 2022
64e5389
xcbuild: add JSON variant of SDKSettings
stephank Sep 14, 2022
2c2b799
xcbuild: exit with error if --find fails
stephank Sep 16, 2022
8ed924c
swift: compiler only build & darwin support
stephank Jul 19, 2022
3c54a5d
swiftPackages.Dispatch: 5.5 -> 5.7
stephank Jul 25, 2022
89f9790
swiftPackages.Foundation: init at 5.7
stephank Jul 25, 2022
0b877d8
swiftPackages.XCTest: init at 5.7
stephank Jul 25, 2022
4f082f5
swiftpm2nix: init
stephank Jul 30, 2022
ece9224
swiftpm: init at 5.7
stephank Jul 25, 2022
dcb0eaf
swift-driver: init at 5.7
stephank Aug 26, 2022
ce2dd8b
sourcekit-lsp: init at 5.7
stephank Jul 30, 2022
d63aef6
swift-docc: init at 5.7
stephank Aug 26, 2022
ae174e2
Merge remote-tracking branch 'origin/staging' into HEAD
stephank Nov 15, 2022
0e08b08
swiftpm: improve setup hook shell conditions
toonn Nov 15, 2022
449e2f1
swift: track version in a central sources.nix
stephank Nov 15, 2022
9cb7163
swiftpm2nix: reduce duplication in generated files
stephank Nov 15, 2022
7d559a3
swiftPackages: update swiftpm2nix generated files
stephank Nov 15, 2022
a8f9977
swiftpm: typo fixes
toonn Nov 15, 2022
aae3556
swiftpm: factor out cmake glue installation
stephank Nov 15, 2022
a4c8c87
swift: remove NIX_BUILD_TOP
stephank Nov 15, 2022
da15d44
swiftPackages.Foundation: add note about CF
stephank Nov 21, 2022
72ba293
swift: remove internal wrapperParams arg
stephank Nov 21, 2022
a567024
swift: add canary to verify no compiler dependency
stephank Nov 21, 2022
813c58d
swiftPackages.Dispatch: fix linux build
stephank Nov 21, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
651 changes: 651 additions & 0 deletions pkgs/development/compilers/swift/compiler/default.nix

Large diffs are not rendered by default.

18 changes: 18 additions & 0 deletions pkgs/development/compilers/swift/compiler/patches/clang-wrap.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
Wrap the clang produced during the build

--- a/tools/driver/CMakeLists.txt
+++ b/tools/driver/CMakeLists.txt
@@ -59,6 +59,13 @@ endif()

add_dependencies(clang clang-resource-headers)

+# Nix: wrap the clang build.
+add_custom_command(
+ TARGET clang POST_BUILD
+ COMMAND nix-swift-make-clang-wrapper $<TARGET_FILE:clang>
+ VERBATIM
+)
+
if(NOT CLANG_LINKS_TO_CREATE)
set(CLANG_LINKS_TO_CREATE clang++ clang-cl clang-cpp)
endif()
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
The compiler fails if LLVM modules are enabled and it cannot write its module
cache. This patch detects and rejects the fake, non-existant $HOME used in Nix
builds.

We could simply return false in `cache_directory`, but that completely disables
module caching, and may unnecessarily slow down builds. Instead, let it use
'/tmp/.cache'.

--- a/lib/Support/Unix/Path.inc
+++ b/lib/Support/Unix/Path.inc
@@ -1380,6 +1380,9 @@ bool user_config_directory(SmallVectorImpl<char> &result) {
if (!home_directory(result)) {
return false;
}
+ if (std::equal(result.begin(), result.end(), "/homeless-shelter")) {
+ return false;
+ }
append(result, ".config");
return true;
}
@@ -1401,6 +1404,9 @@ bool cache_directory(SmallVectorImpl<char> &result) {
if (!home_directory(result)) {
return false;
}
+ if (std::equal(result.begin(), result.end(), "/homeless-shelter")) {
+ system_temp_directory(true/*ErasedOnReboot*/, result);
+ }
append(result, ".cache");
return true;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
This patch fixes dylib references during bootstrapping. It's possible
`LIBSWIFT_BUILD_MODE=BOOTSTRAPPING` is not really well tested on Darwin,
because official builds don't use it.

In the near future, Swift will require an existing Swift toolchain to
bootstrap, and we will likely have to replace this any way.

--- a/stdlib/cmake/modules/AddSwiftStdlib.cmake
+++ b/stdlib/cmake/modules/AddSwiftStdlib.cmake
@@ -1035,6 +1035,10 @@ function(add_swift_target_library_single target name)
set(install_name_dir "${SWIFTLIB_SINGLE_DARWIN_INSTALL_NAME_DIR}")
endif()

+ if(DEFINED SWIFTLIB_SINGLE_BOOTSTRAPPING)
+ set(install_name_dir "${lib_dir}/${output_sub_dir}")
+ endif()
+
set_target_properties("${target}"
PROPERTIES
INSTALL_NAME_DIR "${install_name_dir}")
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
CMake tries to read a list field from SDKSettings.plist, but the output of
facebook/xcbuild PlistBuddy is incompatible with Apple's.

Simply set the supported architectures to the one target architecture we're
building for.

--- a/cmake/modules/SwiftConfigureSDK.cmake
+++ b/cmake/modules/SwiftConfigureSDK.cmake
@@ -189,7 +189,7 @@ macro(configure_sdk_darwin
endif()

# Remove any architectures not supported by the SDK.
- remove_sdk_unsupported_archs(${name} ${xcrun_name} ${SWIFT_SDK_${prefix}_PATH} SWIFT_SDK_${prefix}_ARCHITECTURES)
+ set(SWIFT_SDK_${prefix}_ARCHITECTURES "@swiftArch@")

list_intersect(
"${SWIFT_DARWIN_MODULE_ARCHS}" # lhs
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
--- a/lib/Driver/ToolChains.cpp
+++ b/lib/Driver/ToolChains.cpp
@@ -1475,7 +1475,17 @@ const char *ToolChain::getClangLinkerDriver(

// If there is a linker driver in the toolchain folder, use that instead.
if (auto tool = llvm::sys::findProgramByName(LinkerDriver, {toolchainPath}))
- LinkerDriver = Args.MakeArgString(tool.get());
+ return Args.MakeArgString(tool.get());
+ }
+
+ // For Nix, prefer linking using the wrapped system clang, instead of using
+ // the unwrapped clang packaged with swift. The latter is unable to link, but
+ // we still want to use it for other purposes (clang importer).
+ if (auto nixCC = llvm::sys::Process::GetEnv("NIX_CC")) {
+ llvm::SmallString<128> binDir(nixCC.getValue());
+ llvm::sys::path::append(binDir, "bin");
+ if (auto tool = llvm::sys::findProgramByName(LinkerDriver, {binDir.str()}))
+ return Args.MakeArgString(tool.get());
}

return LinkerDriver;
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
Swift normally looks for the Clang resource dir in a subdir/symlink of its own
resource dir. We provide a symlink to the Swift build-time Clang as a default
there, but we also here patch two checks to try locate it via NIX_CC.

The first (ClangImporter.cpp) happens when Swift code imports C modules. The
second (ToolChains.cpp) happens when Swift is used to link the final product.

--- a/lib/ClangImporter/ClangImporter.cpp
+++ b/lib/ClangImporter/ClangImporter.cpp
@@ -68,6 +68,7 @@
#include "llvm/Support/FileSystem.h"
#include "llvm/Support/Memory.h"
#include "llvm/Support/Path.h"
+#include "llvm/Support/Process.h"
#include "llvm/Support/YAMLParser.h"
#include "llvm/Support/YAMLTraits.h"
#include <algorithm>
@@ -809,6 +810,17 @@ importer::addCommonInvocationArguments(

const std::string &overrideResourceDir = importerOpts.OverrideResourceDir;
if (overrideResourceDir.empty()) {
+ // Prefer the Clang resource directory from NIX_CC, to allow swapping in a
+ // different stdenv.
+ // TODO: Figure out how to provide a user override for this. Probably a
+ // niche use case, though, and for now a user can unset NIX_CC to work
+ // around it if necessary.
+ if (auto nixCC = llvm::sys::Process::GetEnv("NIX_CC")) {
+ llvm::SmallString<128> resourceDir(nixCC.getValue());
+ llvm::sys::path::append(resourceDir, "resource-root");
+ invocationArgStrs.push_back("-resource-dir");
+ invocationArgStrs.push_back(std::string(resourceDir.str()));
+ } else {
llvm::SmallString<128> resourceDir(searchPathOpts.RuntimeResourcePath);

// Adjust the path to refer to our copy of the Clang resource directory
@@ -824,6 +836,7 @@ importer::addCommonInvocationArguments(
// Set the Clang resource directory to the path we computed.
invocationArgStrs.push_back("-resource-dir");
invocationArgStrs.push_back(std::string(resourceDir.str()));
+ } // nixCC
} else {
invocationArgStrs.push_back("-resource-dir");
invocationArgStrs.push_back(overrideResourceDir);
--- a/lib/Driver/ToolChains.cpp
+++ b/lib/Driver/ToolChains.cpp
@@ -1372,10 +1372,20 @@ void ToolChain::getClangLibraryPath(const ArgList &Args,
SmallString<128> &LibPath) const {
const llvm::Triple &T = getTriple();

+ // Nix: We provide a `clang` symlink in the default Swift resource root, but
+ // prefer detecting the Clang resource root via NIX_CC, to allow swapping in
+ // a different stdenv. However, always honor a user-provided `-resource-dir`.
+ auto nixCC = llvm::sys::Process::GetEnv("NIX_CC");
+ if (nixCC && !Args.hasArgNoClaim(options::OPT_resource_dir)) {
toonn marked this conversation as resolved.
Show resolved Hide resolved
+ LibPath.assign(nixCC.getValue());
+ llvm::sys::path::append(LibPath, "resource-root");
+ } else {
getResourceDirPath(LibPath, Args, /*Shared=*/true);
// Remove platform name.
llvm::sys::path::remove_filename(LibPath);
- llvm::sys::path::append(LibPath, "clang", "lib",
+ llvm::sys::path::append(LibPath, "clang");
+ } // nixCC
+ llvm::sys::path::append(LibPath, "lib",
T.isOSDarwin() ? "darwin"
: getPlatformNameForTriple(T));
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
Prevents a user-visible warning on every compilation:

ld: warning: directory not found for option '-L.../MacOSX11.0.sdk/usr/lib/swift'

--- a/lib/Driver/ToolChains.cpp
+++ b/lib/Driver/ToolChains.cpp
@@ -1455,9 +1455,11 @@ void ToolChain::getRuntimeLibraryPaths(SmallVectorImpl<std::string> &runtimeLibP
runtimeLibPaths.push_back(std::string(scratchPath.str()));
}

+ if (!SDKPath.startswith("@storeDir@")) {
scratchPath = SDKPath;
llvm::sys::path::append(scratchPath, "usr", "lib", "swift");
runtimeLibPaths.push_back(std::string(scratchPath.str()));
+ }
}
}

--- a/lib/Frontend/CompilerInvocation.cpp
+++ b/lib/Frontend/CompilerInvocation.cpp
@@ -185,7 +185,9 @@ static void updateRuntimeLibraryPaths(SearchPathOptions &SearchPathOpts,
RuntimeLibraryImportPaths.push_back(std::string(LibPath.str()));
}

- LibPath = SearchPathOpts.getSDKPath();
+ auto SDKPath = SearchPathOpts.getSDKPath();
+ if (!SDKPath.startswith("@storeDir@")) {
+ LibPath = SDKPath;
llvm::sys::path::append(LibPath, "usr", "lib", "swift");
if (!Triple.isOSDarwin()) {
// Use the non-architecture suffixed form with directory-layout
@@ -200,6 +202,7 @@ static void updateRuntimeLibraryPaths(SearchPathOptions &SearchPathOpts,
llvm::sys::path::append(LibPath, swift::getMajorArchitectureName(Triple));
}
RuntimeLibraryImportPaths.push_back(std::string(LibPath.str()));
+ }
}
SearchPathOpts.setRuntimeLibraryImportPaths(RuntimeLibraryImportPaths);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
Patch paths to use the separate 'lib' output. One of the things this patch
fixes is the output of `swift -frontend -print-target-info`, which swiftpm uses
to set rpath on Linux.

The check if the executable path starts with 'out' is necessary for
bootstrapping, or the compiler will fail when run from the build directory.

--- a/lib/Frontend/CompilerInvocation.cpp
+++ b/lib/Frontend/CompilerInvocation.cpp
@@ -49,11 +49,16 @@ swift::CompilerInvocation::CompilerInvocation() {
void CompilerInvocation::computeRuntimeResourcePathFromExecutablePath(
StringRef mainExecutablePath, bool shared,
llvm::SmallVectorImpl<char> &runtimeResourcePath) {
+ if (mainExecutablePath.startswith("@storeDir@")) {
+ auto libPath = StringRef("@lib@");
+ runtimeResourcePath.append(libPath.begin(), libPath.end());
+ } else {
runtimeResourcePath.append(mainExecutablePath.begin(),
mainExecutablePath.end());

llvm::sys::path::remove_filename(runtimeResourcePath); // Remove /swift
llvm::sys::path::remove_filename(runtimeResourcePath); // Remove /bin
+ }
appendSwiftLibDir(runtimeResourcePath, shared);
}

46 changes: 46 additions & 0 deletions pkgs/development/compilers/swift/compiler/patches/swift-wrap.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
Wrap the swift compiler produced during the build

--- a/tools/driver/CMakeLists.txt
+++ b/tools/driver/CMakeLists.txt
@@ -16,6 +16,13 @@ if(${LIBSWIFT_BUILD_MODE} MATCHES "BOOTSTRAPPING.*")
swiftDriverTool
libswiftStub)

+ # Nix: wrap the swift build.
+ add_custom_command(
+ TARGET swift-frontend-bootstrapping0 POST_BUILD
+ COMMAND nix-swift-make-swift-wrapper $<TARGET_FILE:swift-frontend-bootstrapping0>
+ VERBATIM
+ )
+
swift_create_post_build_symlink(swift-frontend-bootstrapping0
SOURCE "swift-frontend${CMAKE_EXECUTABLE_SUFFIX}"
DESTINATION "swiftc${CMAKE_EXECUTABLE_SUFFIX}"
@@ -34,6 +41,13 @@ if(${LIBSWIFT_BUILD_MODE} MATCHES "BOOTSTRAPPING.*")
swiftDriverTool
libswift-bootstrapping1)

+ # Nix: wrap the swift build.
+ add_custom_command(
+ TARGET swift-frontend-bootstrapping1 POST_BUILD
+ COMMAND nix-swift-make-swift-wrapper $<TARGET_FILE:swift-frontend-bootstrapping1>
+ VERBATIM
+ )
+
swift_create_post_build_symlink(swift-frontend-bootstrapping1
SOURCE "swift-frontend${CMAKE_EXECUTABLE_SUFFIX}"
DESTINATION "swiftc${CMAKE_EXECUTABLE_SUFFIX}"
@@ -50,6 +64,13 @@ target_link_libraries(swift-frontend
swiftDriverTool
libswift)

+# Nix: wrap the swift build.
+add_custom_command(
+ TARGET swift-frontend POST_BUILD
+ COMMAND nix-swift-make-swift-wrapper $<TARGET_FILE:swift-frontend>
+ VERBATIM
+)
+
# Create a `swift-driver` executable adjacent to the `swift-frontend` executable
# to ensure that `swiftc` forwards to the standalone driver when invoked.
swift_create_early_driver_copies(swift-frontend)
Loading