Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sccache #161

Open
wants to merge 1 commit into
base: branch/22.08
Choose a base branch
from
Open

Add sccache #161

wants to merge 1 commit into from

Conversation

tinywrkb
Copy link
Contributor

@tinywrkb tinywrkb commented Sep 8, 2022

While Flatpak Builder doesn't make it easy, it's possible to take advantage of sccache for local builds with local storage cache, but still have the packaging work correctly with Flathub CI.

  1. Move your sccache local cache folder under your ccache folder. Flatpak Builder already mounts the ccache folder onto /run/ccache, so our sccache folder will also be accessible.
  2. Move your sccache config into the sccache cache folder. The main setting you might want to have is the cache storage size.
# https://github.com/mozilla/sccache/blob/main/docs/Configuration.md
[cache.disk]
size = 3221225472 # 3GiB
  1. You should symlink $XDG_CACHE_HOME/sccache -> $CCACHE_DIR/sccache to not break this for the host, or update the relevant environment variables.
  2. Add these environment variables to your packaging manifest.
build-options:
  env:
    - SCCACHE_DIR=/run/ccache/sccache
    - SCCACHE_CONF=/run/ccache/sccache/config
  1. Add this module as the first one in your packaging manifest. Currently, a symlink doesn't work, so we need to copy the binary.
  - name: sccache
    buildsystem: simple
    build-commands:
      - |
        if [ "$CCACHE_DIR" == "/run/ccache" ]; then
          install -Dm755 /usr/lib/sdk/rust-stable/bin/sccache ${FLATPAK_DEST}/bin/rustc
        fi
    cleanup:
      - '*'

@flathubbot
Copy link
Contributor

Started test build 108930

@flathubbot
Copy link
Contributor

Build 108930 successful
To test this build, install it from the testing repository:

flatpak install --user https://dl.flathub.org/build-repo/106576/org.freedesktop.Sdk.Extension.rust-stable.flatpakref

@tinywrkb
Copy link
Contributor Author

tinywrkb commented Sep 8, 2022

Oh! And instead of copying the binary, you can probably do something like this:

build-options:
  env:
    - RUSTC_WRAPPER=$(if [ "$CCACHE_DIR" == "/run/ccache" ]; then echo /usr/lib/sdk/rust-stable/bin/sccache; fi;)
...
build-commands:
 - eval RUSTC_WRAPPER="${RUSTC_WRAPPER}" cargo fetch
 - eval RUSTC_WRAPPER="${RUSTC_WRAPPER}" cargo build --release

Or even take this another step and set environment variables globally for Cargo, instead of per-module with module-specific paths.

build-options:
  env:
    - CARGO_VARS=
      HOME=${FLATPAK_BUILDER_BUILDDIR}
      CARGO_HOME=${FLATPAK_BUILDER_BUILDDIR}/cargo
      RUSTC_WRAPPER=$(if [ "$CCACHE_DIR" == "/run/ccache" ]; then echo /usr/lib/sdk/rust-stable/bin/sccache; fi;)
...
build-commands:
 - eval ${CARGO_VARS} cargo fetch
 - eval ${CARGO_VARS} cargo build --release

@tinywrkb
Copy link
Contributor Author

tinywrkb commented Sep 8, 2022

Forgot to mention why in the last example I set HOME and why you don't see the usual Cargo command options.
Setting HOME=${FLATPAK_BUILDER_BUILDDIR} allows you to have a second Cargo config file, ${FLATPAK_BUILDER_BUILDDIR}/.cargo/config, in addition to ${CARGO_HOME}/config which is added by flatpak-cargo-generator.

Here's one example (requires GCC 12.1+):

[build]
target = "x86_64-unknown-linux-gnu"

[target.x86_64-unknown-linux-gnu]
rustflags = [
  "-C", "link-arg=-fuse-ld=mold"
]
linker = "x86_64-unknown-linux-gnu-gcc"

[net]
offline = true

[term]
verbose = true

Another example, if you have a musl-based toolchain, and you're statically linking applications (of course not going to work with this extension, but will with this one):

[build]
target = "x86_64-unknown-linux-musl"

[target.x86_64-unknown-linux-musl]
rustflags = [
  "-C", "target-feature=+crt-static",
  "-C", "link-arg=-fuse-ld=mold",
  "-C", "link-arg=-l:libc.a",
  "-C", "link-arg=-l:libgcc.a"
]
linker = "x86_64-linux-musl-gcc"

[net]
offline = true

[term]
verbose = true

@tinywrkb
Copy link
Contributor Author

Any opinion about this? I already have my own sccache extension, so I can deal without this, but I would prefer to avoid adding code that optionally depends on private SDK extension to Flathub packaging of applications.

@alatiera
Copy link
Member

alatiera commented Jan 6, 2023

Hi, thanks for the PR! Sorry I didn't saw it sooner.

Looks like good stuff, and we can merge it. Could you rebase the branch and update to the latest releases if any?

Would also be nice to move the setup documentation from this PR comments into some readme file for how to do this.

And I am assuming sccache will only get used if you've setup the wrapper and env vars and it won't affect existing projects, right?

@tytan652 tytan652 mentioned this pull request Apr 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants