This project uses pre-commit to validate that certain files (e.g. fmf or python) are not broken. Please install pre-commit using pip install pre-commit as described here. Then navigate to your clone of this project and install the git hook scripts using cd ~/llvm-snapshots && pre-commit install. This will run pre-commit on every git commit.
Konrad Kleine <[email protected]>
This document is supposed to give you an idea of what goes into maintaining the LLVM snapshots for Fedora. Don’t feel like you have read this document from top to bottom but instead consider it a reference. There is a Frequently Asked Questions (FAQ) section at the botton that probably contains many answers. Feel free to add your own there.
|
Warning
|
This page needs to be updated to reflect the latest updates. |
This is home for the generation of daily
-
-
See generate-snapshot-tarballs workflow
-
-
-
See fedora-copr-build workflow
-
For each LLVM subproject that we build as a snapshot, we have a branch called upstream-snapshot in the appropriate /rpms/<PACKAGE> directory on the Fedora Package Source. For clang the branch can be found here for example:
There’s a github actions workflow, that automatically builds the snapshots every night from the latest upstream LLVM source:
https://github.com/fedora-llvm-team/llvm-snapshots/blob/main/.github/workflows/fedora-copr-build.yml
This workflow contains copr CLI calls that are needed in order to:
-
Build the snapshots for today in a distinct
@fedora-llvm-team/llvm-snapshots-incubator-<YYYYMMDD>NoteReplace <YYYYMMDD>with a date in reversed year month day order (e.g.20230223). -
Take the snapshots from yesterday and make them available ("fork" in Copr) under @fedora-llvm-team/llvm-snapshots, if all builds succeeded.
When you repeat these two steps every day, you get a project for each day with with @fedora-llvm-team/llvm-snapshots "pointing to" the build from yesterday.
Building LLVM takes a lot of time which is why our snapshots are usually a day behind the main branch of upstream LLVM.
In the @fedora-llvm-team/llvm-snapshots project, you’ll never see failing builds, because the "fork" mechanism in Copr only forks successful builds. To see why a build is missing you have to go to the individual project for a particular day.
Go to the Fedora Copr project and look for anything red in the monitor for today (see How do I find the monitor?).
If many things are red in one line, it’s possibly due to a shared problem, e.g. with a patch not being applicable or something alike.
Also, if a column (a combination of an arch and OS) is mostly red, it’s probably
due to a problem with depending packages not being built. For example if clang
is red and llvm is as well, then most likely llvm not building is the reason
why clang is red as well. Having said that, you should always look at the
package that builds first. Open up the spec file of a package to find out if it
depends on another one. Make sure you look at a spec-file in the
upstream-snapshot branch. For LLVM that would be
llvm.spec.
Suppose you want to work on the compiler-rt package with nothing checkout locally, yet.
-
Export your Fedora Account System username so you can copy and paste the commands below:
$ export FAS_USER=<YOUR_USERMAME>
NoteThis is optional and only for copy pasting the below commands. Use the username to login in here: https://id.fedoraproject.org/login. -
Clone the original source code’s
upstream-snapshotbranch and navigate to that directory.$ fedpkg clone compiler-rt -b upstream-snapshot ~/dev/llvm-rpms/compiler-rt-snapshot $ cd ~/dev/llvm-rpms/compiler-rt-snapshot
NoteI have a directory called ~/dev/llvm-rpmswith a distinct directory for the different LLVM RPMs (e.g.~/dev/llvm-rpms/clang-rhel,~/dev/llvm-rpms/clang-rawhide,~/dev/llvm-rpms/clang-centos,~/dev/llvm-rpms/clang-snapshot). I’m going to use that structuring below as well. Over time I found this very useful to quickly use file diffs and peek into other branches without usinggitall the time but maybemeld. -
Go to https://src.fedoraproject.org/rpms/compiler-rt and click the "fork" button in the top-right corner to create fork just for you. Then add it as a remote:
$ git remote add $FAS_USER \ ssh://[email protected]/forks/$FAS_USER/rpms/compiler-rt.gitYou should have two git remote now:
originand one that is named after your FAS login. -
Make your changes to
compiler-rtand commit them locally.$ git add -p $ git commit -m "My changes"
-
Push your local changes to a branch of your liking (e.g.
mybranch) in your fork$ git push $FAS_USER HEAD:mybranch
-
Create a pull-requst for your changes so they can go into the
upstream-snapshotbranch by executing this and heading over to your browser:$ xdg-open https://src.fedoraproject.org/fork/$FAS_USER/rpms/compiler-rt/diff/upstream-snapshot..mybranch
-
Wait for your changes to be approved and merged. Maybe ping someone from the team.
Then your changes will automatically be taken into account for the next daily build.
-
If you want to kick-off a build yourself, you can run:
$ copr build-package \ --name compiler-rt \ -r fedora-rawhide-x86_64 \ --timeout $((30*3600)) \ @fedora-llvm-team/llvm-snapshots-incubator-`date +%Y%m%d`This will build the
compiler-rtpackage with your changes as soon as they landed in theupstream-snapshotbranch in today’s Copr project.
| upstream |
I have the $ git clone \ --origin upstream \ --branch main \ [email protected]:llvm/llvm-project.git \ ~/llvm-project This ensures the upstream work is tracked under the |
||
| fedora |
Then you need to add another remote
called $ cd ~/llvm-project $ git remote add fedora ssh://[email protected]/llvm-project.git
|
I have each LLVM subproject file project (e.g. clang) cloned with the appropriate tool (e.g. fedpkg, centpkg and rhpkg).
$ fedpkg clone clang -b rawhide ~/dev/llvm-rpms/clang-rawhide #(1) $ fedpkg clone clang -b upstream-snapshot ~/dev/llvm-rpms/clang-snapshot #(2) $ centpkg clone clang -b c9s ~/dev/llvm-rpms/clang-centos #(3) $ rhpkg clone clang -b rhel-9-main ~/dev/llvm-rpms/clang-rhel #(4)
-
This is for the regular fedora work on rawhide.
-
This is for the work on the LLVM snapshots.
-
This is for the work on CentOS stream.
-
This is for the internal work on RHEL.
Every now and then you’ll find out that the rawhide branch of a package
contains commits that you don’t have yet in the upstream-snapshot branch.
That’s when you need to merge the rawhide branch into the upstream-snapshot
branch. DO NOT REBASE!. This is how you can do it for clang as an example:
$ cd ~/dev/llvm-rpms/clang-snapshot $ git fetch $ git merge \ --no-ff \#(1) --summary \ --log origin/rawhide #(2) $ vim clang.spec #(3) $ git add clang.spec #(4) $ git merge --continue #(5) $ git push origin HEAD:upstream-snapshot #(6)
-
The
--no-ffprevents any rebasing to happen, which is desireable here. I understand that it is different for feature development in most other projects. But this is different. -
The
--logwill add information to the commit message about what commits fromrawhidewere merged into theupstream-snapshotbranch. -
Resolve conflicts that happen when merging.
-
Add the files that had conflicts when merging.
-
Continue the merge
-
Push the merged state back to the
upstream-snapshotbranch.
|
Important
|
This will ensure that you’ll keep the complete history of the
upstream-snapshot branch which is very important. Trust me! Sometimes it can
be quite confusing to not know if a patch is new or already in upstream and
you’re wondering if you removed it before. All of this information would be lost
if you rebased instead of merging. The other benefit is that you just have to
deal with conflicts of the final revision and no every patch that exists
downstream.
|
| source-tarball |
Every night at 00:00 am we build a source tarball using the
|
| copr-builds |
The Fedora Copr builds are controlled by the fedora-copr-build.yml. This runs at 00:45am every night. This gives the |
Unfortunately you cannot run fedpkg prep locally in order to check why a patch cannot be applied in Copr. This is because we’re relying on a rather nebulous beature: the spec file evaluation of the Version:-tag with custom lua macros applied.
In case Fedora Copr tells you that a patch is not applicable, you probably want
to check if the patch is already in the
upstream/main branch.
If the patch has already landed upstream, then you can remove the corresponding RPM Patch
tag from the <project>.spec file and also git rm -f <mypatch>.patch from the
project’s git repo.
If the patch hasn’t landed upstream, then you probably need to update the patch.
Navigate to your llvm-project clone and see if the patch exists in the
fedora/streamline branch. Sometimes package
maintainers are unaware of this branch and add their patches to the project’s
spec file right away. And that’s perfectly fine. We can cope with that.
Update the fedora/streamline branch by rebasing
onto the latest changes from upstream/main.
$ cd ~/llvm-project $ git fetch fedora $ git fetch upstream $ # You don't need the -b and the --track if you already have this branch $ git checkout -b streamline --track fedora/streamline $ git rebase upstream/main #(1) ...potentially resolve rebasing conflicts... $ git push -f fedora HEAD:streamline #(2)
-
We don’t want to merge here because of the way we generate patches from the
fedora/streamlinebranch. We usegit format-patchto generate the patches and any resolved conflicts in a merge commit won’t be picked up by it. Again, trust me. I’ve spend hours finding out why a change wasn’t picked up bygit format-patchand it was simply becuase of merge commits. -
You have to force push
-fand you need to be careful not to overwrite somebody else’s changes that happened in between.
Now that the fedora/streamline branch is up to
date, take the patch file from the RPM project’s directory and copy it to the
llvm-project’s root dir. Here’s an example of how I did that with clang
today:
$ cd ~/dev/llvm-rpms/clang-snapshot $ cp 0006-PATCH-Driver-Add-a-gcc-equivalent-triple-to-the-list.patch ~/llvm-project $ cd ~/llvm-project $ git checkout streamline $ git am 0006-PATCH-Driver-Add-a-gcc-equivalent-triple-to-the-list.patch
You might need to resolve conflicts and then do git am --continue. But after
that the patch is now in the fedora/streamline
branch.
Don’t forget to push the changes back:
$ git push -f fedora HEAD:streamline
Now continue with: How to generate patch files that go into the specfile?
I’ll show you how to generate the patch files for the clang package. This is
especially interesting because this package consumes two tarballs, one for
clang and one for clang-tools-extra. Yet, the clang.spec file has just one
list of patch files. The question is how to delegate a portion of this list of
patches to the clang tarball and the rest to the clang-tools-extra tarball.
For this, we have to begin by generating patch files for each sub-project
individually even though the original patches in the
fedora/streamline branch might be touching both
projects at once.
$ cd ~/llvm-project
$ git fetch upstream
$ git fetch fedora
$ git checkout streamline
$ rm *.patch #(1)
$ git format-patch --keep-subject upstream/main..HEAD -- clang #(2)
0001-Reorganize-gtest-integration.patch
0002-ToolChain-Add-lgcc_s-to-the-linker-flags-when-using-.patch
0003-Make-funwind-tables-the-default-on-all-archs.patch
0004-Don-t-install-static-libraries.patch
0005-Prefer-gcc-toolchains-with-libgcc_s.so-when-not-stat.patch
0006-Driver-Add-a-gcc-equivalent-triple-to-the-list-of-tr.patch
0007-Work-around-gcc-miscompile.patch
0008-cmake-Allow-shared-libraries-to-customize-the-soname.patch
0009-Revert-replace-clang-LLVM_ENABLE_PLUGINS-CLANG_PLUGI.patch
$ git format-patch --keep-subject upstream/main..HEAD -- clang-tools-extra #(3)
0001-Revert-replace-clang-LLVM_ENABLE_PLUGINS-CLANG_PLUGI.patch
0002-Revert-Reland-enable-plugins-for-clang-tidy.patch
$ mv -v {0001,0201}-Revert-replace-clang-LLVM_ENABLE_PLUGINS-CLANG_PLUGI.patch #(4)
$ mv -v {0002,0202}-Revert-Reland-enable-plugins-for-clang-tidy.patch
-
Remove all left-over patch files
-
Generate patches for
clangthat go ontop ofupstream/main. -
Generate patches for
clang-tools-extrathat go ontop ofupstream/main. -
These two steps exist just to make it match up with the
Patch201:andPatch202:tags in the spec file.
Now move those files over to the RPM project directory:
$ cd ~/dev/llvm-rpms/clang-snapshot $ mv ~/llvm-project/*.patch .
Weave those patches in the spec file and make sure you use Patch tags with
numbers higher or equal to 200 for the patches targeting clang-tools-extra.
|
Note
|
Look for
%autopatch
in the clang.spec to find out how patch tags are applied to different
tarballs.
|
Now push the changes back to the
upstream/upstream-snapshot branch:
$ git push origin HEAD:upstream-snapshot #(1)
-
You might need to force (
-f) push here.
You can find the snapshot monitor for LLVM Fedora builds on Copr here:
The above link brings you to the latest "forked" build. It will only contain successful builds.
To find out where this build came from, take a look at the title of the project. There it should say something like:
( forked from @fedora-llvm-team/llvm-snapshots-incubator-20230221 )
Go to the project from which @fedora-llvm-team/llvm-snapshots was forked to find failing builds.
As described in the overview the monitor @fedora-llvm-team/llvm-snapshots Copr project is always reflecting the state of yesterday. The nice benefit is that if a build fails today, you have one day to fix things before s*** hits the fan.
This is slightly more advanced but helpful if you need to fix build errors locally.
# Enable the llvm-snapshot-builder repo and install the llvm-snapshot-builder package
# This is needed because the *.spec files of the repos use special macros provided by
# this package. We need it on the host and in mock unfortunately. On the host this is
# needed to download the source with spectool.
# NOTE: This only needs to be done once and NOT for every package.
dnf install -y 'dnf-command(copr)'
dnf copr enable -y @fedora-llvm-team/llvm-snapshot-builder
dnf install -y llvm-snapshot-builder
# Make sure you have an rpm tree, because temporary files may be placed there.
rpmdev-setuptree -d
# Checkout project into buildroot and fetch the sources and patches
fedpkg clone -b upstream-snapshot clang /tmp/workdir/buildroot
cd /tmp/workdir/buildroot
spectool -g *.spec
# Install llvm-snapshot-builder into chroot
mock \
-r fedora-36-x86_64 \
--addrepo https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshot-builder/fedora-36-x86_64/ \
--install llvm-snapshot-builder
# Build with mock
fedpkg \
--release f36 \
mockbuild -N \
-- \
--addrepo https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots/fedora-36-x86_64/devel
# Install vim (optionally)
mock -r fedora-36-x86_64 --install vim
# Open a shell in the mock buildroot
fedpkg --release f36 mockbuild --shell
We also have a Makefile in case we encounter an error with the snapshots and
want to rebuild locally to fix errors. These are the make targets to choose from:
- clone-%
-
Clones the upstream-snapshot branch of the given package package (%) into the buildroot.
- build-%
-
Clones and builds the package (%) and then installs it in the chroot.
- init-mock
-
Initializes the mock chroot.
- build-and-install-%
-
For the package (%) an SRPM and an RPM is built and then it is installed in the chroot.
- shell
-
Opens up a shell to inspect the mock chroot.
- install-vim
-
Allows you to use vim inside of mock.
- clean-mock
-
Cleans the mock chroot
- clean-buildroot
-
Removes the buildroot directory
- clean
-
Cleans the mock chroot and removes the buildroot.
- clean-%
-
Removes the buildroot dir for the given package (%).
- copr-build-%
-
Builds the package (%) in copr by using the tooling used for the automated snapshot generation.
- help
-
Display this help text.
- help-html
-
Display this help text as an HTML definition list for better documentation generation
- help-adoc
-
Display this help text as an ASCIDoc definition list for better documentation generation