233 lines
9.8 KiB
Markdown
233 lines
9.8 KiB
Markdown
# Updating LLVM
|
|
|
|
<!-- toc -->
|
|
|
|
<!-- date-check: Aug 2024 -->
|
|
Rust supports building against multiple LLVM versions:
|
|
|
|
* Tip-of-tree for the current LLVM development branch is usually supported
|
|
within a few days. PRs for such fixes are tagged with `llvm-main`.
|
|
* The latest released major version is always supported.
|
|
* The one or two preceding major versions are usually supported.
|
|
|
|
By default, Rust uses its own fork in the [rust-lang/llvm-project repository].
|
|
This fork is based on a `release/$N.x` branch of the upstream project, where
|
|
`$N` is either the latest released major version, or the current major version
|
|
in release candidate phase. The fork is never based on the `main` development
|
|
branch.
|
|
|
|
Our LLVM fork only accepts:
|
|
|
|
* Backports of changes that have already landed upstream.
|
|
* Workarounds for build issues affecting our CI environment.
|
|
|
|
With the exception of one grandfathered-in patch for SGX enablement, we do not
|
|
accept functional patches that have not been upstreamed first.
|
|
|
|
There are three types of LLVM updates, with different procedures:
|
|
|
|
* Backports while the current major LLVM version is supported.
|
|
* Backports while the current major LLVM version is no longer supported (or
|
|
the change is not eligible for upstream backport).
|
|
* Update to a new major LLVM version.
|
|
|
|
## Backports (upstream supported)
|
|
|
|
While the current major LLVM version is supported upstream, fixes should be
|
|
backported upstream first, and the release branch then merged back into the
|
|
Rust fork.
|
|
|
|
1. Make sure the bugfix is in upstream LLVM.
|
|
2. If this hasn't happened already, request a backport to the upstream release
|
|
branch. If you have LLVM commit access, follow the [backport process].
|
|
Otherwise, open an issue requesting the backport. Continue once the
|
|
backport has been approved and merged.
|
|
3. Identify the branch that rustc is currently using. The `src/llvm-project`
|
|
submodule is always pinned to a branch of the
|
|
[rust-lang/llvm-project repository].
|
|
4. Fork the rust-lang/llvm-project repository.
|
|
5. Check out the appropriate branch (typically named `rustc/a.b-yyyy-mm-dd`).
|
|
6. Add a remote for the upstream repository using
|
|
`git remote add upstream https://github.com/llvm/llvm-project.git` and
|
|
fetch it using `git fetch upstream`.
|
|
7. Merge the `upstream/release/$N.x` branch.
|
|
8. Push this branch to your fork.
|
|
9. Send a Pull Request to rust-lang/llvm-project to the same branch as before.
|
|
Be sure to reference the Rust and/or LLVM issue that you're fixing in the PR
|
|
description.
|
|
10. Wait for the PR to be merged.
|
|
11. Send a PR to rust-lang/rust updating the `src/llvm-project` submodule with
|
|
your bugfix. This can be done locally with `git submodule update --remote
|
|
src/llvm-project` typically.
|
|
12. Wait for PR to be merged.
|
|
|
|
An example PR:
|
|
[#59089](https://github.com/rust-lang/rust/pull/59089)
|
|
|
|
## Backports (upstream not supported)
|
|
|
|
Upstream LLVM releases are only supported for two to three months after the
|
|
GA release. Once upstream backports are no longer accepted, changes should be
|
|
cherry-picked directly to our fork.
|
|
|
|
1. Make sure the bugfix is in upstream LLVM.
|
|
2. Identify the branch that rustc is currently using. The `src/llvm-project`
|
|
submodule is always pinned to a branch of the
|
|
[rust-lang/llvm-project repository].
|
|
3. Fork the rust-lang/llvm-project repository.
|
|
4. Check out the appropriate branch (typically named `rustc/a.b-yyyy-mm-dd`).
|
|
5. Add a remote for the upstream repository using
|
|
`git remote add upstream https://github.com/llvm/llvm-project.git` and
|
|
fetch it using `git fetch upstream`.
|
|
6. Cherry-pick the relevant commit(s) using `git cherry-pick -x`.
|
|
7. Push this branch to your fork.
|
|
8. Send a Pull Request to rust-lang/llvm-project to the same branch as before.
|
|
Be sure to reference the Rust and/or LLVM issue that you're fixing in the PR
|
|
description.
|
|
9. Wait for the PR to be merged.
|
|
10. Send a PR to rust-lang/rust updating the `src/llvm-project` submodule with
|
|
your bugfix. This can be done locally with `git submodule update --remote
|
|
src/llvm-project` typically.
|
|
11. Wait for PR to be merged.
|
|
|
|
An example PR:
|
|
[#59089](https://github.com/rust-lang/rust/pull/59089)
|
|
|
|
## New LLVM Release Updates
|
|
|
|
<!-- date-check: Jul 2023 -->
|
|
|
|
Unlike bugfixes,
|
|
updating to a new release of LLVM typically requires a lot more work.
|
|
This is where we can't reasonably cherry-pick commits backwards,
|
|
so we need to do a full update.
|
|
There's a lot of stuff to do here,
|
|
so let's go through each in detail.
|
|
|
|
1. LLVM announces that its latest release version has branched.
|
|
This will show up as a branch in the [llvm/llvm-project repository],
|
|
typically named `release/$N.x`,
|
|
where `$N` is the version of LLVM that's being released.
|
|
|
|
1. Create a new branch in the [rust-lang/llvm-project repository]
|
|
from this `release/$N.x` branch,
|
|
and name it `rustc/a.b-yyyy-mm-dd`,
|
|
where `a.b` is the current version number of LLVM in-tree
|
|
at the time of the branch,
|
|
and the remaining part is the current date.
|
|
|
|
1. Apply Rust-specific patches to the llvm-project repository.
|
|
All features and bugfixes are upstream,
|
|
but there's often some weird build-related patches
|
|
that don't make sense to upstream.
|
|
These patches are typically the latest patches in the
|
|
rust-lang/llvm-project branch that rustc is currently using.
|
|
|
|
1. Build the new LLVM in the `rust` repository.
|
|
To do this,
|
|
you'll want to update the `src/llvm-project` repository to your branch,
|
|
and the revision you've created.
|
|
It's also typically a good idea to update `.gitmodules` with the new
|
|
branch name of the LLVM submodule.
|
|
Make sure you've committed changes to
|
|
`src/llvm-project` to ensure submodule updates aren't reverted.
|
|
Some commands you should execute are:
|
|
|
|
* `./x build src/llvm-project` - test that LLVM still builds
|
|
* `./x build` - build the rest of rustc
|
|
|
|
You'll likely need to update [`llvm-wrapper/*.cpp`][`llvm-wrapper`]
|
|
to compile with updated LLVM bindings.
|
|
Note that you should use `#ifdef` and such to ensure
|
|
that the bindings still compile on older LLVM versions.
|
|
|
|
Note that `profile = "compiler"` and other defaults set by `./x setup`
|
|
download LLVM from CI instead of building it from source.
|
|
You should disable this temporarily to make sure your changes are being used.
|
|
This is done by having the following setting in `config.toml`:
|
|
|
|
```toml
|
|
[llvm]
|
|
download-ci-llvm = false
|
|
```
|
|
|
|
1. Test for regressions across other platforms. LLVM often has at least one bug
|
|
for non-tier-1 architectures, so it's good to do some more testing before
|
|
sending this to bors! If you're low on resources you can send the PR as-is
|
|
now to bors, though, and it'll get tested anyway.
|
|
|
|
Ideally, build LLVM and test it on a few platforms:
|
|
|
|
* Linux
|
|
* macOS
|
|
* Windows
|
|
|
|
Afterwards, run some docker containers that CI also does:
|
|
|
|
* `./src/ci/docker/run.sh wasm32`
|
|
* `./src/ci/docker/run.sh arm-android`
|
|
* `./src/ci/docker/run.sh dist-various-1`
|
|
* `./src/ci/docker/run.sh dist-various-2`
|
|
* `./src/ci/docker/run.sh armhf-gnu`
|
|
|
|
1. Prepare a PR to `rust-lang/rust`. Work with maintainers of
|
|
`rust-lang/llvm-project` to get your commit in a branch of that repository,
|
|
and then you can send a PR to `rust-lang/rust`. You'll change at least
|
|
`src/llvm-project` and will likely also change [`llvm-wrapper`] as well.
|
|
|
|
<!-- date-check: mar 2025 -->
|
|
> For prior art, here are some previous LLVM updates:
|
|
> - [LLVM 17](https://github.com/rust-lang/rust/pull/115959)
|
|
> - [LLVM 18](https://github.com/rust-lang/rust/pull/120055)
|
|
> - [LLVM 19](https://github.com/rust-lang/rust/pull/127513)
|
|
> - [LLVM 20](https://github.com/rust-lang/rust/pull/135763)
|
|
|
|
Note that sometimes it's easiest to land [`llvm-wrapper`] compatibility as a PR
|
|
before actually updating `src/llvm-project`.
|
|
This way,
|
|
while you're working through LLVM issues,
|
|
others interested in trying out the new LLVM can benefit from work you've done
|
|
to update the C++ bindings.
|
|
|
|
1. Over the next few months,
|
|
LLVM will continually push commits to its `release/a.b` branch.
|
|
We will often want to have those bug fixes as well.
|
|
The merge process for that is to use `git merge` itself to merge LLVM's
|
|
`release/a.b` branch with the branch created in step 2.
|
|
This is typically
|
|
done multiple times when necessary while LLVM's release branch is baking.
|
|
|
|
1. LLVM then announces the release of version `a.b`.
|
|
|
|
1. After LLVM's official release,
|
|
we follow the process of creating a new branch on the
|
|
rust-lang/llvm-project repository again,
|
|
this time with a new date.
|
|
It is only then that the PR to update Rust to use that version is merged.
|
|
|
|
The commit history of `rust-lang/llvm-project`
|
|
should look much cleaner as a `git rebase` is done,
|
|
where just a few Rust-specific commits are stacked on top of stock LLVM's release branch.
|
|
|
|
### Caveats and gotchas
|
|
|
|
Ideally the above instructions are pretty smooth, but here's some caveats to
|
|
keep in mind while going through them:
|
|
|
|
* LLVM bugs are hard to find, don't hesitate to ask for help!
|
|
Bisection is definitely your friend here
|
|
(yes LLVM takes forever to build, yet bisection is still your friend).
|
|
Note that you can make use of [Dev Desktops],
|
|
which is an initiative to provide the contributors with remote access to powerful hardware.
|
|
* If you've got general questions, [wg-llvm] can help you out.
|
|
* Creating branches is a privileged operation on GitHub, so you'll need someone
|
|
with write access to create the branches for you most likely.
|
|
|
|
|
|
[rust-lang/llvm-project repository]: https://github.com/rust-lang/llvm-project
|
|
[llvm/llvm-project repository]: https://github.com/llvm/llvm-project
|
|
[`llvm-wrapper`]: https://github.com/rust-lang/rust/tree/master/compiler/rustc_llvm/llvm-wrapper
|
|
[wg-llvm]: https://rust-lang.zulipchat.com/#narrow/stream/187780-t-compiler.2Fwg-llvm
|
|
[Dev Desktops]: https://forge.rust-lang.org/infra/docs/dev-desktop.html
|
|
[backport process]: https://llvm.org/docs/GitHub.html#backporting-fixes-to-the-release-branches
|