Only use `DIST_TRY_BUILD` for try jobs that were not selected explicitly
Some CI jobs (x64 Linux, ARM64 Linux and x64 MSVC) use the `opt-dist` tool to build an optimized toolchain using PGO and BOLT. When performing a default try build for x64 Linux, in most cases we want to run perf. on that artifact. To reduce the latency of this common use-case, `opt-dist` skips building several components not needed for perf., and it also skips running post-optimization tests, when it detects that the job is executed as a try job (not a merge/auto job).
This is useful, but it also means that if you *want* to run the tests, you had to go to `jobs.yml` and manually comment this environment variable, create a WIP commit, do a try build, and then remove the WIP commit, which is annoying (in the similar way that modifying what gets run in try builds was annoying before we had the `try-job` annotations).
I thought that we could introduce some additional PR description marker like `try-job-run-tests`, but it's hard to discover that such things exist.
Instead, I think that there's a much simpler heuristic for determining whether `DIST_TRY_BUILD` should be used (that I implemented in this PR):
- If you do just ``@bors` try`, without any custom try jobs selected, `DIST_TRY_BUILD` will be activated, to finish the build as fast as possible.
- If you specify any custom try jobs, you are most likely doing experiments and you want to see if tests pass and everything builds as it should. The `DIST_TRY_BUILD` variable will thus *not* be set in this case.
In this way, if you want to run dist tests, you can just add the `try-job: dist-x86_64-linux` line to the PR description, and you don't need to create any WIP commits.
r? `@marcoieni`
(https://github.com/rust-lang/rust),
`ErrorGuaranteed` was replaced by fatal errors.
As a result, `tcx.analysis()` now aborts directly
instead of returning an error guard.
To accommodate this change, this update replaces
`tcx.analysis()` with `typeck()`
to perform type checking in the example.
Continuing the work from #137350.
Removes the unused methods: `expect_variant`, `expect_field`,
`expect_foreign_item`.
Every method gains a `hir_` prefix.
Retire the legacy `Makefile`-based `run-make` test infra
The final piece of [porting run-make tests to use Rust #121876](https://github.com/rust-lang/rust/issues/121876).
Closes#121876.
Closes#40713.
Closes#81791 (no longer using `wc`).
Closes#56475 (no longer a problem in current form of that test; we don't ignore the test on `aarch64-unknown-linux-gnu`).
### Summary
This PR removes the legacy `Makefile`-based `run-make` test infra which has served us well over the years. The legacy infra is no longer needed since we ported all of `Makefile`-based `run-make` tests to the new `rmake.rs` infra.
Additionally, this PR:
- Removes `tests/run-make/tools.mk` since no more `Makefile`-based tests remain.
- Updates `tests/run-make/README.md` and rustc-dev-guide docs to remove mention about `Makefile`-based `run-make` tests
- Update test suite requirements in rustc-dev-guide on Windows to no longer need MSYS2 (they should also now run successfully on native Windows MSVC).
- Update `triagebot.toml` to stop backlinking to #121876.
**Thanks to everyone who helped in this effort to modernize the `run-make` test infra and test suite!**
r? bootstrap
Support raw-dylib link kind on ELF
raw-dylib is a link kind that allows rustc to link against a library without having any library files present.
This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker.
While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future.
This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time.
I was inspired by Björn's comments in https://internals.rust-lang.org/t/bundle-zig-cc-in-rustup-by-default/22096/27
Tracking issue: #135694
r? bjorn3
try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-msvc-2
try-job: test-various
Rewrite the `ci.py` script in Rust
It would seem that I would learn by now that any script written in Python will become unmaintainable sooner or later, but alas..
r? `@marcoieni`
try-job: aarch64-gnu
try-job: dist-x86_64-linux-alt
try-job: x86_64-msvc-ext2
Fixes: https://github.com/rust-lang/rust/issues/137013
raw-dylib is a link kind that allows rustc to link against a library
without having any library files present.
This currently only exists on Windows. rustc will take all the symbols
from raw-dylib link blocks and put them in an import library, where they
can then be resolved by the linker.
While import libraries don't exist on ELF, it would still be convenient
to have this same functionality. Not having the libraries present at
build-time can be convenient for several reasons, especially
cross-compilation. With raw-dylib, code linking against a library can be
cross-compiled without needing to have these libraries available on the
build machine. If the libc crate makes use of this, it would allow
cross-compilation without having any libc available on the build
machine. This is not yet possible with this implementation, at least
against libc's like glibc that use symbol versioning.
The raw-dylib kind could be extended with support for symbol versioning
in the future.
This implementation is very experimental and I have not tested it very
well. I have tested it for a toy example and the lz4-sys crate, where it
was able to successfully link a binary despite not having a
corresponding library at build-time.
stabilize stage management for rustc tools
https://github.com/rust-lang/rust/pull/135990 got out of control due to excessive complexity. This PR aims to achieve the same goal with a simpler approach, likely through multiple smaller PRs. I will keep the other one read-only and open as a reference for future work.
This work stabilizes the staging logic for `ToolRustc` programs, so you no longer need to handle build and target compilers separately in steps. Previously, most tools didn't do this correctly, which was causing the compiler to be built twice (e.g., `x test cargo --stage 1` would compile the stage 2 compiler before, but now it only compiles the stage 1 compiler).
I also tried to document how we should write `ToolRustc` steps as they are quite different and require more attention than other tools.
Next goal is to stabilize how stages are handled for the rustc itself. Currently, `x build --stage 1` builds the stage 1 compiler which is fine, but `x build compiler --stage 1` builds stage 2 compiler.
~~for now, r? ghost~~
Added project-specific Zed IDE settings
This repository currently has project-specific VS Code IDE settings in `.vscode` and `compiler/rustc_codegen_cranelift/.vscode`. Now there are equivalent project-specific Zed IDE settings alongside those.
This fixes `rust-analyzer` not being able to properly handle this project.
Note that:
1. The contents of `src/tools/rust-analyzer/.vscode` could not be translated to Zed, as they aren't basic IDE settings.
2. One of the VS Code settings in `.vscode` has no corresponding setting in Zed, and so this has been noted like this:
```json
"_settings_only_in_vs_code_not_yet_in_zed": {
"git.detectSubmodulesLimit": 20
},
```
While there were comments indicating which nightly versions the examples
were tested with, those versions did not work for me: neither did the
examples compile, nor did they produce the expected output.
This commit fixes the compilation issues, using nightly-2025-02-13 for
all examples (previously the version differed between the examples) and,
in the case of the `rustc_driver` examples, also fixes the argument
passing: rustc ignores the first argument, so we need to pass the
filename as the second (otherwise we only get the help text printed).
Note that the `rustc-interface-getting-diagnostics.rs` example still
does not produce any output, which I assume is not how it is intended.
However, I don't know enough to fix it.
To avoid inconsistencies between the documented version and the actually
required version I've moved the version comment from the Markdown into
the Rust code where it hopefully won't be forgotten as easily.
Finally I've clarified in the examples' README that you also need to use
the proper nightly version when compiling the examples, not just when
running them.
Add profiling of bootstrap commands using Chrome events
Since we now have support for tracing in bootstrap, and the execution of most commands is centralized within a few functions, it's quite trivial to also trace command execution, and visualize it using the Chrome profiler. This can be helpful both to profile what takes time in bootstrap and also to get a visual idea of what happens in a given bootstrap invocation (since the execution of external commands is usually the most interesting thing).
This is how it looks:
I first tried to use [tracing-flame](https://github.com/tokio-rs/tracing/tree/master/tracing-flame), but the output wasn't very useful, because the event/stackframe names were bootstrap code locations, instead of the command contents.
r? ``@jieyouxu``
valtree performance tuning
Summary: This PR makes type checking of code with many type-level constants faster.
After https://github.com/rust-lang/rust/pull/136180 was merged, we observed a small perf regression (https://github.com/rust-lang/rust/pull/136318#issuecomment-2635562821). This happened because that PR introduced additional copies in the fast reject code path for consts, which is very hot for certain crates: 6c1d960d88/compiler/rustc_type_ir/src/fast_reject.rs (L486-L487)
This PR improves the performance again by properly interning the valtrees so that copying and comparing them becomes faster. This will become especially useful with `feature(adt_const_params)`, so the fast reject code doesn't have to do a deep compare of the valtrees.
Note that we can't just compare the interned consts themselves in the fast reject, because sometimes `'static` lifetimes in the type are be replaced with inference variables (due to canonicalization) on one side but not the other.
A less invasive alternative that I considered is simply avoiding copies introduced by https://github.com/rust-lang/rust/pull/136180 and comparing the valtrees it in-place (see commit: 9e91e50ac5 / perf results: https://github.com/rust-lang/rust/pull/136593#issuecomment-2642303245), however that was still measurably slower than interning.
There are some minor regressions in secondary benchmarks: These happen due to changes in memory allocations and seem acceptable to me. The crates that make heavy use of valtrees show no significant changes in memory usage.
Parallel-compiler-related cleanup
Parallel-compiler-related cleanup
I carefully split changes into commits. Commit messages are self-explanatory. Squashing is not recommended.
cc "Parallel Rustc Front-end" https://github.com/rust-lang/rust/issues/113349
r? SparrowLii
``@rustbot`` label: +WG-compiler-parallel
dev-guide: Link to `t-lang` procedures for new features
I was confused in https://github.com/rust-lang/rust/pull/136867, because while I did remember that such a procedure existed, but I couldn't seem to find it in the dev guide.
Improve documentation when adding a new target
https://github.com/rust-lang/rust/pull/133631#issuecomment-2607877936 shows that it can be a bit difficult process-wise to add a new target.
I've added a bit of text to the docs, suggesting that users add the target defintion/spec first, and later work on `std` support.
I also found that we have two places where we document how to add a new target. I've linked these for now, but they should probably be merged somehow in the future.
`@rustbot` label A-docs
r? compiler
CC `@workingjubilee` who's worked a lot on target specs IIRC.
Simplify and consolidate the way we handle construct `OutlivesEnvironment` for lexical region resolution
This is best reviewed commit-by-commit. I tried to consolidate the API for lexical region resolution *first*, then change the API when it was finally behind a single surface.
r? lcnr or reassign
Tidy Python improvements
Fixes display of Python formatting diffs in tidy, and refactors the code to make it simpler and more robust. Also documents Python formatting and linting in the Rustc dev guide.
Fixes: https://github.com/rust-lang/rust/issues/135942
r? `@onur-ozkan`
Implement `needs-subprocess` directive, and cleanup a bunch of tests to use `needs-{subprocess,threads}`
### Summary
Closes#128295.
- Implements `//@ needs-subprocess` directive in compiletest as requested in #128295. However, compiletest is a host tool, so we can't just try to spawn process because that spawns the process on *host*, not the *target*, under cross-compilation scenarios.
- The short-term solution is to add *Yet Another* list of allow-list targets.
- The long-term solution is to first check if a `$target` supports std, then try to run a binary to do run-time capability detection *on the target*. But that is tricky because you have to build-and-run a binary *for the target*.
- This PR picks the short-term solution, because the long-term solution is highly non-trivial, and it's already an improvement over individual `ignore-*`s all over the place.
- Opened an issue about the long-term solution in #135928.
- Documents `//@ needs-subprocess` in rustc-dev-guide.
- Replace `ignore-{wasm,wasm32,emscripten,sgx}` with `needs-{subprocess,threads}` where suitable in tests.
- Some drive-by test changes as I was trying to figure out if I could use `needs-{subprocess,threads}` and found some bits needlessly distracting.
Count of tests that use `ignore-{wasm,wasm32,emscripten,sgx}` before and after this PR:
| State | `ignore-sgx` | `ignore-wasm` | `ignore-emscripten` |
| - | - | - | - |
| Before this PR | 96 | 88 | 207 |
| After this PR | 36 | 38 | 61 |
<details>
<summary>Commands used to find out locally</summary>
```
--- before
[17:40] Joe:rust (fresh) | rg --no-ignore -l "ignore-sgx" tests | wc -l
96
[17:40] Joe:rust (fresh) | rg --no-ignore -l "ignore-wasm" tests | wc -l
88
[17:40] Joe:rust (fresh) | rg --no-ignore -l "ignore-emscripten" tests | wc -l
207
--- after
[17:39] Joe:rust (needs-subprocess-thread) | rg --no-ignore -l "ignore-sgx" tests | wc -l
36
[17:39] Joe:rust (needs-subprocess-thread) | rg --no-ignore -l "ignore-wasm" tests | wc -l
38
[17:39] Joe:rust (needs-subprocess-thread) | rg --no-ignore -l "ignore-emscripten" tests | wc -l
61
```
</details>
### Review advice
- Best reviewed commit-by-commit.
- Non-trivial test changes (not mechanically simple replacements) are split into individual commits to help with review. Their individual commit messages give some basic description of the changes.
- I *could* split some test changes out into another PR, but I found that I needed to change some tests to `needs-threads`, some to `needs-subprocess`, and some needed to use *both*, so they might conflict and become very annoying.
---
r? ``@ghost`` (need to run try jobs)
try-job: x86_64-msvc-1
try-job: i686-msvc-1
try-job: i686-mingw
try-job: x86_64-mingw-1
try-job: x86_64-apple-1
try-job: aarch64-apple
try-job: aarch64-gnu
try-job: test-various
try-job: armhf-gnu
remove pointless allowed_through_unstable_modules on TryFromSliceError
This got added in https://github.com/rust-lang/rust/pull/132482 but the PR does not explain why. `@lukas-code` do you still remember? Also Cc `@Noratrieb` as reviewer of that PR.
If I understand the issue description correctly, all paths under which this type is exported are stable now: `core::array::TryFromSliceError` and `std::array::TryFromSliceError`. If that is the case, we shouldn't have the attribute; it's a terrible hack that should only be used when needed to maintain backward compatibility. Getting some historic information right is IMO *not* sufficient justification to risk accidentally exposing this type via more unstable paths today or in the future.
Get rid of RunCompiler
The various `set_*` methods that have been removed can be replaced by setting the respective fields in the `Callbacks::config` implementation. `set_using_internal_features` was often forgotten and it's equivalent is now done automatically.
This documents how to determine which settings are used in CI, since I
see this question come up regularly. We currently don't have a great way
to answer the question, but at least there is something.
binarycat
Jakub Beránek
许杰友 Jieyou Xu (Joe)
xizheyin
Jieyou Xu
Matthias Krüger
bit-aloo
jyn
Yang Lin
Tshepang Mbambo
KonaeAkira
Josh Stone
Nicholas Nethercote
Oli Scherer
Deadbeef
moxian
bors
Zalathar
Noratrieb
Boxy
Matthias Krüger
onur-ozkan
Ed Page
Chai T. Rex
Florian Brucker
Martin Liska
yukang
Jubilee
Jacob Pratt
Lukas Markeffsky
Guillaume Gomez
Mads Marquart
Askar Safin
The rustc-dev-guide Cronjob Bot
Urgau
Michael Howell
MarcoIeni
Rehmatpal Singh
Yuki Okushi
Ryan Cumming
Michael Goulet
chiichen
yegeunyang
Joren-vanGoethem
Eric Huss
León Orell Valerian Liehr
Ada Alakbarova
bjorn3
Patrick Oliveira
lcnr
Ryan Mehri
Noah Lev
Ralf Jung
Samson
Max Heller
clubby789
Chris Bloodsworth
Connor Tsui
Orion Gonzalez
Keith Wansbrough
Camille Gillot
Taylor Cramer
Yutaro Ohno
Sam Estep