Perform deeper compiletest path normalization for `$TEST_BUILD_DIR` to account for compare-mode/debugger cases, and normalize long type file filename hashes
Fixes#136510.
### Summary
- Whereas previously `$TEST_BUILD_DIR` is a normalization of `/path/to/build/test/<test_suite_name>/`, we now more deeply normalize. `$TEST_BUILD_DIR` now becomes a normalization of `/path/to/build/test/<test_suite_name>/<subdirs>/$name.$revision.$compare_mode.$debugger/` to normalize away path name differences when `--compare-mode` and/or `--debugger` are specified.
- We also centralize the normalization of long type name hashes
cf. https://github.com/rust-lang/rust/pull/136328#discussion_r1936760908.
### Review advice
- Best reviewed commit-by-commit.
- Split into 3 commits:
- **Commit 1**: compiletest changes to have `$TEST_BUILD_DIR` more deeply normalize.
- **Commit 2**: remove per-test hacks for long type path hash normalizations, and rebless tests *specifically* affected by that.
- **Commit 3**: rebless other tests that were changed as a side-effect of deeper `$TEST_BUILD_DIR` normalizations.
**Commit 2** is created via first finding tests that try to perform long type file hash normalizations on an ad hoc, per-test basis:
```
rg --no-ignore -l --no-ignore -F -e "long-type" tests/ui/**/*.rs
```
<details>
<summary>Tests with ad hoc long-type hash normalizations</summary>
```
tests/ui/type_length_limit.rs
tests/ui/traits/on_unimplemented_long_types.rs
tests/ui/regions/issue-102374.rs
tests/ui/recursion/recursion.rs
tests/ui/recursion/issue-83150.rs
tests/ui/recursion/issue-23122-2.rs
tests/ui/methods/inherent-bound-in-probe.rs
tests/ui/issues/issue-67552.rs
tests/ui/issues/issue-37311-type-length-limit/issue-37311.rs
tests/ui/issues/issue-20413.rs
tests/ui/issues/issue-8727.rs
tests/ui/infinite/infinite-instantiation.rs
tests/ui/infinite/infinite-instantiation-struct-tail-ice-114484.rs
tests/ui/higher-ranked/trait-bounds/hrtb-doesnt-borrow-self-1.rs
tests/ui/higher-ranked/trait-bounds/hrtb-doesnt-borrow-self-2.rs
tests/ui/higher-ranked/trait-bounds/hang-on-deeply-nested-dyn.rs
tests/ui/error-codes/E0275.rs
tests/ui/diagnostic-width/secondary-label-with-long-type.rs
tests/ui/diagnostic-width/long-e0277.rs
tests/ui/diagnostic-width/non-copy-type-moved.rs
tests/ui/diagnostic-width/long-E0308.rs
tests/ui/diagnostic-width/E0271.rs
tests/ui/diagnostic-width/binop.rs
```
</details>
These ad hoc normalizations were removed, and they are reblessed.
r? `````@lqd`````
Make `ptr_cast_add_auto_to_object` lint into hard error
In Rust 1.81, we added a FCW lint (including linting in dependencies) against pointer casts that add an auto trait to dyn bounds. This was part of work making casts of pointers involving trait objects stricter, and was part of the work needed to restabilize trait upcasting.
We considered just making this a hard error, but opted against it at that time due to breakage found by crater. This breakage was mostly due to the `anymap` crate which has been a persistent problem for us.
It's now a year later, and the fact that this is not yet a hard error is giving us pause about stabilizing arbitrary self types and `derive(CoercePointee)`. So let's see about making a hard error of this.
r? ghost
cc ```@adetaylor``` ```@Darksonn``` ```@BoxyUwU``` ```@RalfJung``` ```@compiler-errors``` ```@oli-obk``` ```@WaffleLapkin```
Related:
- https://github.com/rust-lang/rust/pull/135881
- https://github.com/rust-lang/rust/issues/136702
- https://github.com/rust-lang/rust/pull/136776
Tracking:
- https://github.com/rust-lang/rust/issues/127323
- https://github.com/rust-lang/rust/issues/44874
- https://github.com/rust-lang/rust/issues/123430
Count char width at most once in `Formatter::pad`
When both width and precision flags are specified, then `Formatter::pad` counts the character width twice. Instead, record the character width when truncating it to the precision, so it does not need to be recomputed. Simplify control flow so the cases are more clear.
Related:
- 6c9e708f4be (`fmt::Formatter::pad`: don't call chars().count() more than one time, 2021-09-01): Reduce counting chars from thrice to twice in worst case
- ede39aeb331 (feat: reinterpret `precision` field for strings, 2016-06-29): Change meaning of precision for strings
- b820748ff5a (Implement formatting arguments for strings and integers, 2013-08-10): Implement `Formatter::pad`
Retire the legacy `Makefile`-based `run-make` test infra
The final piece of [porting run-make tests to use Rust #121876](https://github.com/rust-lang/rust/issues/121876).
Closes#121876.
Closes#40713.
Closes#81791 (no longer using `wc`).
Closes#56475 (no longer a problem in current form of that test; we don't ignore the test on `aarch64-unknown-linux-gnu`).
### Summary
This PR removes the legacy `Makefile`-based `run-make` test infra which has served us well over the years. The legacy infra is no longer needed since we ported all of `Makefile`-based `run-make` tests to the new `rmake.rs` infra.
Additionally, this PR:
- Removes `tests/run-make/tools.mk` since no more `Makefile`-based tests remain.
- Updates `tests/run-make/README.md` and rustc-dev-guide docs to remove mention about `Makefile`-based `run-make` tests
- Update test suite requirements in rustc-dev-guide on Windows to no longer need MSYS2 (they should also now run successfully on native Windows MSVC).
- Update `triagebot.toml` to stop backlinking to #121876.
**Thanks to everyone who helped in this effort to modernize the `run-make` test infra and test suite!**
r? bootstrap
dec2flt: Clean up float parsing modules
This is the first portion of my work adding support for parsing and printing `f16`. Changes in `float.rs` replace the magic constants with expressions and add some use of generics to better support the new float types. Everything else is related to documentation or naming; there are no functional changes in this PR.
This can be reviewed by commit.
mgca: Lower all const paths as `ConstArgKind::Path`
When `#![feature(min_generic_const_args)]` is enabled, we now lower all
const paths in generic arg position to `hir::ConstArgKind::Path`. We
then lower assoc const paths to `ty::ConstKind::Unevaluated` since we
can no longer use the anon const expression lowering machinery. In the
process of implementing this, I factored out `hir_ty_lowering` code that
is now shared between lowering assoc types and assoc consts.
This PR also introduces a `#[type_const]` attribute for trait assoc
consts that are allowed as const args. However, we still need to
implement code to check that assoc const definitions satisfy
`#[type_const]` if present (basically is it a const path or a
monomorphic anon const).
r? `@BoxyUwU`
Support raw-dylib link kind on ELF
raw-dylib is a link kind that allows rustc to link against a library without having any library files present.
This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker.
While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future.
This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time.
I was inspired by Björn's comments in https://internals.rust-lang.org/t/bundle-zig-cc-in-rustup-by-default/22096/27
Tracking issue: #135694
r? bjorn3
try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-msvc-2
try-job: test-various
Rollup of 12 pull requests
Successful merges:
- #135767 (Future incompatibility warning `unsupported_fn_ptr_calling_conventions`: Also warn in dependencies)
- #137852 (Remove layouting dead code for non-array SIMD types.)
- #137863 (Fix pretty printing of unsafe binders)
- #137882 (do not build additional stage on compiler paths)
- #137894 (Revert "store ScalarPair via memset when one side is undef and the other side can be memset")
- #137902 (Make `ast::TokenKind` more like `lexer::TokenKind`)
- #137921 (Subtree update of `rust-analyzer`)
- #137922 (A few cleanups after the removal of `cfg(not(parallel))`)
- #137939 (fix order on shl impl)
- #137946 (Fix docker run-local docs)
- #137955 (Always allow rustdoc-json tests to contain long lines)
- #137958 (triagebot.toml: Don't label `test/rustdoc-json` as A-rustdoc-search)
r? `@ghost`
`@rustbot` modify labels: rollup
triagebot.toml: Don't label `test/rustdoc-json` as A-rustdoc-search
This happened because `test/rustdoc-js` is a prefix of `test/rustdoc-json`, and triagebot works on prefixes.
Maybe this should be fixed in triagebot, but this works now.
This happened on #137956 and #137955.
Always allow rustdoc-json tests to contain long lines
The rustdoc-json test syntax often requires very long lines, so the checks for long lines aren't really useful.
`@aDotInTheVoid` told me she'd like this and
r? jieyouxu
you're gonna tell me that the implementation is terrible. at least the performance seems reasonable: 2.5s after and 2.5s before.
A few cleanups after the removal of `cfg(not(parallel))`
I noticed a few small things that are no longer needed after the removal of `cfg(not(parallel))` in #132282.
One of the later changes adjusts several imports, so viewing the changes individually is recommended.
r? SparrowLii (or reroll)
do not build additional stage on compiler paths
When calling `x build compiler (or rustc) --stage N` bootstrap builds stage N+1 compiler, which is clearly not what we requested. This doesn't happen when running `x build --stage N` without explicitly targeting the compiler.
The changes applied fix this issue.
r? ghost
Optimize empty provenance range checks.
Currently it gets the pointers in the range and checks if the result is empty, but it can be done faster if you combine those two steps.
r? `@oli-obk`
`librustdoc`: return `impl fmt::Display` in more places instead of writing to strings
Continuation of #136784 , another attempt at landing the larger parts of #136748 .
I'd like to, gradually, make all of the building blocks for rendering docs in `librustdoc` return `impl fmt::Display` instead of returning `Strings`, or receiving a `&mut String` (or `&mut impl fmt::Write`). Another smaller end goal is to be able to get rid of [`write_str`](8dac72bb1d/src/librustdoc/html/format.rs (L40-L42)).
This PR is a large step in that direction.
Most of the changes are quite mechanical, and split up into separate commits for easier reviewing (hopefully). I took `print_item` and then started by converting all the functions it called (and their dependencies), and the last commit does the conversion for `print_item` itself. Ignoring whitespace should make reviewing a bit easier.
And most importantly, perf run shows pretty good results locally, hopefully CI will also show green 😁
r? `@GuillaumeGomez` , if you feel like it.
Rewrite the `ci.py` script in Rust
It would seem that I would learn by now that any script written in Python will become unmaintainable sooner or later, but alas..
r? `@marcoieni`
try-job: aarch64-gnu
try-job: dist-x86_64-linux-alt
try-job: x86_64-msvc-ext2
Fixes: https://github.com/rust-lang/rust/issues/137013
The embedded bitcode should always be prepared for LTO/ThinLTO
Fixes#115344. Fixes#117220.
There are currently two methods for generating bitcode that used for LTO. One method involves using `-C linker-plugin-lto` to emit object files as bitcode, which is the typical setting used by cargo. The other method is through `-C embed-bitcode=yes`.
When using with `-C embed-bitcode=yes -C lto=no`, we run a complete non-LTO LLVM pipeline to obtain bitcode, then the bitcode is used for LTO. We run the Call Graph Profile Pass twice on the same module.
This PR is doing something similar to LLVM's `buildFatLTODefaultPipeline`, obtaining the bitcode for embedding after running `buildThinLTOPreLinkDefaultPipeline`.
r? nikic
Don't infer attributes of virtual calls based on the function body
Fixes (after backport) #137646.
Since we don't know the exact implementation of the virtual call, it might write to parameters, we can't infer the readonly attribute.
raw-dylib is a link kind that allows rustc to link against a library
without having any library files present.
This currently only exists on Windows. rustc will take all the symbols
from raw-dylib link blocks and put them in an import library, where they
can then be resolved by the linker.
While import libraries don't exist on ELF, it would still be convenient
to have this same functionality. Not having the libraries present at
build-time can be convenient for several reasons, especially
cross-compilation. With raw-dylib, code linking against a library can be
cross-compiled without needing to have these libraries available on the
build machine. If the libc crate makes use of this, it would allow
cross-compilation without having any libc available on the build
machine. This is not yet possible with this implementation, at least
against libc's like glibc that use symbol versioning.
The raw-dylib kind could be extended with support for symbol versioning
in the future.
This implementation is very experimental and I have not tested it very
well. I have tested it for a toy example and the lz4-sys crate, where it
was able to successfully link a binary despite not having a
corresponding library at build-time.
Make -Z unpretty=mir suggest -Z dump-mir as well for discoverability
While debugging something else, I got quite annoyed with `-Z unpretty=mir` showing me post-processed MIR instead of the one just after it is built. I ended up asking on Zulip and got pointed to `-Z dump-mir`. While this feature is documented in the rustc dev guide, I think it'd be good if the possibility of making use of it was staring you in the face while you need it.
revert accidental change in get_closest_merge_commit
This was accidentally merged as part of https://github.com/rust-lang/rust/pull/137594. I need this local diff to be able to debug miri syncs, and then typed `git commit -a` too fast and didn't realize it includes this change... sorry for that.
r? ``@Kobzol``
Change interners to start preallocated with an increased capacity
Inspired by https://github.com/rust-lang/rust/issues/137005.
Added a `with_capacity` function to `InternedSet`. Changed the `CtxtInterners` to start with `InternedSets` preallocated with a capacity.
This *does* increase memory usage at very slightly(by ~1 MB at the start), altough that increase quickly disaperars for larger crates(since they require such capacity anyway).
A local perf run indicates this improves compiletimes for small crates(like `ripgrep`), without a negative effect on larger ones.
Include version number of libs being built in cargo lib metadata (esp. `librustc_driver*.so`)
Previously, on a non-stable channel, it's possible for two builds from different versioned sources (e.g. 1.84.0 vs 1.84.1) to produce a `librustc_driver*.so` with the same filename hashes. This causes problems with side-by-side installs wrt. linker search paths because 1.84.1 rustc bin and 1.84.0 rustc bin may try to link to the "same" `librustc_driver*.so` (same filename hash) but fail because the contents of the so is actually different.
We try to mitigate this by including the version number of artifacts being built via `__CARGO_DEFAULT_LIB_METADATA` (kind of an ugly hack, but I don't think cargo has a way for us to tell cargo to use a package version override).
Fixes#136701 (mitigates, really).
### Testing
Tested manually[^host] by:
```bash
$ cat src/version
1.86.0
$ ./x build library # w/ compiler profile, (non-stable) dev channel
$ lddtree build/host/stage1/bin/rustc
rustc => build/host/stage1/bin/rustc (interpreter => /lib64/ld-linux-x86-64.so.2)
librustc_driver-ea1b1b2291881cc4.so => build/host/stage1/bin/../lib/librustc_driver-ea1b1b2291881cc4.so
[...]
```
and observing that changing `src/version` to bump a point release causes `librustc_driver*.so` to have a different hash while sources are unmodified otherwise.
```bash
$ cat src/version
1.86.1
$ ./x build library # w/ compiler profile, (non-stable) dev channel
$ lddtree build/host/stage1/bin/rustc
rustc => build/host/stage1/bin/rustc (interpreter => /lib64/ld-linux-x86-64.so.2)
librustc_driver-746badadbcb74721.so => build/host/stage1/bin/../lib/librustc_driver-746badadbcb74721.so
[...]
```
cc `@clan` `@demize` could you check that if you backport this change against 1.84.{0,1} as reported in #136701, that the produced `rustc` binary works, under the context of the Gentoo build system setup?
[^host]: on a `x86_64-unknown-linux-gnu` host, no cross
Build GCC on CI
Previously, we have downloaded a specific commit of GCC and prebuilt it inside Docker using the `build-gccjit.sh` script. This PR removes that scripts and uses the bootstrap GCC step. This allows us to use the `src/gcc` submodule for determining which GCC should be built, and it also moves the logic closer to LLVM, which is also built by bootstrap.
A few things to note:
- The `sccache` option is currently in the `llvm` block, so the GCC build uses `llvm.ccache`, which is a bit weird :) We could either add `gcc.ccache`, or (what I think would be better) to just move `ccache` to the `build` section, as I don't think that it will be necessary to use ccache for LLVM, but not for GCC.
- When the GCC codegen backend is built, it needs to depend on a step that first builds GCC. This is currently done in a hacky way. The proper solution is to create a separate step for the GCC codegen backend, but that is a larger change. Let me know what you think.
r? `@onur-ozkan`
try-job: i686-msvc-1
try-job: x86_64-mingw-1
Make `#[used]` work when linking with `ld64`
To make `#[used]` work in static libraries, we use the `symbols.o` trick introduced in https://github.com/rust-lang/rust/pull/95604.
However, the linker shipped with Xcode, ld64, works a bit differently from other linkers; in particular, [it completely ignores undefined symbols by themselves](https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/parsers/macho_relocatable_file.cpp#L2455-L2468), and only consider them if they have relocations (something something atoms something fixups, I don't know the details).
So to make the `symbols.o` file work on ld64, we need to actually insert a relocation. That's kinda cumbersome to do though, since the relocation must be valid, and hence must point to a valid piece of machine code, and is hence very architecture-specific.
Fixes https://github.com/rust-lang/rust/issues/133491, see that for investigation.
---
Another option would be to pass `-u _foo` to the final linker invocation. This has the problem that `-u` causes the linker to not be able to dead-strip the symbol, which is undesirable. (If we did this, we would possibly also want to do it by putting the arguments in a file by itself, and passing that file via ``@`,` e.g. ``@undefined_symbols.txt`,` similar to https://github.com/rust-lang/rust/issues/52699, though that [is only supported since Xcode 12](https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes#Linking), and I'm not sure we wanna bump that).
Various other options that are probably all undesirable as they affect link time performance:
- Pass `-all_load` to the linker.
- Pass `-ObjC` to the linker (the Objective-C support in the linker has different code paths that load more of the binary), and instrument the binaries that contain `#[used]` symbols.
- Pass `-force_load` to libraries that contain `#[used]` symbols.
Failed attempt: Embed `-u _foo` in the object file with `LC_LINKER_OPTION`, akin to https://github.com/rust-lang/rust/issues/121293. Doesn't work, both because `ld64` doesn't read that from archive members unless it already has a reason to load the member (which is what this PR is trying to make it do), and because `ld64` only support the `-l`, `-needed-l`, `-framework` and `-needed_framework` flags in there.
---
TODO:
- [x] Support all Apple architectures.
- [x] Ensure that this works regardless of the actual type of the symbol.
- [x] Write up more docs.
- [x] Wire up a few proper tests.
`@rustbot` label O-apple
librustdoc: Use `pulldown-cmark-escape` for HTML escaping
Implementation of `@notriddle` 's [suggestion](https://github.com/rust-lang/rust/pull/137274#issuecomment-2669001585).
Somewhat related to #137274 , but the two PRs should be complementary.
Local perf results look like a nice improvement! (so would love a perf run on the CI)
Emit getelementptr inbounds nuw for pointer::add()
Lower pointer::add (via intrinsic::offset with unsigned offset) to getelementptr inbounds nuw on LLVM versions that support it. This lets LLVM make use of the pre-condition that the offset addition does not wrap in an unsigned sense. Together with inbounds, this also implies that the offset is non-negative.
Fixes https://github.com/rust-lang/rust/issues/137217.
vectorcall ABI: require SSE2
According to the official docs at https://learn.microsoft.com/en-us/cpp/cpp/vectorcall, SSE2 is required for this ABI. Add a check that enforces this.
I put this together with the other checks ensuring the target features required for a function are present... however, since the ABI is known pre-monomorphization, it would be possible to do this check earlier, which would have the advantage of checking even in `cargo check`. It would have the disadvantage of spreading this code in yet more places.
The first commit just does a little refactoring of the mono-time ABI check to make it easier to add the new check.
Cc `@workingjubilee`
try-job: dist-i586-gnu-i586-i686-musl
许杰友 Jieyou Xu (Joe)
bors
Matthias Krüger
Zalathar
Noratrieb
León Orell Valerian Liehr
Boxy
Jakub Beránek
DianQK
jyn