Stabilize proc_macro::Span::{file, local_file}.
Stabilizes this part of https://github.com/rust-lang/rust/issues/54725:
```rust
impl Span {
pub fn file(&self) -> String; // Mapped/artificial file name, for display purposes.
pub fn local_file(&self) -> Option<PathBuf>; // Real file name as it exists on the local file system.
}
```
See also the naming discussion in https://github.com/rust-lang/rust/issues/139903
Remove global `next_disambiguator` state and handle it with a `DisambiguatorState` type
This removes `Definitions.next_disambiguator` as it doesn't guarantee deterministic def paths when `create_def` is called in parallel. Instead a new `DisambiguatorState` type is passed as a mutable reference to `create_def` to help create unique def paths. `create_def` calls with distinct `DisambiguatorState` instances must ensure that that the def paths are unique without its help.
Anon associated types did rely on this global state for uniqueness and are changed to use (method they're defined in + their position in the method return type) as the `DefPathData` to ensure uniqueness. This also means that the method they're defined in appears in error messages, which is nicer.
`DefPathData::NestedStatic` is added to use for nested data inside statics instead of reusing `DefPathData::AnonConst` to avoid conflicts with those.
cc `@oli-obk`
Initial support for dynamically linked crates
This PR is an initial implementation of [rust-lang/rfcs#3435](https://github.com/rust-lang/rfcs/pull/3435) proposal.
### component 1: interface generator
Interface generator - a tool for generating a stripped version of crate source code. The interface is like a C header, where all function bodies are omitted. For example, initial crate:
```rust
#[export]
#[repr(C)]
pub struct S {
pub x: i32
}
#[export]
pub extern "C" fn foo(x: S) {
m1::bar(x);
}
pub fn bar(x: crate::S) {
// some computations
}
```
generated interface:
```rust
#[export]
#[repr(C)]
pub struct S {
pub x: i32,
}
#[export]
pub extern "C" fn foo(x: S);
pub fn bar(x: crate::S);
```
The interface generator was implemented as part of the pretty-printer. Ideally interface should only contain exportable items, but here is the first problem:
- pass for determining exportable items relies on privacy information, which is totally available only in HIR
- HIR pretty-printer uses pseudo-code(at least for attributes)
So, the interface generator was implemented in AST. This has led to the fact that non-exportable items cannot be filtered out, but I don't think this is a major issue at the moment.
To emit an interface use a new `sdylib` crate type which is basically the same as `dylib`, but it doesn't contain metadata, and also produces the interface as a second artifact. The current interface name is `lib{crate_name}.rs`.
#### Why was it decided to use a design with an auto-generated interface?
One of the main objectives of this proposal is to allow building the library and the application with different compiler versions. This requires either a metadata format compatible across rustc versions or some form of a source code. The option with a stable metadata format has not been investigated in detail, but it is not part of RFC either. Here is the the related discussion: https://github.com/rust-lang/rfcs/pull/3435#discussion_r1202872373
Original proposal suggests using the source code for the dynamic library and all its dependencies. Metadata is obtained from `cargo check`. I decided to use interface files since it is more or less compatible with the original proposal, but also allows users to hide the source code.
##### Regarding the design with interfaces
in Rust, files generally do not have a special meaning, unlike C++. A translation unit i.e. a crate is not a single file, it consists of modules. Modules, in turn, can be declared either in one file or divided into several. That's why the "interface file" isn't a very coherent concept in Rust. I would like to avoid adding an additional level of complexity for users until it is proven necessary. Therefore, the initial plan was to make the interfaces completely invisible to users i. e. make them auto-generated. I also planned to put them in the dylib, but this has not been done yet. (since the PR is already big enough, I decided to postpone it)
There is one concern, though, which has not yet been investigated(https://github.com/rust-lang/rust/pull/134767#issuecomment-2736471828):
> Compiling the interface as pretty-printed source code doesn't use correct macro hygiene (mostly relevant to macros 2.0, stable macros do not affect item hygiene). I don't have much hope for encoding hygiene data in any stable way, we should rather support a way for the interface file to be provided manually, instead of being auto-generated, if there are any non-trivial requirements.
### component 2: crate loader
When building dynamic dependencies, the crate loader searches for the interface in the file system, builds the interface without codegen and loads it's metadata. Routing rules for interface files are almost the same as for `rlibs` and `dylibs`. Firstly, the compiler checks `extern` options and then tries to deduce the path himself.
Here are the code and commands that corresponds to the compilation process:
```rust
// simple-lib.rs
#![crate_type = "sdylib"]
#[extern]
pub extern "C" fn foo() -> i32 {
42
}
```
```rust
// app.rs
extern crate simple_lib;
fn main() {
assert!(simple_lib::foo(), 42);
}
```
```
// Generate interface, build library.
rustc +toolchain1 lib.rs
// Build app. Perhaps with a different compiler version.
rustc +toolchain2 app.rs -L.
```
P.S. The interface name/format and rules for file system routing can be changed further.
### component 3: exportable items collector
Query for collecting exportable items. Which items are exportable is defined [here](https://github.com/m-ou-se/rfcs/blob/export/text/0000-export.md#the-export-attribute) .
### component 4: "stable" mangling scheme
The mangling scheme proposed in the RFC consists of two parts: a mangled item path and a hash of the signature.
#### mangled item path
For the first part of the symbol it has been decided to reuse the `v0` mangling scheme as it much less dependent on compiler internals compared to the `legacy` scheme.
The exception is disambiguators (https://doc.rust-lang.org/rustc/symbol-mangling/v0.html#disambiguator):
For example, during symbol mangling rustc uses a special index to distinguish between two impls of the same type in the same module(See `DisambiguatedDefPathData`). The calculation of this index may depend on private items, but private items should not affect the ABI. Example:
```rust
#[export]
#[repr(C)]
pub struct S<T>(pub T);
struct S1;
pub struct S2;
impl S<S1> {
extern "C" fn foo() -> i32 {
1
}
}
#[export]
impl S<S2> {
// Different symbol names can be generated for this item
// when compiling the interface and source code.
pub extern "C" fn foo() -> i32 {
2
}
}
```
In order to make disambiguation independent of the compiler version we can assign an id to each impl according to their relative order in the source code.
The second example is `StableCrateId` which is used to disambiguate different crates. `StableCrateId` consists of crate name, `-Cmetadata` arguments and compiler version. At the moment, I have decided to keep only the crate name, but a more consistent approach to crate disambiguation could be added in the future.
Actually, there are more cases where such disambiguation can be used. For instance, when mangling internal rustc symbols, but it also hasn't been investigated in detail yet.
#### hash of the signature
Exportable functions from stable dylibs can be called from safe code. In order to provide type safety, 128 bit hash with relevant type information is appended to the symbol ([description from RFC](https://github.com/m-ou-se/rfcs/blob/export/text/0000-export.md#name-mangling-and-safety)). For now, it includes:
- hash of the type name for primitive types
- for ADT types with public fields the implementation follows [this](https://github.com/m-ou-se/rfcs/blob/export/text/0000-export.md#types-with-public-fields) rules
`#[export(unsafe_stable_abi = "hash")]` syntax for ADT types with private fields is not yet implemented.
Type safety is a subtle thing here. I used the approach from RFC, but there is the ongoing research project about it. [https://rust-lang.github.io/rust-project-goals/2025h1/safe-linking.html](https://rust-lang.github.io/rust-project-goals/2025h1/safe-linking.html)
### Unresolved questions
Interfaces:
1. Move the interface generator to HIR and add an exportable items filter.
2. Compatibility of auto-generated interfaces and macro hygiene.
3. There is an open issue with interface files compilation: https://github.com/rust-lang/rust/pull/134767#issuecomment-2736471828
4. Put an interface into a dylib.
Mangling scheme:
1. Which information is required to ensure type safety and how should it be encoded? ([https://rust-lang.github.io/rust-project-goals/2025h1/safe-linking.html](https://rust-lang.github.io/rust-project-goals/2025h1/safe-linking.html))
2. Determine all other possible cases, where path disambiguation is used. Make it compiler independent.
We also need a semi-stable API to represent types. For example, the order of fields in the `VariantDef` must be stable. Or a semi-stable representation for AST, which ensures that the order of the items in the code is preserved.
There are some others, mentioned in the proposal.
Rollup of 6 pull requests
Successful merges:
- #137280 (stabilize ptr::swap_nonoverlapping in const)
- #140457 (Use target-cpu=z13 on s390x codegen const vector test)
- #140619 (Small adjustments to `check_attribute_safety` to make the logic more obvious)
- #140625 (Suggest `retain_mut` over `retain` as `Vec::extract_if` alternative)
- #140627 (Allow linking rustc and rustdoc against the same single tracing crate)
- #140630 (Async drop source info fix for proxy-drop-coroutine)
r? `@ghost`
`@rustbot` modify labels: rollup
Use target-cpu=z13 on s390x codegen const vector test
The default s390x cpu(z10) does not have vector support. Setting target-cpu at least to z13 enables vectorisation for s390x architecture and makes the test pass.
stabilize ptr::swap_nonoverlapping in const
Closes https://github.com/rust-lang/rust/issues/133668
The blocking issue mentioned there is resolved by documentation. We may in the future actually support such code, but that is blocked on https://github.com/rust-lang/const-eval/issues/72 which is non-trivial to implement. Meanwhile, this completes stabilization of all `const fn` in `ptr`. :)
Here's a version of the problematic example to play around with:
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=6c390452379fb593e109b8f8ee854d2a
Should be FCP'd with both `@rust-lang/libs-api` and `@rust-lang/lang` since `swap_nonoverlapping` is documented to work as an "untyped" operation but due to the limitation mentioned above, that's not entirely true during const evaluation. I expect this limitation will only be hit in niche corner cases, so the benefits of having this function work most of the time outweigh the downsides of users running into this problem. (Note that unsafe code could already hit this limitation before this PR by doing cursed pointer casts, but having it hidden inside `swap_nonoverlapping` feels a bit different.)
compiletest: Support matching on non-json lines in compiler output
and migrate most of remaining `error-pattern`s to it.
Such diagnostics use a new diagnostic kind `RAW`.
Also emit an error for `error-pattern`s that can be replaced with line annotations.
Also remove a number of conditions to check both line annotations and `error-pattern`s in more cases.
Also respect `//@ check-stdout` when collecting "actual errors" for comparing against line annotations.
(A couple of tiny refactorings is also included.)
Continuation of https://github.com/rust-lang/rust/pull/139760.
r? `@jieyouxu`
Set groundwork for proper const normalization
r? lcnr
Updates a lot of our normalization/alias infrastructure to be setup to handle mgca aliases and normalization once const items are represented more like aliases than bodies. Inherent associated consts are still super busted, I didn't update the assertions that IACs the right arg setup because that winds up being somewhat involved to do *before* proper support for normalizing const aliases is implemented.
I dont *intend* for this to have any effect on stable. We continue normalizing via ctfe on stable and the codepaths in `project` for consts should only be reachable with mgca or ace.
Use a closure instead of three chained iterators
Fixes the perf regression from #123948
That PR had chained a third option to the iterator which apparently didn't optimize well
mono collector: Reduce # of locking while walking the graph
While profiling Zed's dev build I've noticed that while most of the time `upstream_monomorphizations` takes a lot of time in monomorpization_collector, in some cases (e.g. build of `editor` itself) the rest of monomorphization_collector_graph_walk dominates it. Most of the time is spent in collect_items_rec.
This PR aims to reduce the number of locks taking place; instead of locking output MonoItems once per children of current node, we do so once per *current node*. We also get to reuse locks for mentioned and used items. While this commit does not reduce Wall time of Zed's build, it does shave off CPU time (measured with `cargo build -j1`) from 48s to 47s. I've also tested it with parallel frontend against Zed and ripgrep and found no regressions.
Rollup of 12 pull requests
Successful merges:
- #138703 (chore: remove redundant words in comment)
- #139186 (Refactor `diy_float`)
- #139780 (docs: Add example to `Iterator::take` with `by_ref`)
- #139802 (Fix some grammar errors and hyperlinks in doc for `trait Allocator`)
- #140034 (simd_select_bitmask: the 'padding' bits in the mask are just ignored)
- #140062 (std: mention `remove_dir_all` can emit `DirectoryNotEmpty` when concurrently written into)
- #140420 (rustdoc: Fix doctest heuristic for main fn wrapping)
- #140460 (Fix handling of LoongArch target features not supported by LLVM 19)
- #140538 (rustc-dev-guide subtree update)
- #140544 (Clean up "const" situation in format_args!(). )
- #140552 (allow `#[rustc_std_internal_symbol]` in combination with `#[naked]`)
- #140556 (Improve error output in case `nodejs` or `npm` is not installed for rustdoc-gui test suite)
r? `@ghost`
`@rustbot` modify labels: rollup
allow `#[rustc_std_internal_symbol]` in combination with `#[naked]`
The need for this came up in https://github.com/rust-lang/compiler-builtins/pull/897, but in general this seems useful and valid to allow.
Based on a quick scan, I don't think changes to the generated assembly are needed.
cc ``@bjorn3``
Clean up "const" situation in format_args!().
This cleans up the "const" situation in the format_args!() expansion/lowering.
Rather than marking the Argument::new_display etc. functions as non-const, this marks the Arguments::new_v1 functions as non-const.
Example expansion/lowering of format_args!() in const:
```rust
// Error: cannot call non-const formatting macro in constant functions
const {
fmt::Arguments::new_v1( // Now the error is produced here.
&["Hello, ", "!\n"],
&[
fmt::Argument::new_display(&world) // The error used to be produced here.
],
)
}
```
rustdoc: Fix doctest heuristic for main fn wrapping
Fixes#140412 which regressed in #140220 that I reviewed. As mentioned in https://github.com/rust-lang/rust/pull/140220#issuecomment-2837061779, at the time I didn't have the time to re-review its latest changes and should've therefore invalided my previous "r=me" and blocked the PR on another review given the fragile nature of the doctest impl. This didn't happen which is my fault.
Contains some other small changes. Diff best reviewed modulo whitespace.
r? ``@GuillaumeGomez``
std: mention `remove_dir_all` can emit `DirectoryNotEmpty` when concurrently written into
Closes#139958
The current documentation for `std::fs::remove_dir_all` function does not explicitly mention the error types that may be returned in concurrent scenarios. Specifically, when one thread attempts to remove a directory tree while another thread simultaneously writes files to that directory, the function may return an `io::ErrorKind::DirectoryNotEmpty` error, but this behavior is not clearly mentioned in the current documentation.
r? libs
simd_select_bitmask: the 'padding' bits in the mask are just ignored
Fixes https://github.com/rust-lang/rust/issues/137942: we documented simd_select_bitmask to require the 'padding' bits in the mask (the mask can sometimes be longer than the vector; I am referring to these extra bits as 'padding' here) to be zero, mostly because nobody felt like doing the research for what should be done when they are non-zero. However, codegen is already perfectly happy just ignoring them, so in practice they can have any value. Some of the intrinsic wrappers in stdarch have trouble ensuring that they are zero. So let's just adjust the docs and Miri to permit non-zero 'padding' bits.
Cc ````@Amanieu```` ````@workingjubilee````
Fix some grammar errors and hyperlinks in doc for `trait Allocator`
I was reading the allocator docs and noticed some weird sentences and missing hyperlink, so I fixed them and made this small PR.
* "while until either" could also be changed to "for a while until either", but I just deleted "while".
* fixed sentence with incorrect "at" and "has/have".
* linked [*currently allocated*] similar to other methods. All other hyperlinks are fine.
docs: Add example to `Iterator::take` with `by_ref`
If you want to logically split an iterator after `n` items, you might first discover `take`. Before this change, you'd find that `take` consumes the iterator, and you'd probably be stuck. The answer involves `by_ref`, but that's hard to discover, especially since `by_ref` is a bit abstract and `Iterator` has many methods.
After this change, you'd see the example showing `take` along with `by_ref`, which allows you to continue using the rest of the iterator. `by_ref` had a good example involving `take` already, so this change just duplicates that existing example under `take`.
Refactor `diy_float`
The refactor replaces bespoke algorithms with functions already inside the standard library, improving both codegen and readability.
Decouple SCC annotations from SCCs
This rewires SCC annotations to have them be a separate, visitor-type data structure. It was broken out of #130227, which needed them to be able to remove unused annotations after computation without recomputing the SCCs themselves.
As a drive-by it also removes some redundant code from the hot loop in SCC construction for a performance improvement.
r? lcnr
shared-generics: Do not share instantiations that contain local-only types
In Zed shared-generics loading takes up a significant chunk of time in incremental build, as rustc deserializes rmeta of all dependencies of a crate. I've recently realized that shared-generics includes all instantiations of some_generic_function in the following snippet:
```rs
pub fn some_generic_function(_: impl Fn()) {}
pub fn non_generic_function() {
some_generic_function(|| {});
some_generic_function(|| {});
some_generic_function(|| {});
some_generic_function(|| {});
some_generic_function(|| {});
some_generic_function(|| {});
some_generic_function(|| {});
}
```
even though none of these instantiations can actually be created from outside of `non_generic_function`. This is a dummy example, but we do rely on invoking callbacks with FnOnce a lot in our codebase.
This PR makes shared-generics account for visibilities of generic arguments; an item is only considered for exporting if it is reachable from the outside or if all of it's arguments are visible outside of the local crate.
This PR reduces incremental build time for Zed (touch editor.rs scenario) from 12.4s to 10.4s. I'd love to see a perf run if possible; per my checks this PR does not incur new instantiations in downstream crates, so if there'd be perf regressions, I'd expect them to come from newly-introduced visibility checks.
Add a jobserver proxy to ensure at least one token is always held
This adds a jobserver proxy to ensure at least one token is always held by `rustc`. Currently with `-Z threads` `rustc` can temporarily give up all its tokens, causing `cargo` to spawn additional `rustc` instances beyond the job limit.
The current behavior causes an issue with `cargo fix` which has a global lock preventing concurrent `rustc` instances, but it also holds a jobserver token, causing a deadlock when `rustc` gives up its token. That is fixed by this PR.
Fixes https://github.com/rust-lang/rust/issues/67385.
Fixes https://github.com/rust-lang/rust/issues/133873.
Fixes https://github.com/rust-lang/rust/issues/140093.
bors
Tshepang Mbambo
Ralf Jung
Trevor Gross
Vadim Petrochenkov
Oli Scherer
The Miri Cronjob Bot
Guillaume Gomez
Laurențiu Nicola
Laurențiu Nicola