From 8789c2cf9b73b68c6a6fbab64ef92ec50d9c3f72 Mon Sep 17 00:00:00 2001 From: Samson <16504129+sagudev@users.noreply.github.com> Date: Tue, 7 Jan 2025 21:29:45 +0100 Subject: [PATCH 001/107] Fix rib example related zulip thread: https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Ribs.20in.20name.20resolution --- src/name-resolution.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/name-resolution.md b/src/name-resolution.md index 2727b814..719ebce8 100644 --- a/src/name-resolution.md +++ b/src/name-resolution.md @@ -120,9 +120,9 @@ even though they should be visible by ordinary scoping rules. An example: fn do_something(val: T) { // <- New rib in both types and values (1) // `val` is accessible, as is the helper function // `T` is accessible - let helper = || { // New rib on `helper` (2) and another on the block (3) + let helper = || { // New rib on the block (2) // `val` is accessible here - }; // End of (3) + }; // End of (2), new rib on `helper` (3) // `val` is accessible, `helper` variable shadows `helper` function fn helper() { // <- New rib in both types and values (4) // `val` is not accessible here, (4) is not transparent for locals @@ -130,7 +130,7 @@ fn do_something(val: T) { // <- New rib in both types and values (1) } // End of (4) let val = T::default(); // New rib (5) // `val` is the variable, not the parameter here -} // End of (5), (2) and (1) +} // End of (5), (3) and (1) ``` Because the rules for different namespaces are a bit different, each namespace From 4f883945452097bc02969fdc728ed5806c9f8766 Mon Sep 17 00:00:00 2001 From: Deadbeef Date: Thu, 20 Feb 2025 21:39:45 +0800 Subject: [PATCH 002/107] Rewrite effects checking chapter --- src/SUMMARY.md | 2 +- src/effects.md | 170 ++++++++++++++++++++++++++++++++++--------------- 2 files changed, 118 insertions(+), 54 deletions(-) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 91c4aeac..b1423acd 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -177,7 +177,7 @@ - [Inference details](./opaque-types-impl-trait-inference.md) - [Return Position Impl Trait In Trait](./return-position-impl-trait-in-trait.md) - [Region inference restrictions][opaque-infer] -- [Effect checking](./effects.md) +- [Const condition checking](./effects.md) - [Pattern and Exhaustiveness Checking](./pat-exhaustive-checking.md) - [Unsafety Checking](./unsafety-checking.md) - [MIR dataflow](./mir/dataflow.md) diff --git a/src/effects.md b/src/effects.md index 1fda7bcb..c30a521f 100644 --- a/src/effects.md +++ b/src/effects.md @@ -1,66 +1,130 @@ -# Effects and effect checking +# Effects and const condition checking -Note: all of this describes the implementation of the unstable `effects` and -`const_trait_impl` features. None of this implementation is usable or visible from -stable Rust. +## The `HostEffect` predicate -The implementation of const traits and `~const` bounds is a limited effect system. -It is used to allow trait bounds on `const fn` to be used within the `const fn` for -method calls. Within the function, in order to know whether a method on a trait -bound is `const`, we need to know whether there is a `~const` bound for the trait. -In order to know whether we can instantiate a `~const` bound on a `const fn`, we -need to know whether there is a `const_trait` impl for the type and trait being -used (or whether the `const fn` is used at runtime, then any type implementing the -trait is ok, just like with other bounds). +[`HostEffectPredicate`]s are a kind of predicate from `~const Tr` or `const Tr` +bounds. It has a trait reference, and a `constness` which could be `Maybe` or +`Const` depending on the bound. Because `~const Tr`, or rather `Maybe` bounds +apply differently based on whichever contexts they are in, they have different +behavior than normal bounds. Where normal trait bounds on a function such as +`T: Tr` are collected within the [`predicates_of`] query to be proven when a +function is called and to be assumed within the function, bounds such as +`T: ~const Tr` will behave as a normal trait bound and add `T: Tr` to the result +from `predicates_of`, but also adds a `HostEffectPredicate` to the +[`const_conditions`] query. -We perform these checks via a const generic boolean that gets attached to all -`const fn` and `const trait`. The following sections will explain the desugarings -and the way we perform the checks at call sites. +On the other hand, `T: const Tr` bounds do not change meaning across contexts, +therefore they will result in `HostEffect(T: Tr, const)` being added to +`predicates_of`, and not `const_conditions`. -The const generic boolean is inverted to the meaning of `const`. In the compiler -it is called `host`, because it enables "host APIs" like `static` items, network -access, disk access, random numbers and everything else that isn't available in -`const` contexts. So `false` means "const", `true` means "not const" and if it's -a generic parameter, it means "maybe const" (meaning we're in a const fn or const -trait). +[`HostEffectPredicate`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/predicate/struct.HostEffectPredicate.html +[`predicates_of`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.predicates_of +[`const_conditions`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.const_conditions -## `const fn` +## The `const_conditions` query -All `const fn` have a `#[rustc_host] const host: bool` generic parameter that is -hidden from users. Any `~const Trait` bounds in the generics list or `where` bounds -of a `const fn` get converted to `Trait + Trait` bounds. The `Trait` -exists so that associated types of the generic param can be used from projections -like `::Assoc`, because there are no `` projections for now. +`predicates_of` represents a set of predicates that need to be proven to use an +item. For example, to use `foo` in the example below: -## `#[const_trait] trait`s +```rust +fn foo() where T: Default {} +``` -The `#[const_trait]` attribute gives the marked trait a `#[rustc_host] const host: bool` -generic parameter. All functions of the trait "inherit" this generic parameter, just like -they have all the regular generic parameters of the trait. Any `~const Trait` super-trait -bounds get desugared to `Trait + Trait` in order to allow using associated -types and consts of the super traits in the trait declaration. This is necessary, because -`::Assoc` is always `>::Assoc` as there is -no `` syntax. +We must be able to prove that `T` implements `Default`. In a similar vein, +`const_conditions` represents a set of predicates that need to be proven to use +an item *in const contexts*. If we adjust the example above to use `const` trait +bounds: -## `typeck` performing method and function call checks. +```rust +const fn foo() where T: ~const Default {} +``` -When generic parameters are instantiated for any items, the `host` generic parameter -is always instantiated as an inference variable. This is a special kind of inference var -that is not part of the type or const inference variables, similar to how we have -special inference variables for type variables that we know to be an integer, but not -yet which one. These separate inference variables fall back to `true` at -the end of typeck (in `fallback_effects`) to ensure that `let _ = some_fn_item_name;` -will keep compiling. +Then `foo` would get a `HostEffect(T: Default, maybe)` in the `const_conditions` +query, suggesting that in order to call `foo` from const contexts, one must +prove that `T` has a const implementation of `Default`. -All actually used (in function calls, casts, or anywhere else) function items, will -have the `enforce_context_effects` method invoked. -It trivially returns if the function being called has no `host` generic parameter. +## Enforcement of `const_conditions` -In order to error if a non-const function is called in a const context, we have not -yet disabled the const-check logic that happens on MIR, because -`enforce_context_effects` does not yet perform this check. +`const_conditions` are currently checked in various places. -The function call's `host` parameter is then equated to the context's `host` value, -which almost always trivially succeeds, as it was an inference var. If the inference -var has already been bound (since the function item is invoked twice), the second -invocation checks it against the first. +Every call in HIR from a const context (which includes `const fn` and `const` +items) will check that `const_conditions` of the function we are calling hold. +This is done in [`FnCtxt::enforce_context_effects`]. Note that we don't check +if the function is only referred to but not called, as the following code needs +to compile: + +```rust +const fn hi() -> T { + T::default() +} +const X: fn() -> u32 = hi::; +``` + +For a trait `impl` to be well-formed, we must be able to prove the +`const_conditions` of the trait from the `impl`'s environment. This is checked +in [`wfcheck::check_impl`]. + +Here's an example: + +```rust +#[const_trait] +trait Bar {} +#[const_trait] +trait Foo: ~const Bar {} +// `const_conditions` contains `HostEffect(Self: Bar, maybe)` + +impl const Bar for () {} +impl const Foo for () {} +// ^ here we check `const_conditions` for the impl to be well-formed +``` + +Methods of trait impls must not have stricter bounds than the method of the +trait that they are implementing. To check that the methods are compatible, a +hybrid environment is constructed with the predicates of the `impl` plus the +predicates of the trait method, and we attempt to prove the predicates of the +impl method. We do the same for `const_conditions`: + +```rust +#[const_trait] +trait Foo { + fn hi(); +} + +impl Foo for Vec { + fn hi(); + // ^ we can't prove `T: ~const PartialEq` given `T: ~const Clone` and + // `T: ~const Default`, therefore we know that the method on the impl + // is stricter than the method on the trait. +} +``` + +These checks are done in [`compare_method_predicate_entailment`]. A similar +function that does the same check for associated types is called +[`compare_type_predicate_entailment`]. Both of these need to consider +`const_conditions` when in const contexts. + +In MIR, as part of const checking, `const_conditions` of items that are called +are revalidated again in [`Checker::revalidate_conditional_constness`]. + +[`compare_method_predicate_entailment`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/compare_impl_item/fn.compare_method_predicate_entailment.html +[`compare_type_predicate_entailment`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/compare_impl_item/fn.compare_type_predicate_entailment.html +[`FnCtxt::enforce_context_effects`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_typeck/fn_ctxt/struct.FnCtxt.html#method.enforce_context_effects +[`wfcheck::check_impl`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/wfcheck/fn.check_impl.html +[`Checker::revalidate_conditional_constness`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_const_eval/check_consts/check/struct.Checker.html#method.revalidate_conditional_constness + +## Proving `HostEffectPredicate`s + +`HostEffectPredicate`s are implemented both in the [old solver] and the [new +trait solver]. In general, we can prove a `HostEffect` predicate when either of +these conditions are met: + +* The predicate can be assumed from caller bounds; +* The type has a `const` `impl` for the trait, *and* that const conditions on +the impl holds; or +* The type has a built-in implementation for the trait in const contexts. For +example, `Fn` may be implemented by function items if their const conditions +are satisfied, or `Destruct` is implemented in const contexts if the type can +be dropped at compile time. + +[old solver]: https://doc.rust-lang.org/nightly/nightly-rustc/src/rustc_trait_selection/traits/effects.rs.html +[new trait solver]: https://doc.rust-lang.org/nightly/nightly-rustc/src/rustc_next_trait_solver/solve/effect_goals.rs.html From aab48065ca644fe6a780872514db7f819e51a4e7 Mon Sep 17 00:00:00 2001 From: moxian Date: Wed, 5 Mar 2025 15:28:06 -0800 Subject: [PATCH 003/107] Don't suggest explicitly `cfg`-gating `trace!` calls in bootstrap --- src/building/bootstrapping/debugging-bootstrap.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/src/building/bootstrapping/debugging-bootstrap.md b/src/building/bootstrapping/debugging-bootstrap.md index 24b9783d..35d33ebd 100644 --- a/src/building/bootstrapping/debugging-bootstrap.md +++ b/src/building/bootstrapping/debugging-bootstrap.md @@ -129,7 +129,7 @@ Both `tracing::*` macros and the `tracing::instrument` proc-macro attribute need ```rs #[cfg(feature = "tracing")] -use tracing::{instrument, trace}; +use tracing::instrument; struct Foo; @@ -138,7 +138,6 @@ impl Step for Foo { #[cfg_attr(feature = "tracing", instrument(level = "trace", name = "Foo::should_run", skip_all))] fn should_run(run: ShouldRun<'_>) -> ShouldRun<'_> { - #[cfg(feature = "tracing")] trace!(?run, "entered Foo::should_run"); todo!() @@ -154,7 +153,6 @@ impl Step for Foo { ), )] fn run(self, builder: &Builder<'_>) -> Self::Output { - #[cfg(feature = "tracing")] trace!(?run, "entered Foo::run"); todo!() From 79e42cfb0bbc7eb6062149c657f6f0ed0ef18438 Mon Sep 17 00:00:00 2001 From: Deadbeef Date: Sat, 8 Mar 2025 16:38:44 +0800 Subject: [PATCH 004/107] consider `explicit_implied_const_bounds` --- src/effects.md | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/src/effects.md b/src/effects.md index c30a521f..c7aa2714 100644 --- a/src/effects.md +++ b/src/effects.md @@ -112,6 +112,34 @@ are revalidated again in [`Checker::revalidate_conditional_constness`]. [`wfcheck::check_impl`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/wfcheck/fn.check_impl.html [`Checker::revalidate_conditional_constness`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_const_eval/check_consts/check/struct.Checker.html#method.revalidate_conditional_constness +## `explicit_implied_const_bounds` on associated types and traits + +Bounds on associated types, opaque types, and supertraits such as +```rust +trait Foo: ~const PartialEq { + type X: ~const PartialEq; +} + +fn foo() -> impl ~const PartialEq { + // ^ unimplemented syntax +} +``` + +Have their bounds represented differently. Unlike `const_conditions` which need +to be proved for callers, and can be assumed inside the definition (e.g. trait +bounds on functions), these bounds need to be proved at definition (at the impl, +or when returning the opaque) but can be assumed for callers. The non-const +equivalent of these bounds are called [`explicit_item_bounds`]. + +These bounds are checked in [`compare_impl_item::check_type_bounds`] for HIR +typeck, [`evaluate_host_effect_from_item_bounds`] in the old solver and +[`consider_additional_alias_assumptions`] in the new solver. + +[`explicit_item_bounds`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.explicit_item_bounds +[`compare_impl_item::check_type_bounds`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/compare_impl_item/fn.check_type_bounds.html +[`evaluate_host_effect_from_item_bounds`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/traits/effects/fn.evaluate_host_effect_from_item_bounds.html +[`consider_additional_alias_assumptions`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_next_trait_solver/solve/assembly/trait.GoalKind.html#tymethod.consider_additional_alias_assumptions + ## Proving `HostEffectPredicate`s `HostEffectPredicate`s are implemented both in the [old solver] and the [new @@ -120,7 +148,8 @@ these conditions are met: * The predicate can be assumed from caller bounds; * The type has a `const` `impl` for the trait, *and* that const conditions on -the impl holds; or +the impl holds, *and* that the `explicit_implied_const_bounds` on the trait +holds; or * The type has a built-in implementation for the trait in const contexts. For example, `Fn` may be implemented by function items if their const conditions are satisfied, or `Destruct` is implemented in const contexts if the type can From 723b887eff068e08cfdaececd8fb573a90b2a92e Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 8 Mar 2025 20:33:05 +0200 Subject: [PATCH 005/107] numbers were not sequential, so stop trying --- src/backend/updating-llvm.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/backend/updating-llvm.md b/src/backend/updating-llvm.md index 92d4ce32..bc8207bb 100644 --- a/src/backend/updating-llvm.md +++ b/src/backend/updating-llvm.md @@ -116,14 +116,14 @@ so let's go through each in detail. at the time of the branch, and the remaining part is the current date. -2. Apply Rust-specific patches to the llvm-project repository. +1. Apply Rust-specific patches to the llvm-project repository. All features and bugfixes are upstream, but there's often some weird build-related patches that don't make sense to upstream. These patches are typically the latest patches in the rust-lang/llvm-project branch that rustc is currently using. -3. Build the new LLVM in the `rust` repository. +1. Build the new LLVM in the `rust` repository. To do this, you'll want to update the `src/llvm-project` repository to your branch, and the revision you've created. @@ -151,7 +151,7 @@ so let's go through each in detail. download-ci-llvm = false ``` -4. Test for regressions across other platforms. LLVM often has at least one bug +1. Test for regressions across other platforms. LLVM often has at least one bug for non-tier-1 architectures, so it's good to do some more testing before sending this to bors! If you're low on resources you can send the PR as-is now to bors, though, and it'll get tested anyway. @@ -170,7 +170,7 @@ so let's go through each in detail. * `./src/ci/docker/run.sh dist-various-2` * `./src/ci/docker/run.sh armhf-gnu` -5. Prepare a PR to `rust-lang/rust`. Work with maintainers of +1. Prepare a PR to `rust-lang/rust`. Work with maintainers of `rust-lang/llvm-project` to get your commit in a branch of that repository, and then you can send a PR to `rust-lang/rust`. You'll change at least `src/llvm-project` and will likely also change [`llvm-wrapper`] as well. @@ -194,7 +194,7 @@ so let's go through each in detail. others interested in trying out the new LLVM can benefit from work you've done to update the C++ bindings. -3. Over the next few months, +1. Over the next few months, LLVM will continually push commits to its `release/a.b` branch. We will often want to have those bug fixes as well. The merge process for that is to use `git merge` itself to merge LLVM's @@ -202,9 +202,9 @@ so let's go through each in detail. This is typically done multiple times when necessary while LLVM's release branch is baking. -4. LLVM then announces the release of version `a.b`. +1. LLVM then announces the release of version `a.b`. -5. After LLVM's official release, +1. After LLVM's official release, we follow the process of creating a new branch on the rust-lang/llvm-project repository again, this time with a new date. From fc6a11aeb4af722a78574783457393e6f0ed1e7b Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 8 Mar 2025 20:35:42 +0200 Subject: [PATCH 006/107] only a few are needed as examples --- src/backend/updating-llvm.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/src/backend/updating-llvm.md b/src/backend/updating-llvm.md index bc8207bb..be5fed9c 100644 --- a/src/backend/updating-llvm.md +++ b/src/backend/updating-llvm.md @@ -177,12 +177,6 @@ so let's go through each in detail. > For prior art, here are some previous LLVM updates: - > - [LLVM 11](https://github.com/rust-lang/rust/pull/73526) - > - [LLVM 12](https://github.com/rust-lang/rust/pull/81451) - > - [LLVM 13](https://github.com/rust-lang/rust/pull/87570) - > - [LLVM 14](https://github.com/rust-lang/rust/pull/93577) - > - [LLVM 15](https://github.com/rust-lang/rust/pull/99464) - > - [LLVM 16](https://github.com/rust-lang/rust/pull/109474) > - [LLVM 17](https://github.com/rust-lang/rust/pull/115959) > - [LLVM 18](https://github.com/rust-lang/rust/pull/120055) > - [LLVM 19](https://github.com/rust-lang/rust/pull/127513) From ebead304f40a4bdc5e86e5d21c46ce7554db37f8 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 8 Mar 2025 20:40:44 +0200 Subject: [PATCH 007/107] link to latest major llvm update pr --- src/backend/updating-llvm.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/backend/updating-llvm.md b/src/backend/updating-llvm.md index be5fed9c..0b45956b 100644 --- a/src/backend/updating-llvm.md +++ b/src/backend/updating-llvm.md @@ -175,11 +175,12 @@ so let's go through each in detail. and then you can send a PR to `rust-lang/rust`. You'll change at least `src/llvm-project` and will likely also change [`llvm-wrapper`] as well. - + > For prior art, here are some previous LLVM updates: > - [LLVM 17](https://github.com/rust-lang/rust/pull/115959) > - [LLVM 18](https://github.com/rust-lang/rust/pull/120055) > - [LLVM 19](https://github.com/rust-lang/rust/pull/127513) + > - [LLVM 20](https://github.com/rust-lang/rust/pull/135763) Note that sometimes it's easiest to land [`llvm-wrapper`] compatibility as a PR before actually updating `src/llvm-project`. From 28b1ec713082f5cc3cfaa3fe9f0f0336547d64b4 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 8 Mar 2025 22:58:09 +0200 Subject: [PATCH 008/107] fix text - There is more than just target and stage - There is only 3 stages, so don't mention them specially --- src/tests/directives.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/tests/directives.md b/src/tests/directives.md index 00bb2bc4..934f7985 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -122,8 +122,7 @@ for more details. These directives are used to ignore the test in some situations, which means the test won't be compiled or run. -* `ignore-X` where `X` is a target detail or stage will ignore the test - accordingly (see below) +* `ignore-X` where `X` is a target detail or other criteria on which to ignore the test (see below) * `only-X` is like `ignore-X`, but will *only* run the test on that target or stage * `ignore-test` always ignores the test. This can be used to temporarily disable From efa11d0d5c58776351b739b2814910ea04c9c3dc Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 8 Mar 2025 23:51:17 +0200 Subject: [PATCH 009/107] ignore-stage0 and only-stage0 do not exist --- src/tests/directives.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tests/directives.md b/src/tests/directives.md index 00bb2bc4..078efc27 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -142,7 +142,7 @@ Some examples of `X` in `ignore-X` or `only-X`: matches that target as well as the emscripten targets. - Pointer width: `32bit`, `64bit` - Endianness: `endian-big` -- Stage: `stage0`, `stage1`, `stage2` +- Stage: `stage1`, `stage2` - Channel: `stable`, `beta` - When cross compiling: `cross-compile` - When [remote testing] is used: `remote` From 1d3b59fa4aedd6ab4b2df4ade0a2da9ca7d8a4ac Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 10 Mar 2025 10:29:30 +0200 Subject: [PATCH 010/107] use new terminology --- src/tests/running.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/tests/running.md b/src/tests/running.md index 6ce65092..58564bc4 100644 --- a/src/tests/running.md +++ b/src/tests/running.md @@ -24,8 +24,8 @@ collection. The test results are cached and previously successful tests are `ignored` during testing. The stdout/stderr contents as well as a timestamp file for every test -can be found under `build//test/` for the given -``. To force-rerun a test (e.g. in case the test runner fails to +can be found under `build//test/` for the given +``. To force-rerun a test (e.g. in case the test runner fails to notice a change) you can use the `--force-rerun` CLI option. > **Note on requirements of external dependencies** From 20b3dc1ee009a58c3a2c4fc485eadb7bc2e28b0e Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 10 Mar 2025 10:35:40 +0200 Subject: [PATCH 011/107] already mentioned before showing code snippet --- src/tests/running.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tests/running.md b/src/tests/running.md index 6ce65092..16cf8dad 100644 --- a/src/tests/running.md +++ b/src/tests/running.md @@ -49,7 +49,7 @@ test suite ([`tests/ui`]): ./x test tests/ui ``` -This will run the `ui` test suite. Of course, the choice of test suites is +Of course, the choice of test suites is somewhat arbitrary, and may not suit the task you are doing. For example, if you are hacking on debuginfo, you may be better off with the debuginfo test suite: From 916cd09d33dc7766566ff13d7c824b76a78796d4 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 10 Mar 2025 10:38:30 +0200 Subject: [PATCH 012/107] add a pause, for readability --- src/tests/running.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/tests/running.md b/src/tests/running.md index 16cf8dad..05a56579 100644 --- a/src/tests/running.md +++ b/src/tests/running.md @@ -112,8 +112,8 @@ crates, you have to specify those explicitly. ./x test --stage 1 library/std ``` -By listing which test suites you want to run you avoid having to run tests for -components you did not change at all. +By listing which test suites you want to run, +you avoid having to run tests for components you did not change at all.
Note that bors only runs the tests with the full stage 2 build; therefore, while From 12b836b37c3c181a9429984c05b14eea252d3740 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 10 Mar 2025 11:12:44 +0200 Subject: [PATCH 013/107] clean --bless text --- src/tests/running.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/src/tests/running.md b/src/tests/running.md index 05a56579..eb763215 100644 --- a/src/tests/running.md +++ b/src/tests/running.md @@ -172,16 +172,18 @@ additional arguments to the compiler when building the tests. ## Editing and updating the reference files If you have changed the compiler's output intentionally, or you are making a new -test, you can pass `--bless` to the test subcommand. E.g. if some tests in -`tests/ui` are failing, you can run +test, you can pass `--bless` to the test subcommand. + +As an example, +if some tests in `tests/ui` are failing, you can run this command: ```text ./x test tests/ui --bless ``` -to automatically adjust the `.stderr`, `.stdout` or `.fixed` files of -all tests. Of course you can also target just specific tests with the -`--test-args your_test_name` flag, just like when running the tests. +It automatically adjusts the `.stderr`, `.stdout`, or `.fixed` files of all `test/ui` tests. +Of course you can also target just specific tests with the `--test-args your_test_name` flag, +just like when running the tests without the `--bless` flag. ## Configuring test running From 53f8a2f1d6a626f28678a7edaebbf1c41f278285 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 10 Mar 2025 11:15:32 +0200 Subject: [PATCH 014/107] add missing punctuation --- src/tests/running.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tests/running.md b/src/tests/running.md index eb763215..35deec4b 100644 --- a/src/tests/running.md +++ b/src/tests/running.md @@ -192,7 +192,7 @@ There are a few options for running tests: * `config.toml` has the `rust.verbose-tests` option. If `false`, each test will print a single dot (the default). If `true`, the name of every test will be printed. This is equivalent to the `--quiet` option in the [Rust test - harness](https://doc.rust-lang.org/rustc/tests/) + harness](https://doc.rust-lang.org/rustc/tests/). * The environment variable `RUST_TEST_THREADS` can be set to the number of concurrent threads to use for testing. From 074e4787bdb0d8fd76398fb87dfc6f34746ab4f9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Mon, 10 Mar 2025 13:35:50 +0100 Subject: [PATCH 015/107] Modify try-job documentation --- src/tests/ci.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/src/tests/ci.md b/src/tests/ci.md index ae6adb67..7fd4c169 100644 --- a/src/tests/ci.md +++ b/src/tests/ci.md @@ -134,15 +134,17 @@ There are several use-cases for try builds: the [dist-x86_64-linux] CI job. - Run a specific CI job (e.g. Windows tests) on a PR, to quickly test if it passes the test suite executed by that job. You can select which CI jobs will - be executed in the try build by adding up to 10 lines containing `try-job: - ` to the PR description. All such specified jobs will be executed + be executed in the try build by adding lines containing `try-job: + ` to the PR description. All such specified jobs will be executed in the try build once the `@bors try` command is used on the PR. If no try jobs are specified in this way, the jobs defined in the `try` section of - [`jobs.yml`] will be executed by default. + [`jobs.yml`] will be executed by default. Each pattern can either be an exact + name of a job or a glob pattern that matches multiple jobs, for example + `*msvc*` or `*-alt`. You can start at most 20 jobs in a single try build. > **Using `try-job` PR description directives** > -> 1. Identify which set of try-jobs (max 10) you would like to exercise. You can +> 1. Identify which set of try-jobs you would like to exercise. You can > find the name of the CI jobs in [`jobs.yml`]. > > 2. Amend PR description to include (usually at the end of the PR description) @@ -153,9 +155,10 @@ There are several use-cases for try builds: > > try-job: x86_64-msvc > try-job: test-various +> try-job: *-alt > ``` > -> Each `try-job` directive must be on its own line. +> Each `try-job` pattern must be on its own line. > > 3. Run the prescribed try jobs with `@bors try`. As aforementioned, this > requires the user to either (1) have `try` permissions or (2) be delegated From dd0359476d06336734803e1c081b17a507147708 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Mon, 10 Mar 2025 13:56:42 +0100 Subject: [PATCH 016/107] Handle backticks in try job patterns --- src/tests/ci.md | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/src/tests/ci.md b/src/tests/ci.md index 7fd4c169..0c0f750a 100644 --- a/src/tests/ci.md +++ b/src/tests/ci.md @@ -133,29 +133,34 @@ There are several use-cases for try builds: Again, a working compiler build is needed for this, which can be produced by the [dist-x86_64-linux] CI job. - Run a specific CI job (e.g. Windows tests) on a PR, to quickly test if it - passes the test suite executed by that job. You can select which CI jobs will - be executed in the try build by adding lines containing `try-job: - ` to the PR description. All such specified jobs will be executed - in the try build once the `@bors try` command is used on the PR. If no try - jobs are specified in this way, the jobs defined in the `try` section of - [`jobs.yml`] will be executed by default. Each pattern can either be an exact - name of a job or a glob pattern that matches multiple jobs, for example - `*msvc*` or `*-alt`. You can start at most 20 jobs in a single try build. + passes the test suite executed by that job. + +You can select which CI jobs will +be executed in the try build by adding lines containing `try-job: +` to the PR description. All such specified jobs will be executed +in the try build once the `@bors try` command is used on the PR. If no try +jobs are specified in this way, the jobs defined in the `try` section of +[`jobs.yml`] will be executed by default. + +Each pattern can either be an exact name of a job or a glob pattern that matches multiple jobs, +for example `*msvc*` or `*-alt`. You can start at most 20 jobs in a single try build. When using +glob patterns, you might want to wrap them in backticks (`` ` ``) to avoid GitHub rendering +the pattern as Markdown. > **Using `try-job` PR description directives** > > 1. Identify which set of try-jobs you would like to exercise. You can > find the name of the CI jobs in [`jobs.yml`]. > -> 2. Amend PR description to include (usually at the end of the PR description) -> e.g. +> 2. Amend PR description to include a set of patterns (usually at the end +> of the PR description), for example: > > ```text > This PR fixes #123456. > > try-job: x86_64-msvc > try-job: test-various -> try-job: *-alt +> try-job: `*-alt` > ``` > > Each `try-job` pattern must be on its own line. From 7f5b5c36f82be0ff0e270f648adc90f3042b5c2a Mon Sep 17 00:00:00 2001 From: Nicholas Nethercote Date: Fri, 21 Feb 2025 18:33:05 +1100 Subject: [PATCH 017/107] Move methods from `Map` to `TyCtxt`, part 4. Continuing the work from #137350. Removes the unused methods: `expect_variant`, `expect_field`, `expect_foreign_item`. Every method gains a `hir_` prefix. --- src/hir.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/hir.md b/src/hir.md index 51893d53..75f5a9e2 100644 --- a/src/hir.md +++ b/src/hir.md @@ -139,12 +139,12 @@ defined in the map. By matching on this, you can find out what sort of node the `HirId` referred to and also get a pointer to the data itself. Often, you know what sort of node `n` is – e.g. if you know that `n` must be some HIR expression, you can do -[`tcx.hir().expect_expr(n)`][expect_expr], which will extract and return the +[`tcx.hir_expect_expr(n)`][expect_expr], which will extract and return the [`&hir::Expr`][Expr], panicking if `n` is not in fact an expression. [find]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html#method.find [`Node`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/enum.Node.html -[expect_expr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html#method.expect_expr +[expect_expr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.expect_expr [Expr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/struct.Expr.html Finally, you can use the HIR map to find the parents of nodes, via From bbcfc6902791892d0427b3d16871a08a521e04c3 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Thu, 13 Mar 2025 03:51:51 +0200 Subject: [PATCH 018/107] less text for same effect --- src/building/new-target.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/building/new-target.md b/src/building/new-target.md index cd215277..14d10d4a 100644 --- a/src/building/new-target.md +++ b/src/building/new-target.md @@ -4,12 +4,11 @@ These are a set of steps to add support for a new target. There are numerous end states and paths to get there, so not all sections may be relevant to your desired goal. -See also the associated documentation in the -[target tier policy][target_tier_policy_add]. +See also the associated documentation in the [target tier policy]. -[target_tier_policy_add]: https://doc.rust-lang.org/rustc/target-tier-policy.html#adding-a-new-target +[target tier policy]: https://doc.rust-lang.org/rustc/target-tier-policy.html#adding-a-new-target ## Specifying a new LLVM From 2aa579f75233f3e63e3ffa8ee51ab5a362e93f8b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=AE=B8=E6=9D=B0=E5=8F=8B=20Jieyou=20Xu=20=28Joe=29?= <39484203+jieyouxu@users.noreply.github.com> Date: Thu, 13 Mar 2025 15:19:22 +0800 Subject: [PATCH 019/107] Preparing for merge from rustc --- rust-version | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust-version b/rust-version index ce21bb8e..eb779d9a 100644 --- a/rust-version +++ b/rust-version @@ -1 +1 @@ -4ecd70ddd1039a3954056c1071e40278048476fa +8536f201ffdb2c24925d7f9e87996d7dca93428b From 589573c793a440c8830dfe54892e216430843130 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=AE=B8=E6=9D=B0=E5=8F=8B=20Jieyou=20Xu=20=28Joe=29?= <39484203+jieyouxu@users.noreply.github.com> Date: Thu, 13 Mar 2025 15:14:59 +0800 Subject: [PATCH 020/107] Document `fetch.prunetags = true` gotcha during rustc-pull --- README.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/README.md b/README.md index 2464ffbb..6a25a91f 100644 --- a/README.md +++ b/README.md @@ -82,6 +82,7 @@ cargo +stable install josh-proxy --git https://github.com/josh-project/josh --ta Older versions of `josh-proxy` may not round trip commits losslessly so it is important to install this exact version. ### Pull changes from `rust-lang/rust` into this repository + 1) Checkout a new branch that will be used to create a PR into `rust-lang/rustc-dev-guide` 2) Run the pull command ``` @@ -95,3 +96,15 @@ Older versions of `josh-proxy` may not round trip commits losslessly so it is im $ cargo run --manifest-path josh-sync/Cargo.toml rustc-push ``` 2) Create a PR from `` into `rust-lang/rust` + +#### Minimal git config + +For simplicity (ease of implementation purposes), the josh-sync script simply calls out to system git. This means that the git invocation may be influenced by global (or local) git configuration. + +You may observe "Nothing to pull" even if you *know* rustc-pull has something to pull if your global git config sets `fetch.prunetags = true` (and possibly other configurations may cause unexpected outcomes). + +To minimize the likelihood of this happening, you may wish to keep a separate *minimal* git config that *only* has `[user]` entries from global git config, then repoint system git to use the minimal git config instead. E.g. + +``` +$ GIT_CONFIG_GLOBAL=/path/to/minimal/gitconfig GIT_CONFIG_SYSTEM='' cargo +stable run --manifest-path josh-sync/Cargo.toml -- rustc-pull +``` From 46d4f952c7e3ca2a6b8a237d96d836588919fa04 Mon Sep 17 00:00:00 2001 From: Josh Stone Date: Thu, 13 Mar 2025 12:15:29 -0700 Subject: [PATCH 021/107] Remove the doc for `no-system-llvm` This compiletest directive was removed in rust-lang/rust#120265. --- src/tests/directives.md | 1 - 1 file changed, 1 deletion(-) diff --git a/src/tests/directives.md b/src/tests/directives.md index 7f50cd23..bbebd5d9 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -197,7 +197,6 @@ settings: The following directives will check LLVM support: -- `no-system-llvm` — ignores if the system llvm is used - `exact-llvm-major-version: 19` — ignores if the llvm major version does not match the specified llvm major version. - `min-llvm-version: 13.0` — ignored if the LLVM version is less than the given From c20c046b0f346cd6f722e3d42fc6419660576852 Mon Sep 17 00:00:00 2001 From: KonaeAkira Date: Thu, 13 Mar 2025 23:56:04 +0100 Subject: [PATCH 022/107] Fix grammar and remove redundant info --- src/solve/trait-solving.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/solve/trait-solving.md b/src/solve/trait-solving.md index 345ee0b0..c1eb1a94 100644 --- a/src/solve/trait-solving.md +++ b/src/solve/trait-solving.md @@ -2,8 +2,7 @@ This chapter describes how trait solving works with the new WIP solver located in [`rustc_trait_selection/solve`][solve]. Feel free to also look at the docs for -[the current solver](../traits/resolution.md) and [the chalk solver](../traits/chalk.md) -can be found separately. +[the current solver](../traits/resolution.md) and [the chalk solver](../traits/chalk.md). ## Core concepts From dad3718db833ee4fd2df3e5e21774045b1860c0a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Fri, 14 Mar 2025 15:18:58 +0100 Subject: [PATCH 023/107] Fix MCP links --- src/contributing.md | 2 +- src/implementing_new_features.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/contributing.md b/src/contributing.md index 9817326f..09a7f912 100644 --- a/src/contributing.md +++ b/src/contributing.md @@ -81,7 +81,7 @@ smaller user-facing changes. into a PR that ends up not getting merged!** [See this document][mcpinfo] for more info on MCPs. -[mcpinfo]: https://forge.rust-lang.org/compiler/mcp.html +[mcpinfo]: https://forge.rust-lang.org/compiler/proposals-and-stabilization.html#how-do-i-submit-an-mcp [zulip]: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler ### Performance diff --git a/src/implementing_new_features.md b/src/implementing_new_features.md index fda38ef4..d7561bbb 100644 --- a/src/implementing_new_features.md +++ b/src/implementing_new_features.md @@ -44,7 +44,7 @@ like this; for example, the compiler team recommends filing a Major Change Proposal ([MCP][mcp]) as a lightweight way to garner support and feedback without requiring full consensus. -[mcp]: https://forge.rust-lang.org/compiler/mcp.html#public-facing-changes-require-rfcbot-fcp +[mcp]: https://forge.rust-lang.org/compiler/proposals-and-stabilization.html#how-do-i-submit-an-mcp You don't need to have the implementation fully ready for r+ to propose an FCP, but it is generally a good idea to have at least a proof From e068c546a3198d1940069b7268d0bddfafaa3a6d Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 15 Mar 2025 11:11:43 +0200 Subject: [PATCH 024/107] those should not get shell highlighting --- src/tests/running.md | 48 ++++++++++++++++++++++---------------------- 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/src/tests/running.md b/src/tests/running.md index 03d4123c..c7d92478 100644 --- a/src/tests/running.md +++ b/src/tests/running.md @@ -18,7 +18,7 @@ a subset of test collections, and merge queue CI will exercise all of the test collection.
-```bash +```text ./x test ``` @@ -45,7 +45,7 @@ tests. For example, a good "smoke test" that can be used after modifying rustc to see if things are generally working correctly would be to exercise the `ui` test suite ([`tests/ui`]): -```bash +```text ./x test tests/ui ``` @@ -53,14 +53,14 @@ Of course, the choice of test suites is somewhat arbitrary, and may not suit the task you are doing. For example, if you are hacking on debuginfo, you may be better off with the debuginfo test suite: -```bash +```text ./x test tests/debuginfo ``` If you only need to test a specific subdirectory of tests for any given test suite, you can pass that directory as a filter to `./x test`: -```bash +```text ./x test tests/ui/const-generics ``` @@ -73,7 +73,7 @@ suite, you can pass that directory as a filter to `./x test`: Likewise, you can test a single file by passing its path: -```bash +```text ./x test tests/ui/const-generics/const-test.rs ``` @@ -81,19 +81,19 @@ Likewise, you can test a single file by passing its path: have to use the `--test-args` argument as described [below](#running-an-individual-test). -```bash +```text ./x test src/tools/miri --test-args tests/fail/uninit/padding-enum.rs ``` ### Run only the tidy script -```bash +```text ./x test tidy ``` ### Run tests on the standard library -```bash +```text ./x test --stage 0 library/std ``` @@ -102,13 +102,13 @@ crates, you have to specify those explicitly. ### Run the tidy script and tests on the standard library -```bash +```text ./x test --stage 0 tidy library/std ``` ### Run tests on the standard library using a stage 1 compiler -```bash +```text ./x test --stage 1 library/std ``` @@ -122,7 +122,7 @@ the tests **usually** work fine with stage 1, there are some limitations. ### Run all tests using a stage 2 compiler -```bash +```text ./x test --stage 2 ``` @@ -134,13 +134,13 @@ You almost never need to do this; CI will run these tests for you. You may want to run unit tests on a specific file with following: -```bash +```text ./x test compiler/rustc_data_structures/src/thin_vec/tests.rs ``` But unfortunately, it's impossible. You should invoke the following instead: -```bash +```text ./x test compiler/rustc_data_structures/ --test-args thin_vec ``` @@ -151,7 +151,7 @@ often the test they are trying to fix. As mentioned earlier, you may pass the full file path to achieve this, or alternatively one may invoke `x` with the `--test-args` option: -```bash +```text ./x test tests/ui --test-args issue-1234 ``` @@ -203,7 +203,7 @@ When `--pass $mode` is passed, these tests will be forced to run under the given `$mode` unless the directive `//@ ignore-pass` exists in the test file. For example, you can run all the tests in `tests/ui` as `check-pass`: -```bash +```text ./x test tests/ui --pass check ``` @@ -219,7 +219,7 @@ first look for expected output in `foo.polonius.stderr`, falling back to the usual `foo.stderr` if not found. The following will run the UI test suite in Polonius mode: -```bash +```text ./x test tests/ui --compare-mode=polonius ``` @@ -232,7 +232,7 @@ just `.rs` files, so after [creating a rustup toolchain](../building/how-to-build-and-run.md#creating-a-rustup-toolchain), you can do something like: -```bash +```text rustc +stage1 tests/ui/issue-1234.rs ``` @@ -252,7 +252,7 @@ execution* so be careful where it is used. To do this, first build `remote-test-server` for the remote machine, e.g. for RISC-V -```sh +```text ./x build src/tools/remote-test-server --target riscv64gc-unknown-linux-gnu ``` @@ -264,7 +264,7 @@ On the remote machine, run the `remote-test-server` with the `--bind 0.0.0.0:12345` flag (and optionally `-v` for verbose output). Output should look like this: -```sh +```text $ ./remote-test-server -v --bind 0.0.0.0:12345 starting test server listening on 0.0.0.0:12345! @@ -278,7 +278,7 @@ restrictive IP address when binding. You can test if the `remote-test-server` is working by connecting to it and sending `ping\n`. It should reply `pong`: -```sh +```text $ nc $REMOTE_IP 12345 ping pong @@ -288,7 +288,7 @@ To run tests using the remote runner, set the `TEST_DEVICE_ADDR` environment variable then use `x` as usual. For example, to run `ui` tests for a RISC-V machine with the IP address `1.2.3.4` use -```sh +```text export TEST_DEVICE_ADDR="1.2.3.4:12345" ./x test tests/ui --target riscv64gc-unknown-linux-gnu ``` @@ -362,20 +362,20 @@ codegen-backends = ["llvm", "gcc"] Then you need to install libgccjit 12. For example with `apt`: -```bash +```text $ apt install libgccjit-12-dev ``` Now you can run the following command: -```bash +```text $ ./x test compiler/rustc_codegen_gcc/ ``` If it cannot find the `.so` library (if you installed it with `apt` for example), you need to pass the library file path with `LIBRARY_PATH`: -```bash +```text $ LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/12/ ./x test compiler/rustc_codegen_gcc/ ``` From 7d157cb36ea63d78de4b496f34e04b5501e532c8 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 15 Mar 2025 11:35:15 +0200 Subject: [PATCH 025/107] make 'mdbook test --chapter "Running tests"' pass --- src/tests/running.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tests/running.md b/src/tests/running.md index c7d92478..6fb49810 100644 --- a/src/tests/running.md +++ b/src/tests/running.md @@ -296,7 +296,7 @@ export TEST_DEVICE_ADDR="1.2.3.4:12345" If `remote-test-server` was run with the verbose flag, output on the test machine may look something like -``` +```text [...] run "/tmp/work/test1007/a" run "/tmp/work/test1008/a" From b04865b8d2027cd262b794fc9d6b2dba895cb5e8 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 15 Mar 2025 11:17:03 +0200 Subject: [PATCH 026/107] add some copy-paste goodness --- src/tests/running.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/tests/running.md b/src/tests/running.md index 6fb49810..73f5603d 100644 --- a/src/tests/running.md +++ b/src/tests/running.md @@ -363,20 +363,20 @@ codegen-backends = ["llvm", "gcc"] Then you need to install libgccjit 12. For example with `apt`: ```text -$ apt install libgccjit-12-dev +apt install libgccjit-12-dev ``` Now you can run the following command: ```text -$ ./x test compiler/rustc_codegen_gcc/ +./x test compiler/rustc_codegen_gcc/ ``` If it cannot find the `.so` library (if you installed it with `apt` for example), you need to pass the library file path with `LIBRARY_PATH`: ```text -$ LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/12/ ./x test compiler/rustc_codegen_gcc/ +LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/12/ ./x test compiler/rustc_codegen_gcc/ ``` If you encounter bugs or problems, don't hesitate to open issues on the From 971366b58bce4d3ec4f10b75a10d2a38485e61cd Mon Sep 17 00:00:00 2001 From: Yang Lin Date: Sun, 16 Mar 2025 23:27:10 +0800 Subject: [PATCH 027/107] Adapt to rust-lang/rust#136466: Start removing `rustc_middle::hir::map::Map` Following commit f86f7ad from pull request #136466 in the Rust project (https://github.com/rust-lang/rust), some methods in `Map` were moved to `TyCtxt`. This update reimplements `rustc-drive-example.rs`, `rustc-driver-interacting-with-the-ast.rs`, and `rustc-interface-example.rs` using the new versions of these methods, ensuring compatibility with the nightly-2025-03-08 toolchain. --- examples/rustc-driver-example.rs | 7 +++---- examples/rustc-driver-interacting-with-the-ast.rs | 8 +++----- examples/rustc-interface-example.rs | 7 +++---- 3 files changed, 9 insertions(+), 13 deletions(-) diff --git a/examples/rustc-driver-example.rs b/examples/rustc-driver-example.rs index 984bd3e3..35dd07dd 100644 --- a/examples/rustc-driver-example.rs +++ b/examples/rustc-driver-example.rs @@ -1,4 +1,4 @@ -// Tested with nightly-2025-02-13 +// Tested with nightly-2025-03-08 #![feature(rustc_private)] @@ -71,9 +71,8 @@ impl rustc_driver::Callbacks for MyCallbacks { fn after_analysis(&mut self, _compiler: &Compiler, tcx: TyCtxt<'_>) -> Compilation { // Analyze the program and inspect the types of definitions. - for id in tcx.hir().items() { - let hir = tcx.hir(); - let item = hir.item(id); + for id in tcx.hir_free_items(){ + let item = &tcx.hir_item(id); match item.kind { rustc_hir::ItemKind::Static(_, _, _) | rustc_hir::ItemKind::Fn { .. } => { let name = item.ident; diff --git a/examples/rustc-driver-interacting-with-the-ast.rs b/examples/rustc-driver-interacting-with-the-ast.rs index 3270c722..2a43ba47 100644 --- a/examples/rustc-driver-interacting-with-the-ast.rs +++ b/examples/rustc-driver-interacting-with-the-ast.rs @@ -1,4 +1,4 @@ -// Tested with nightly-2025-02-13 +// Tested with nightly-2025-03-08 #![feature(rustc_private)] @@ -70,11 +70,9 @@ impl rustc_driver::Callbacks for MyCallbacks { } fn after_analysis(&mut self, _compiler: &Compiler, tcx: TyCtxt<'_>) -> Compilation { - // Every compilation contains a single crate. - let hir_krate = tcx.hir(); // Iterate over the top-level items in the crate, looking for the main function. - for id in hir_krate.items() { - let item = hir_krate.item(id); + for id in tcx.hir_free_items(){ + let item = &tcx.hir_item(id); // Use pattern-matching to find a specific node inside the main function. if let rustc_hir::ItemKind::Fn { body, .. } = item.kind { let expr = &tcx.hir_body(body).value; diff --git a/examples/rustc-interface-example.rs b/examples/rustc-interface-example.rs index 70f27c2a..11125cae 100644 --- a/examples/rustc-interface-example.rs +++ b/examples/rustc-interface-example.rs @@ -1,4 +1,4 @@ -// Tested with nightly-2025-02-13 +// Tested with nightly-2025-03-08 #![feature(rustc_private)] @@ -64,9 +64,8 @@ fn main() { println!("{krate:?}"); // Analyze the program and inspect the types of definitions. rustc_interface::create_and_enter_global_ctxt(&compiler, krate, |tcx| { - for id in tcx.hir().items() { - let hir = tcx.hir(); - let item = hir.item(id); + for id in tcx.hir_free_items() { + let item = tcx.hir_item(id); match item.kind { rustc_hir::ItemKind::Static(_, _, _) | rustc_hir::ItemKind::Fn { .. } => { let name = item.ident; From 832299550ce82ff0b9360c8c4e328140c8b72260 Mon Sep 17 00:00:00 2001 From: Yang Lin Date: Sun, 16 Mar 2025 23:43:17 +0800 Subject: [PATCH 028/107] Following commit 401dd84 in the Rust project (https://github.com/rust-lang/rust), `ErrorGuaranteed` was replaced by fatal errors. As a result, `tcx.analysis()` now aborts directly instead of returning an error guard. To accommodate this change, this update replaces `tcx.analysis()` with `typeck()` to perform type checking in the example. --- examples/rustc-interface-getting-diagnostics.rs | 8 +++++--- src/rustc-driver/getting-diagnostics.md | 3 ++- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/examples/rustc-interface-getting-diagnostics.rs b/examples/rustc-interface-getting-diagnostics.rs index 39b236e1..9bb93ab4 100644 --- a/examples/rustc-interface-getting-diagnostics.rs +++ b/examples/rustc-interface-getting-diagnostics.rs @@ -1,4 +1,4 @@ -// Tested with nightly-2025-02-13 +// Tested with nightly-2025-03-08 #![feature(rustc_private)] @@ -86,8 +86,10 @@ fn main() { rustc_interface::run_compiler(config, |compiler| { let krate = rustc_interface::passes::parse(&compiler.sess); rustc_interface::create_and_enter_global_ctxt(&compiler, krate, |tcx| { - // Run the analysis phase on the local crate to trigger the type error. - let _ = tcx.analysis(()); + // Iterate all the items defined and perform type checking. + tcx.par_hir_body_owners(|item_def_id| { + tcx.ensure_ok().typeck(item_def_id); + }); }); // If the compiler has encountered errors when this closure returns, it will abort (!) the program. // We avoid this by resetting the error count before returning diff --git a/src/rustc-driver/getting-diagnostics.md b/src/rustc-driver/getting-diagnostics.md index 1043df6e..518cf4e8 100644 --- a/src/rustc-driver/getting-diagnostics.md +++ b/src/rustc-driver/getting-diagnostics.md @@ -7,7 +7,7 @@ otherwise be printed to stderr. To get diagnostics from the compiler, configure [`rustc_interface::Config`] to output diagnostic to a buffer, -and run [`TyCtxt.analysis`]. +and run [`rustc_hir_typeck::typeck`] for each item. ```rust {{#include ../../examples/rustc-interface-getting-diagnostics.rs}} @@ -16,3 +16,4 @@ and run [`TyCtxt.analysis`]. [`rustc_interface`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_interface/index.html [`rustc_interface::Config`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_interface/interface/struct.Config.html [`TyCtxt.analysis`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_interface/passes/fn.analysis.html +[`rustc_hir_typeck::typeck`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_typeck/fn.typeck.html From f6940a6e7c2b263f1144585e01008e93dfd0f6c8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Sun, 16 Mar 2025 20:42:37 +0100 Subject: [PATCH 029/107] Add a note to rustc-dev-guide --- src/tests/ci.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/tests/ci.md b/src/tests/ci.md index 0c0f750a..2af09a60 100644 --- a/src/tests/ci.md +++ b/src/tests/ci.md @@ -180,6 +180,8 @@ their results can be seen [here](https://github.com/rust-lang-ci/rust/actions), although usually you will be notified of the result by a comment made by bors on the corresponding PR. +Note that if you start the default try job using `@bors try`, it will skip building several `dist` components and running post-optimization tests, to make the build duration shorter. If you want to execute the full build as it would happen before a merge, add an explicit `try-job` pattern with the name of the default try job (currently `dist-x86_64-linux`). + Multiple try builds can execute concurrently across different PRs.
From bfda715ef4d289ad6ab43277dbc783b050905743 Mon Sep 17 00:00:00 2001 From: jyn Date: Sun, 16 Mar 2025 21:06:18 -0400 Subject: [PATCH 030/107] expand ${workspaceFolder} in sample vim config --- src/building/suggested.md | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/src/building/suggested.md b/src/building/suggested.md index 24988381..772b0a77 100644 --- a/src/building/suggested.md +++ b/src/building/suggested.md @@ -123,6 +123,30 @@ Another way is without a plugin, and creating your own logic in your configuration. The following code will work for any checkout of rust-lang/rust (newer than Febuary 2025): ```lua +local function expand_config_variables(option) + local var_placeholders = { + ['${workspaceFolder}'] = function(_) + return vim.lsp.buf.list_workspace_folders()[1] + end, + } + + if type(option) == "table" then + local mt = getmetatable(option) + local result = {} + for k, v in pairs(option) do + result[expand_config_variables(k)] = expand_config_variables(v) + end + return setmetatable(result, mt) + end + if type(option) ~= "string" then + return option + end + local ret = option + for key, fn in pairs(var_placeholders) do + ret = ret:gsub(key, fn) + end + return ret +end lspconfig.rust_analyzer.setup { root_dir = function() local default = lspconfig.rust_analyzer.config_def.default_config.root_dir() @@ -142,7 +166,7 @@ lspconfig.rust_analyzer.setup { -- load rust-lang/rust settings local file = io.open(config) local json = vim.json.decode(file:read("*a")) - client.config.settings["rust-analyzer"] = json.lsp["rust-analyzer"].initialization_options + client.config.settings["rust-analyzer"] = expand_config_variables(json.lsp["rust-analyzer"].initialization_options) client.notify("workspace/didChangeConfiguration", { settings = client.config.settings }) end return true From e4ddc21c8ab4bee43c905b1d4621b4c657c5d0ef Mon Sep 17 00:00:00 2001 From: bit-aloo Date: Sat, 15 Feb 2025 19:41:17 +0530 Subject: [PATCH 031/107] replace config.toml to bootstrap.toml in src:doc:rustc-dev-guide --- src/backend/debugging.md | 4 ++-- src/backend/updating-llvm.md | 2 +- src/building/compiler-documenting.md | 2 +- src/building/how-to-build-and-run.md | 14 +++++++------- src/building/new-target.md | 12 ++++++------ src/building/optimized-build.md | 8 ++++---- src/building/suggested.md | 10 +++++----- src/compiler-debugging.md | 10 +++++----- src/fuzzing.md | 2 +- src/llvm-coverage-instrumentation.md | 6 +++--- src/overview.md | 2 +- src/profile-guided-optimization.md | 2 +- src/profiling.md | 2 +- src/profiling/with_perf.md | 2 +- src/profiling/with_rustc_perf.md | 2 +- src/profiling/wpa_profiling.md | 2 +- src/rustdoc.md | 4 ++-- src/sanitizers.md | 4 ++-- src/tests/ci.md | 2 +- src/tests/compiletest.md | 4 ++-- src/tests/directives.md | 6 +++--- src/tests/docker.md | 2 +- src/tests/running.md | 4 ++-- src/tracing.md | 4 ++-- 24 files changed, 56 insertions(+), 56 deletions(-) diff --git a/src/backend/debugging.md b/src/backend/debugging.md index 80529101..4f8712df 100644 --- a/src/backend/debugging.md +++ b/src/backend/debugging.md @@ -38,7 +38,7 @@ which means that LLVM assertion failures can show up as compiler crashes (not ICEs but "real" crashes) and other sorts of weird behavior. If you are encountering these, it is a good idea to try using a compiler with LLVM assertions enabled - either an "alt" nightly or a compiler you build yourself -by setting `[llvm] assertions=true` in your config.toml - and see whether +by setting `[llvm] assertions=true` in your bootstrap.toml - and see whether anything turns up. The rustc build process builds the LLVM tools into @@ -160,7 +160,7 @@ from `./build//llvm/bin/` with the LLVM IR emitted by rustc. When investigating the implementation of LLVM itself, you should be aware of its [internal debug infrastructure][llvm-debug]. This is provided in LLVM Debug builds, which you enable for rustc -LLVM builds by changing this setting in the config.toml: +LLVM builds by changing this setting in the bootstrap.toml: ``` [llvm] # Indicates whether the LLVM assertions are enabled or not diff --git a/src/backend/updating-llvm.md b/src/backend/updating-llvm.md index 0b45956b..18c822aa 100644 --- a/src/backend/updating-llvm.md +++ b/src/backend/updating-llvm.md @@ -144,7 +144,7 @@ so let's go through each in detail. Note that `profile = "compiler"` and other defaults set by `./x setup` download LLVM from CI instead of building it from source. You should disable this temporarily to make sure your changes are being used. - This is done by having the following setting in `config.toml`: + This is done by having the following setting in `bootstrap.toml`: ```toml [llvm] diff --git a/src/building/compiler-documenting.md b/src/building/compiler-documenting.md index 948571ce..b031462e 100644 --- a/src/building/compiler-documenting.md +++ b/src/building/compiler-documenting.md @@ -36,7 +36,7 @@ like the standard library (std) or the compiler (rustc). - Document internal rustc items Compiler documentation is not built by default. - To create it by default with `x doc`, modify `config.toml`: + To create it by default with `x doc`, modify `bootstrap.toml`: ```toml [build] diff --git a/src/building/how-to-build-and-run.md b/src/building/how-to-build-and-run.md index f9a089c2..067e2871 100644 --- a/src/building/how-to-build-and-run.md +++ b/src/building/how-to-build-and-run.md @@ -159,15 +159,15 @@ similar to the one declared in section [What is `x.py`](#what-is-xpy), but it works as an independent process to execute the `x.py` rather than calling the shell to run the platform related scripts. -## Create a `config.toml` +## Create a `bootstrap.toml` To start, run `./x setup` and select the `compiler` defaults. This will do some initialization -and create a `config.toml` for you with reasonable defaults. If you use a different default (which +and create a `bootstrap.toml` for you with reasonable defaults. If you use a different default (which you'll likely want to do if you want to contribute to an area of rust other than the compiler, such as rustdoc), make sure to read information about that default (located in `src/bootstrap/defaults`) as the build process may be different for other defaults. -Alternatively, you can write `config.toml` by hand. See `config.example.toml` for all the available +Alternatively, you can write `bootstrap.toml` by hand. See `bootstrap.example.toml` for all the available settings and explanations of them. See `src/bootstrap/defaults` for common settings to change. If you have already built `rustc` and you change settings related to LLVM, then you may have to @@ -206,7 +206,7 @@ See the chapters on Note that building will require a relatively large amount of storage space. You may want to have upwards of 10 or 15 gigabytes available to build the compiler. -Once you've created a `config.toml`, you are now ready to run +Once you've created a `bootstrap.toml`, you are now ready to run `x`. There are a lot of options here, but let's start with what is probably the best "go to" command for building a local compiler: @@ -326,7 +326,7 @@ involve proc macros or build scripts, you must be sure to explicitly build targe host platform (in this case, `x86_64-unknown-linux-gnu`). If you want to always build for other targets without needing to pass flags to `x build`, -you can configure this in the `[build]` section of your `config.toml` like so: +you can configure this in the `[build]` section of your `bootstrap.toml` like so: ```toml [build] @@ -336,8 +336,8 @@ target = ["x86_64-unknown-linux-gnu", "wasm32-wasip1"] Note that building for some targets requires having external dependencies installed (e.g. building musl targets requires a local copy of musl). Any target-specific configuration (e.g. the path to a local copy of musl) -will need to be provided by your `config.toml`. -Please see `config.example.toml` for information on target-specific configuration keys. +will need to be provided by your `bootstrap.toml`. +Please see `bootstrap.example.toml` for information on target-specific configuration keys. For examples of the complete configuration necessary to build a target, please visit [the rustc book](https://doc.rust-lang.org/rustc/platform-support.html), diff --git a/src/building/new-target.md b/src/building/new-target.md index 14d10d4a..09ffbe8c 100644 --- a/src/building/new-target.md +++ b/src/building/new-target.md @@ -37,7 +37,7 @@ able to configure Rust to treat your build as the system LLVM to avoid redundant builds. You can tell Rust to use a pre-built version of LLVM using the `target` section -of `config.toml`: +of `bootstrap.toml`: ```toml [target.x86_64-unknown-linux-gnu] @@ -55,8 +55,8 @@ for codegen tests. This tool is normally built with LLVM, but if you use your own preinstalled LLVM, you will need to provide `FileCheck` in some other way. On Debian-based systems, you can install the `llvm-N-tools` package (where `N` is the LLVM version number, e.g. `llvm-8-tools`). Alternately, you can specify -the path to `FileCheck` with the `llvm-filecheck` config item in `config.toml` -or you can disable codegen test with the `codegen-tests` item in `config.toml`. +the path to `FileCheck` with the `llvm-filecheck` config item in `bootstrap.toml` +or you can disable codegen test with the `codegen-tests` item in `bootstrap.toml`. ## Creating a target specification @@ -141,14 +141,14 @@ After this, run `cargo update -p libc` to update the lockfiles. Beware that if you patch to a local `path` dependency, this will enable warnings for that dependency. Some dependencies are not warning-free, and due -to the `deny-warnings` setting in `config.toml`, the build may suddenly start +to the `deny-warnings` setting in `bootstrap.toml`, the build may suddenly start to fail. To work around warnings, you may want to: - Modify the dependency to remove the warnings -- Or for local development purposes, suppress the warnings by setting deny-warnings = false in config.toml. +- Or for local development purposes, suppress the warnings by setting deny-warnings = false in bootstrap.toml. ```toml -# config.toml +# bootstrap.toml [rust] deny-warnings = false ``` diff --git a/src/building/optimized-build.md b/src/building/optimized-build.md index f8ca1d0d..0849464e 100644 --- a/src/building/optimized-build.md +++ b/src/building/optimized-build.md @@ -13,7 +13,7 @@ This page describes how you can use these approaches when building `rustc` yours Link-time optimization is a powerful compiler technique that can increase program performance. To enable (Thin-)LTO when building `rustc`, set the `rust.lto` config option to `"thin"` -in `config.toml`: +in `bootstrap.toml`: ```toml [rust] @@ -34,7 +34,7 @@ Enabling LTO on Linux has [produced] speed-ups by up to 10%. Using a different memory allocator for `rustc` can provide significant performance benefits. If you want to enable the `jemalloc` allocator, you can set the `rust.jemalloc` option to `true` -in `config.toml`: +in `bootstrap.toml`: ```toml [rust] @@ -46,7 +46,7 @@ jemalloc = true ## Codegen units Reducing the amount of codegen units per `rustc` crate can produce a faster build of the compiler. -You can modify the number of codegen units for `rustc` and `libstd` in `config.toml` with the +You can modify the number of codegen units for `rustc` and `libstd` in `bootstrap.toml` with the following options: ```toml @@ -67,7 +67,7 @@ RUSTFLAGS="-C target_cpu=x86-64-v3" ./x build ... ``` If you also want to compile LLVM for a specific instruction set, you can set `llvm` flags -in `config.toml`: +in `bootstrap.toml`: ```toml [llvm] diff --git a/src/building/suggested.md b/src/building/suggested.md index 24988381..506f41b1 100644 --- a/src/building/suggested.md +++ b/src/building/suggested.md @@ -305,7 +305,7 @@ subsequent rebuilds: ``` If you don't want to include the flag with every command, you can enable it in -the `config.toml`: +the `bootstrap.toml`: ```toml [rust] @@ -384,20 +384,20 @@ ln -s ./src/tools/nix-dev-shell/envrc-shell ./.envrc # Use nix-shell ### Note Note that when using nix on a not-NixOS distribution, it may be necessary to set -**`patch-binaries-for-nix = true` in `config.toml`**. Bootstrap tries to detect +**`patch-binaries-for-nix = true` in `bootstrap.toml`**. Bootstrap tries to detect whether it's running in nix and enable patching automatically, but this detection can have false negatives. -You can also use your nix shell to manage `config.toml`: +You can also use your nix shell to manage `bootstrap.toml`: ```nix let config = pkgs.writeText "rustc-config" '' - # Your config.toml content goes here + # Your bootstrap.toml content goes here '' pkgs.mkShell { /* ... */ - # This environment variable tells bootstrap where our config.toml is. + # This environment variable tells bootstrap where our bootstrap.toml is. RUST_BOOTSTRAP_CONFIG = config; } ``` diff --git a/src/compiler-debugging.md b/src/compiler-debugging.md index c16b3ee7..47f39762 100644 --- a/src/compiler-debugging.md +++ b/src/compiler-debugging.md @@ -11,13 +11,13 @@ chapter](./backend/debugging.md)). ## Configuring the compiler By default, rustc is built without most debug information. To enable debug info, -set `debug = true` in your config.toml. +set `debug = true` in your bootstrap.toml. Setting `debug = true` turns on many different debug options (e.g., `debug-assertions`, `debug-logging`, etc.) which can be individually tweaked if you want to, but many people simply set `debug = true`. -If you want to use GDB to debug rustc, please set `config.toml` with options: +If you want to use GDB to debug rustc, please set `bootstrap.toml` with options: ```toml [rust] @@ -35,14 +35,14 @@ debuginfo-level = 2 The default configuration will enable `symbol-mangling-version` v0. This requires at least GDB v10.2, -otherwise you need to disable new symbol-mangling-version in `config.toml`. +otherwise you need to disable new symbol-mangling-version in `bootstrap.toml`. ```toml [rust] new-symbol-mangling = false ``` -> See the comments in `config.example.toml` for more info. +> See the comments in `bootstrap.example.toml` for more info. You will need to rebuild the compiler after changing any configuration option. @@ -373,7 +373,7 @@ error: aborting due to previous error ## Configuring CodeLLDB for debugging `rustc` -If you are using VSCode, and have edited your `config.toml` to request debugging +If you are using VSCode, and have edited your `bootstrap.toml` to request debugging level 1 or 2 for the parts of the code you're interested in, then you should be able to use the [CodeLLDB] extension in VSCode to debug it. diff --git a/src/fuzzing.md b/src/fuzzing.md index 869fc2f7..b588ca10 100644 --- a/src/fuzzing.md +++ b/src/fuzzing.md @@ -123,7 +123,7 @@ what actually results in superior throughput. You may want to build rustc from source with debug assertions to find additional bugs, though this is a trade-off: it can slow down fuzzing by requiring extra work for every execution. To enable debug assertions, add this -to `config.toml` when compiling rustc: +to `bootstrap.toml` when compiling rustc: ```toml [rust] diff --git a/src/llvm-coverage-instrumentation.md b/src/llvm-coverage-instrumentation.md index 3078ae09..6bc21b6d 100644 --- a/src/llvm-coverage-instrumentation.md +++ b/src/llvm-coverage-instrumentation.md @@ -34,7 +34,7 @@ Detailed instructions and examples are documented in the [coverage map]: https://llvm.org/docs/CoverageMappingFormat.html [rustc-book-instrument-coverage]: https://doc.rust-lang.org/nightly/rustc/instrument-coverage.html -## Recommended `config.toml` settings +## Recommended `bootstrap.toml` settings When working on the coverage instrumentation code, it is usually necessary to **enable the profiler runtime** by setting `profiler = true` in `[build]`. @@ -83,7 +83,7 @@ statically links coverage-instrumented binaries with LLVM runtime code In the `rustc` source tree, `library/profiler_builtins` bundles the LLVM `compiler-rt` code into a Rust library crate. Note that when building `rustc`, -`profiler_builtins` is only included when `build.profiler = true` is set in `config.toml`. +`profiler_builtins` is only included when `build.profiler = true` is set in `bootstrap.toml`. When compiling with `-C instrument-coverage`, [`CrateLoader::postprocess()`][crate-loader-postprocess] dynamically loads @@ -115,7 +115,7 @@ human-readable coverage report. > Tests in `coverage-run` mode have an implicit `//@ needs-profiler-runtime` > directive, so they will be skipped if the profiler runtime has not been -> [enabled in `config.toml`](#recommended-configtoml-settings). +> [enabled in `bootstrap.toml`](#recommended-configtoml-settings). Finally, the [`tests/codegen/instrument-coverage/testprog.rs`] test compiles a simple Rust program with `-C instrument-coverage` and compares the compiled program's LLVM IR to diff --git a/src/overview.md b/src/overview.md index 21ab0040..92d0c7b0 100644 --- a/src/overview.md +++ b/src/overview.md @@ -351,7 +351,7 @@ approach is to turn [`RefCell`]s into [`Mutex`]s -- that is, we switch to thread-safe internal mutability. However, there are ongoing challenges with lock contention, maintaining query-system invariants under concurrency, and the complexity of the code base. One can try out the current -work by enabling parallel compilation in `config.toml`. It's still early days, +work by enabling parallel compilation in `bootstrap.toml`. It's still early days, but there are already some promising performance improvements. [`RefCell`]: https://doc.rust-lang.org/std/cell/struct.RefCell.html diff --git a/src/profile-guided-optimization.md b/src/profile-guided-optimization.md index 995752b0..39bc8b5e 100644 --- a/src/profile-guided-optimization.md +++ b/src/profile-guided-optimization.md @@ -120,7 +120,7 @@ The `rustc` version of this can be found in `library/profiler_builtins` which basically packs the C code from `compiler-rt` into a Rust crate. In order for `profiler_builtins` to be built, `profiler = true` must be set -in `rustc`'s `config.toml`. +in `rustc`'s `bootstrap.toml`. [compiler-rt-profile]: https://github.com/llvm/llvm-project/tree/main/compiler-rt/lib/profile diff --git a/src/profiling.md b/src/profiling.md index df987e00..de06bd7c 100644 --- a/src/profiling.md +++ b/src/profiling.md @@ -87,7 +87,7 @@ Example output for the compiler: Since this doesn't seem to work with incremental compilation or `./x check`, you will be compiling rustc _a lot_. -I recommend changing a few settings in `config.toml` to make it bearable: +I recommend changing a few settings in `bootstrap.toml` to make it bearable: ``` [rust] # A debug build takes _a third_ as long on my machine, diff --git a/src/profiling/with_perf.md b/src/profiling/with_perf.md index 51a22d18..742ea1c4 100644 --- a/src/profiling/with_perf.md +++ b/src/profiling/with_perf.md @@ -6,7 +6,7 @@ This is a guide for how to profile rustc with [perf](https://perf.wiki.kernel.or - Get a clean checkout of rust-lang/master, or whatever it is you want to profile. -- Set the following settings in your `config.toml`: +- Set the following settings in your `bootstrap.toml`: - `debuginfo-level = 1` - enables line debuginfo - `jemalloc = false` - lets you do memory use profiling with valgrind - leave everything else the defaults diff --git a/src/profiling/with_rustc_perf.md b/src/profiling/with_rustc_perf.md index eda8c3a1..c47fed24 100644 --- a/src/profiling/with_rustc_perf.md +++ b/src/profiling/with_rustc_perf.md @@ -9,7 +9,7 @@ which will download and build the suite for you, build a local compiler toolchai You can use the `./x perf [options]` command to use this integration. -You can use normal bootstrap flags for this command, such as `--stage 1` or `--stage 2`, for example to modify the stage of the created sysroot. It might also be useful to configure `config.toml` to better support profiling, e.g. set `rust.debuginfo-level = 1` to add source line information to the built compiler. +You can use normal bootstrap flags for this command, such as `--stage 1` or `--stage 2`, for example to modify the stage of the created sysroot. It might also be useful to configure `bootstrap.toml` to better support profiling, e.g. set `rust.debuginfo-level = 1` to add source line information to the built compiler. `x perf` currently supports the following commands: - `benchmark `: Benchmark the compiler and store the results under the passed `id`. diff --git a/src/profiling/wpa_profiling.md b/src/profiling/wpa_profiling.md index a800c571..d2680d40 100644 --- a/src/profiling/wpa_profiling.md +++ b/src/profiling/wpa_profiling.md @@ -44,7 +44,7 @@ compiler we're using to build rustc will aid our analysis greatly by allowing WP symbols correctly. Unfortunately, the stage 0 compiler does not have symbols turned on which is why we'll need to build a stage 1 compiler and then a stage 2 compiler ourselves. -To do this, make sure you have set `debuginfo-level = 1` in your `config.toml` file. This tells +To do this, make sure you have set `debuginfo-level = 1` in your `bootstrap.toml` file. This tells rustc to generate debug information which includes stack frames when bootstrapping. Now you can build the stage 1 compiler: `x build --stage 1 -i library` or however diff --git a/src/rustdoc.md b/src/rustdoc.md index 2a0e212f..35669814 100644 --- a/src/rustdoc.md +++ b/src/rustdoc.md @@ -58,12 +58,12 @@ does is call the `main()` that's in this crate's `lib.rs`, though.) * If you want to copy those docs to a webserver, copy all of `build/host/doc`, since that's where the CSS, JS, fonts, and landing page are. - * For frontend debugging, disable the `rust.docs-minification` option in [`config.toml`]. + * For frontend debugging, disable the `rust.docs-minification` option in [`bootstrap.toml`]. * Use `./x test tests/rustdoc*` to run the tests using a stage1 rustdoc. * See [Rustdoc internals] for more information about tests. -[`config.toml`]: ./building/how-to-build-and-run.md +[`bootstrap.toml`]: ./building/how-to-build-and-run.md ## Code structure diff --git a/src/sanitizers.md b/src/sanitizers.md index 73e888ea..b1654b15 100644 --- a/src/sanitizers.md +++ b/src/sanitizers.md @@ -32,7 +32,7 @@ implementation: * The sanitizer runtime libraries are part of the [compiler-rt] project, and [will be built][sanitizer-build] on [supported targets][sanitizer-targets] - when enabled in `config.toml`: + when enabled in `bootstrap.toml`: ```toml [build] @@ -80,7 +80,7 @@ Sanitizers are validated by code generation tests in [`tests/ui/sanitizer/`][test-ui] directory. Testing sanitizer functionality requires the sanitizer runtimes (built when -`sanitizer = true` in `config.toml`) and target providing support for particular +`sanitizer = true` in `bootstrap.toml`) and target providing support for particular sanitizer. When sanitizer is unsupported on given target, sanitizers tests will be ignored. This behaviour is controlled by compiletest `needs-sanitizer-*` directives. diff --git a/src/tests/ci.md b/src/tests/ci.md index 0c0f750a..cded565d 100644 --- a/src/tests/ci.md +++ b/src/tests/ci.md @@ -435,7 +435,7 @@ To learn more about the dashboard, see the [Datadog CI docs]. ## Determining the CI configuration -If you want to determine which `config.toml` settings are used in CI for a +If you want to determine which `bootstrap.toml` settings are used in CI for a particular job, it is probably easiest to just look at the build log. To do this: diff --git a/src/tests/compiletest.md b/src/tests/compiletest.md index a6996e39..2c35381e 100644 --- a/src/tests/compiletest.md +++ b/src/tests/compiletest.md @@ -525,10 +525,10 @@ data into a human-readable code coverage report. Instrumented binaries need to be linked against the LLVM profiler runtime, so `coverage-run` tests are **automatically skipped** unless the profiler runtime -is enabled in `config.toml`: +is enabled in `bootstrap.toml`: ```toml -# config.toml +# bootstrap.toml [build] profiler = true ``` diff --git a/src/tests/directives.md b/src/tests/directives.md index d5b896a8..81aa35f1 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -160,9 +160,9 @@ settings: stable support for `asm!` - `needs-profiler-runtime` — ignores the test if the profiler runtime was not enabled for the target - (`build.profiler = true` in rustc's `config.toml`) + (`build.profiler = true` in rustc's `bootstrap.toml`) - `needs-sanitizer-support` — ignores if the sanitizer support was not enabled - for the target (`sanitizers = true` in rustc's `config.toml`) + for the target (`sanitizers = true` in rustc's `bootstrap.toml`) - `needs-sanitizer-{address,hwaddress,leak,memory,thread}` — ignores if the corresponding sanitizer is not enabled for the target (AddressSanitizer, hardware-assisted AddressSanitizer, LeakSanitizer, MemorySanitizer or @@ -172,7 +172,7 @@ settings: flag, or running on fuchsia. - `needs-unwind` — ignores if the target does not support unwinding - `needs-rust-lld` — ignores if the rust lld support is not enabled (`rust.lld = - true` in `config.toml`) + true` in `bootstrap.toml`) - `needs-threads` — ignores if the target does not have threading support - `needs-subprocess` — ignores if the target does not have subprocess support - `needs-symlink` — ignores if the target does not support symlinks. This can be diff --git a/src/tests/docker.md b/src/tests/docker.md index a8a388ef..032da1ca 100644 --- a/src/tests/docker.md +++ b/src/tests/docker.md @@ -21,7 +21,7 @@ The [`src/ci/docker/run.sh`] script is used to build a specific Docker image, ru build Rust within the image, and either run tests or prepare a set of archives designed for distribution. The script will mount your local Rust source tree in read-only mode, and an `obj` directory in read-write mode. All the compiler artifacts will be stored in the `obj` directory. The shell will start out in the `obj`directory. From there, it will execute `../src/ci/run.sh` which starts the build as defined by the Docker image. You can run `src/ci/docker/run.sh ` directly. A few important notes regarding the `run.sh` script: -- When executed on CI, the script expects that all submodules are checked out. If some submodule that is accessed by the job is not available, the build will result in an error. You should thus make sure that you have all required submodules checked out locally. You can either do that manually through git, or set `submodules = true` in your `config.toml` and run a command such as `x build` to let bootstrap download the most important submodules (this might not be enough for the given CI job that you are trying to execute though). +- When executed on CI, the script expects that all submodules are checked out. If some submodule that is accessed by the job is not available, the build will result in an error. You should thus make sure that you have all required submodules checked out locally. You can either do that manually through git, or set `submodules = true` in your `bootstrap.toml` and run a command such as `x build` to let bootstrap download the most important submodules (this might not be enough for the given CI job that you are trying to execute though). - `` corresponds to a single directory located in one of the `src/ci/docker/host-*` directories. Note that image name does not necessarily correspond to a job name, as some jobs execute the same image, but with different environment variables or Docker build arguments (this is a part of the complexity that makes it difficult to run CI jobs locally). - If you are executing a "dist" job (job beginning with `dist-`), you should set the `DEPLOY=1` environment variable. - If you are executing an "alternative dist" job (job beginning with `dist-` and ending with `-alt`), you should set the `DEPLOY_ALT=1` environment variable. diff --git a/src/tests/running.md b/src/tests/running.md index 03d4123c..ab97fbf9 100644 --- a/src/tests/running.md +++ b/src/tests/running.md @@ -189,7 +189,7 @@ just like when running the tests without the `--bless` flag. There are a few options for running tests: -* `config.toml` has the `rust.verbose-tests` option. If `false`, each test will +* `bootstrap.toml` has the `rust.verbose-tests` option. If `false`, each test will print a single dot (the default). If `true`, the name of every test will be printed. This is equivalent to the `--quiet` option in the [Rust test harness](https://doc.rust-lang.org/rustc/tests/). @@ -353,7 +353,7 @@ coordinate running tests (see [src/bootstrap/src/core/build_steps/test.rs]). First thing to know is that it only supports linux x86_64 at the moment. We will extend its support later on. -You need to update `codegen-backends` value in your `config.toml` file in the +You need to update `codegen-backends` value in your `bootstrap.toml` file in the `[rust]` section and add "gcc" in the array: ```toml diff --git a/src/tracing.md b/src/tracing.md index af484ab5..0cfdf306 100644 --- a/src/tracing.md +++ b/src/tracing.md @@ -185,11 +185,11 @@ rustc. While calls to `error!`, `warn!` and `info!` are included in every build of the compiler, calls to `debug!` and `trace!` are only included in the program if -`debug-logging=true` is turned on in config.toml (it is +`debug-logging=true` is turned on in bootstrap.toml (it is turned off by default), so if you don't see `DEBUG` logs, especially if you run the compiler with `RUSTC_LOG=rustc rustc some.rs` and only see `INFO` logs, make sure that `debug-logging=true` is turned on in your -config.toml. +bootstrap.toml. ## Logging etiquette and conventions From bfe33b7ee060ebf712d85c5a0608e710b4ef05b4 Mon Sep 17 00:00:00 2001 From: Jieyou Xu Date: Tue, 18 Mar 2025 12:08:16 +0800 Subject: [PATCH 032/107] Preparing for merge from rustc --- rust-version | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust-version b/rust-version index eb779d9a..6baf4339 100644 --- a/rust-version +++ b/rust-version @@ -1 +1 @@ -8536f201ffdb2c24925d7f9e87996d7dca93428b +493c38ba371929579fe136df26eccd9516347c7a From e0d97048419550941b8b1b5a8b2feb96768872c6 Mon Sep 17 00:00:00 2001 From: xizheyin Date: Wed, 5 Mar 2025 13:18:17 +0800 Subject: [PATCH 033/107] Add chapter Remarks on perma-unstable features Signed-off-by: xizheyin --- src/SUMMARY.md | 1 + .../remarks-on-perma-unstable-features.md | 55 +++++++++++++++++++ 2 files changed, 56 insertions(+) create mode 100644 src/rustc-driver/remarks-on-perma-unstable-features.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 106db508..71b58f79 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -124,6 +124,7 @@ - [rustc_driver and rustc_interface](./rustc-driver/intro.md) - [Example: Type checking](./rustc-driver/interacting-with-the-ast.md) - [Example: Getting diagnostics](./rustc-driver/getting-diagnostics.md) + - [Remarks on perma-unstable features](./rustc-driver/remarks-on-perma-unstable-features.md) - [Errors and Lints](diagnostics.md) - [Diagnostic and subdiagnostic structs](./diagnostics/diagnostic-structs.md) - [Translation](./diagnostics/translation.md) diff --git a/src/rustc-driver/remarks-on-perma-unstable-features.md b/src/rustc-driver/remarks-on-perma-unstable-features.md new file mode 100644 index 00000000..c307adc0 --- /dev/null +++ b/src/rustc-driver/remarks-on-perma-unstable-features.md @@ -0,0 +1,55 @@ + +# Remarks on perma unstable features + +## `rustc_private` + +### Overview + +The `rustc_private` feature allows external crates to use compiler internals. + +### Using `rustc-private` with Official Toolchains + +When using the `rustc_private` feature with official Rust toolchains distributed via rustup, you need to install two additional components: + +1. **`rustc-dev`**: Provides compiler libraries +2. **`llvm-tools`**: Provides LLVM libraries required for linking + +#### Installation Steps + +Install both components using rustup: + +```bash +rustup component add rustc-dev llvm-tools +``` + +#### Common Error + +Without the `llvm-tools` component, you'll encounter linking errors like: + +``` +error: linking with `cc` failed: exit status: 1 + | + = note: rust-lld: error: unable to find library -lLLVM-{version} +``` + +### Using `rustc-private` with Custom Toolchains + +For custom-built toolchains or environments not using rustup, additional configuration is typically required: + +#### Requirements + +- LLVM libraries must be available in your system's library search paths +- The LLVM version must match the one used to build your Rust toolchain + +#### Troubleshooting Steps + +1. **Check LLVM installation**: Verify LLVM is installed and accessible +2. **Configure library paths**: You may need to set environment variables: + ```bash + export LD_LIBRARY_PATH=/path/to/llvm/lib:$LD_LIBRARY_PATH + ``` +3. **Check version compatibility**: Ensure your LLVM version is compatible with your Rust toolchain + +### Additional Resources + +- [GitHub Issue #137421](https://github.com/rust-lang/rust/issues/137421): Explains that `rustc_private` linker failures often occur because `llvm-tools` is not installed From db56758b57c00f545f05e4c3864dbf24752c9793 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Tue, 18 Mar 2025 16:16:28 +0100 Subject: [PATCH 034/107] Add Fuchsia ping group notice --- src/tests/fuchsia.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/tests/fuchsia.md b/src/tests/fuchsia.md index e96290b9..92680336 100644 --- a/src/tests/fuchsia.md +++ b/src/tests/fuchsia.md @@ -4,6 +4,14 @@ million lines of Rust code.[^loc] It has caught a large number of [regressions] in the past and was subsequently included in CI. +## What to do if the Fuchsia job breaks? + +Please contact the `fuchsia` ping group and ask them for help. + +```text +@rustbot ping fuchsia +``` + ## Building Fuchsia in CI Fuchsia builds as part of the suite of bors tests that run before a pull request From ba77a8030b2dfaf013f9e5b13184746ff2482c61 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Tue, 18 Mar 2025 16:17:59 +0100 Subject: [PATCH 035/107] Reorder RfL tests page to move the "what if it breaks" section to the top --- src/tests/rust-for-linux.md | 40 ++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/src/tests/rust-for-linux.md b/src/tests/rust-for-linux.md index c674d157..bdf32ffc 100644 --- a/src/tests/rust-for-linux.md +++ b/src/tests/rust-for-linux.md @@ -3,26 +3,7 @@ [Rust for Linux](https://rust-for-linux.com/) (RfL) is an effort for adding support for the Rust programming language into the Linux kernel. -## Building Rust for Linux in CI - -Rust for Linux builds as part of the suite of bors tests that run before a pull -request is merged. - -The workflow builds a stage1 sysroot of the Rust compiler, downloads the Linux -kernel, and tries to compile several Rust for Linux drivers and examples using -this sysroot. RfL uses several unstable compiler/language features, therefore -this workflow notifies us if a given compiler change would break it. - -If you are worried that a pull request might break the Rust for Linux builder -and want to test it out before submitting it to the bors queue, simply add this -line to your PR description: - -> try-job: x86_64-rust-for-linux - -Then when you `@bors try` it will pick the job that builds the Rust for Linux -integration. - -## What to do in case of failure +## What to do if the Rust for Linux job breaks? If a PR breaks the Rust for Linux CI job, then: @@ -48,4 +29,23 @@ ping group to ask for help: @rustbot ping rfl ``` +## Building Rust for Linux in CI + +Rust for Linux builds as part of the suite of bors tests that run before a pull +request is merged. + +The workflow builds a stage1 sysroot of the Rust compiler, downloads the Linux +kernel, and tries to compile several Rust for Linux drivers and examples using +this sysroot. RfL uses several unstable compiler/language features, therefore +this workflow notifies us if a given compiler change would break it. + +If you are worried that a pull request might break the Rust for Linux builder +and want to test it out before submitting it to the bors queue, simply add this +line to your PR description: + +> try-job: x86_64-rust-for-linux + +Then when you `@bors try` it will pick the job that builds the Rust for Linux +integration. + [rfl-ping]: ../notification-groups/rust-for-linux.md From 1bcd02da969fa0ec166dba3c577a233283db2378 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Tue, 18 Mar 2025 16:19:55 +0100 Subject: [PATCH 036/107] Add Fuchsia ping group page --- src/SUMMARY.md | 3 ++- src/notification-groups/fuchsia.md | 12 ++++++++++++ src/tests/fuchsia.md | 3 ++- 3 files changed, 16 insertions(+), 2 deletions(-) create mode 100644 src/notification-groups/fuchsia.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index a090f96f..9e2d0774 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -61,12 +61,13 @@ - [ARM](notification-groups/arm.md) - [Cleanup Crew](notification-groups/cleanup-crew.md) - [Emscripten](notification-groups/emscripten.md) + - [Fuchsia](notification-groups/fuchsia.md) - [LLVM](notification-groups/llvm.md) - [RISC-V](notification-groups/risc-v.md) + - [Rust for Linux](notification-groups/rust-for-linux.md) - [WASI](notification-groups/wasi.md) - [WebAssembly](notification-groups/wasm.md) - [Windows](notification-groups/windows.md) - - [Rust for Linux](notification-groups/rust-for-linux.md) - [Licenses](./licenses.md) - [Editions](guides/editions.md) diff --git a/src/notification-groups/fuchsia.md b/src/notification-groups/fuchsia.md new file mode 100644 index 00000000..a4658e0d --- /dev/null +++ b/src/notification-groups/fuchsia.md @@ -0,0 +1,12 @@ +# Fuchsia notification group + +**Github Label:** [O-fuchsia]
+**Ping command:** `@rustbot ping fuchsia` + +[O-fuchsia]: https://github.com/rust-lang/rust/labels/O-fuchsia + +This list will be used to notify [Fuchsia][fuchsia] maintainers +when the compiler or the standard library changes in a way that would +break the Fuchsia integration. + +[fuchsia]: ../tests/fuchsia.md diff --git a/src/tests/fuchsia.md b/src/tests/fuchsia.md index 92680336..2766c362 100644 --- a/src/tests/fuchsia.md +++ b/src/tests/fuchsia.md @@ -6,7 +6,7 @@ in the past and was subsequently included in CI. ## What to do if the Fuchsia job breaks? -Please contact the `fuchsia` ping group and ask them for help. +Please contact the [fuchsia][fuchsia-ping] ping group and ask them for help. ```text @rustbot ping fuchsia @@ -170,6 +170,7 @@ rustc book][platform-support]. [`public_configs`]: https://gn.googlesource.com/gn/+/main/docs/reference.md#var_public_configs [`//build/config:compiler`]: https://cs.opensource.google/fuchsia/fuchsia/+/main:build/config/BUILD.gn;l=121;drc=c26c473bef93b33117ae417893118907a026fec7 [build system]: https://fuchsia.dev/fuchsia-src/development/build/build_system +[fuchsia-ping]: ../notification-groups/fuchsia.md [^loc]: As of June 2024, Fuchsia had about 2 million lines of first-party Rust code and a roughly equal amount of third-party code, as counted by tokei From f00643aa1cb4ee35f619406fbd7934d6a1093d50 Mon Sep 17 00:00:00 2001 From: binarycat Date: Tue, 18 Mar 2025 12:04:47 -0500 Subject: [PATCH 037/107] add new section on the `rustdoc` test suite --- src/rustdoc-internals/htmldocck.md | 101 +++++++++++++++++++++++++++++ src/rustdoc.md | 14 ++-- src/tests/compiletest.md | 2 +- 3 files changed, 109 insertions(+), 8 deletions(-) create mode 100644 src/rustdoc-internals/htmldocck.md diff --git a/src/rustdoc-internals/htmldocck.md b/src/rustdoc-internals/htmldocck.md new file mode 100644 index 00000000..d69627f4 --- /dev/null +++ b/src/rustdoc-internals/htmldocck.md @@ -0,0 +1,101 @@ +# `rustdoc` tests + +This page is specifically about the `rustdoc` test suite, for other test suites used for testing rustdoc, see [Rustdoc § Tests](../rustdoc.md#tests). + +`htmldocck.py` is a custom checker script that uses [XPath] to verify the HTML output of rustdoc. + +[XPath]: https://en.wikipedia.org/wiki/XPath + +## Directives +Directives to htmldocck are similar to those given to `compiletest` in that they take the form of `//@` comments. + +All `PATH`s in directives are relative to the the rustdoc output directory (`build/TARGET/test/rustdoc/TESTNAME`), +so it is conventional to use a `#![crate_name = "foo"]` attribute to avoid writing paths. +To avoid repetion, `-` can be used in any `PATH` argument to re-use the previous `PATH` argument. + +All arguments take the form of quoted strings, +with the exception of `COUNT` and the special `-` form of `PATH`. + +Directives are assertions that place constraints on the generated HTML. + +All directives (except `files`) can be negated by putting a `!` in front of their name. + +Similar to shell commands, +directives can extend across multiple lines if their last char is `\`. +In this case, the start of the next line should be `//`, with no `@`. + +For example, `//@ !has 'foo/struct.Bar.html'` checks that crate `foo` does not have a page for a struct named `Bar` in the crate root. + +### `has` + +Usage 1: `//@ has PATH` +Usage 2: `//@ has PATH XPATH PATTERN` + +In the first form, `has` checks that a given file exists. + +In the second form, `has` is an alias for `matches`, +except `PATTERN` is a whitespace-normalized[^1] string instead of a regex. + +### `matches` + +Usage: `//@ matches PATH XPATH PATTERN` + +Checks that the text of each element selected by `XPATH` in `PATH` matches the python-flavored regex `PATTERN`. + +### `matchesraw` + +Usage: `//@ matchesraw PATH PATTERN` + +Checks that the contents of the file `PATH` matches the regex `PATTERN`. + +### `hasraw` + +Usage: `//@ hasraw PATH PATTERN` + +Same as `matchesraw`, except `PATTERN` is a whitespace-normalized[^1] string instead of a regex. + +### `count` + +Usage: `//@ count PATH XPATH COUNT` + +Checks that there are exactly `COUNT` matches for `XPATH` within the file `PATH`. + +### `snapshot` + +Usage: `//@ snapshot NAME PATH XPATH` + +Creates a snapshot test named NAME. +A snapshot test captures a subtree of the DOM, at the location +determined by the XPath, and compares it to a pre-recorded value +in a file. The file's name is the test's name with the `.rs` extension +replaced with `.NAME.html`, where NAME is the snapshot's name. + +htmldocck supports the `--bless` option to accept the current subtree +as expected, saving it to the file determined by the snapshot's name. +compiletest's `--bless` flag is forwarded to htmldocck. + +### `has-dir` + +Usage: `//@ has-dir PATH` + +Checks for the existance of directory `PATH`. + +### `files` + +Usage: `//@ files PATH ENTRIES` + +Checks that the directory `PATH` contains exactly `ENTRIES`. +`ENTRIES` is a python list of strings inside a quoted string, +as if it were to be parsed by `eval`. +(note that the list is actually parsed by `shlex.split`, +so it cannot contain arbitrary python expressions). + +Example: `//@ files "foo/bar" '["index.html", "sidebar-items.js"]'` + +[^1]: Whitespace normalization means that all spans of consecutive whitespace are replaced with a single space. The files themselves are also whitespace-normalized. + +## Limitations +All `XPATH` arguments must start with `//` due to a flaw in the implemention. + +Only well-formed HTML can be parsed (hopefully rustdoc doesn't output mismatched tags). + diff --git a/src/rustdoc.md b/src/rustdoc.md index 35669814..5ac62aa2 100644 --- a/src/rustdoc.md +++ b/src/rustdoc.md @@ -77,27 +77,27 @@ does is call the `main()` that's in this crate's `lib.rs`, though.) `doctest.rs`. * The Markdown renderer is loaded up in `html/markdown.rs`, including functions for extracting doctests from a given block of Markdown. -* The tests on the structure of rustdoc HTML output are located in `tests/rustdoc`, where - they're handled by the test runner of bootstrap and the supplementary script - `src/etc/htmldocck.py`. * Frontend CSS and JavaScript are stored in `html/static/`. ## Tests -* All paths in this section are relative to `tests` in the rust-lang/rust repository. -* Tests on search engine and index are located in `rustdoc-js` and `rustdoc-js-std`. +* Tests on search engine and index are located in `tests/rustdoc-js` and `tests/rustdoc-js-std`. The format is specified [in the search guide](rustdoc-internals/search.md#testing-the-search-engine). * Tests on the "UI" of rustdoc (the terminal output it produces when run) are in - `rustdoc-ui` + `tests/rustdoc-ui` * Tests on the "GUI" of rustdoc (the HTML, JS, and CSS as rendered in a browser) - are in `rustdoc-gui`. These use a [NodeJS tool called + are in `tests/rustdoc-gui`. These use a [NodeJS tool called browser-UI-test](https://github.com/GuillaumeGomez/browser-UI-test/) that uses puppeteer to run tests in a headless browser and check rendering and interactivity. * Additionally, JavaScript type annotations are written using [TypeScript-flavored JSDoc] comments and an external d.ts file. The code itself is plain, valid JavaScript; we only use tsc as a linter. +* The tests on the structure of rustdoc HTML output are located in `tests/rustdoc`, + where they're handled by the test runner of bootstrap and + the supplementary script `src/etc/htmldocck.py`. + [These tests have several extra directives available to them](./rustdoc-internals/htmldocck.md). [TypeScript-flavored JSDoc]: https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html diff --git a/src/tests/compiletest.md b/src/tests/compiletest.md index 2c35381e..ee66b787 100644 --- a/src/tests/compiletest.md +++ b/src/tests/compiletest.md @@ -78,7 +78,7 @@ The following test suites are available, with links for more information: ### Rustdoc test suites -See [Rustdoc tests](../rustdoc.md#tests) for more details. +See [Rustdoc § Tests](../rustdoc.md#tests) for more details. | Test suite | Purpose | |------------------|--------------------------------------------------------------------------| From 8b501562a8e0b21fb304394a3cb6df18c0f54cc3 Mon Sep 17 00:00:00 2001 From: binarycat Date: Tue, 18 Mar 2025 12:10:43 -0500 Subject: [PATCH 038/107] add htmldocck.md to SUMMARY.md --- src/SUMMARY.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 9e2d0774..fb468777 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -95,7 +95,7 @@ - [Parallel Compilation](./parallel-rustc.md) - [Rustdoc internals](./rustdoc-internals.md) - [Search](./rustdoc-internals/search.md) - + - [`rustdoc` tests](./rustdoc-internals/htmldocck.md) # Source Code Representation - [Prologue](./part-3-intro.md) From f248d2f57cd99ae6f4962582b9ecc574b60ceb05 Mon Sep 17 00:00:00 2001 From: binarycat Date: Tue, 18 Mar 2025 13:10:26 -0500 Subject: [PATCH 039/107] htmldocck: expand limitations and mention compiletest directives --- src/rustdoc-internals/htmldocck.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/src/rustdoc-internals/htmldocck.md b/src/rustdoc-internals/htmldocck.md index d69627f4..b37c1ac3 100644 --- a/src/rustdoc-internals/htmldocck.md +++ b/src/rustdoc-internals/htmldocck.md @@ -9,6 +9,10 @@ This page is specifically about the `rustdoc` test suite, for other test suites ## Directives Directives to htmldocck are similar to those given to `compiletest` in that they take the form of `//@` comments. +In addition to the directives listed here, +`rustdoc` tests also support most +[compiletest directives](../tests/directives.html). + All `PATH`s in directives are relative to the the rustdoc output directory (`build/TARGET/test/rustdoc/TESTNAME`), so it is conventional to use a `#![crate_name = "foo"]` attribute to avoid writing paths. To avoid repetion, `-` can be used in any `PATH` argument to re-use the previous `PATH` argument. @@ -95,7 +99,9 @@ Example: `//@ files "foo/bar" '["index.html", "sidebar-items.js"]'` [^1]: Whitespace normalization means that all spans of consecutive whitespace are replaced with a single space. The files themselves are also whitespace-normalized. ## Limitations -All `XPATH` arguments must start with `//` due to a flaw in the implemention. - -Only well-formed HTML can be parsed (hopefully rustdoc doesn't output mismatched tags). +`htmldocck.py` uses the xpath implementation from the standard library. +This leads to several limitations: +* All `XPATH` arguments must start with `//` due to a flaw in the implemention. +* Many XPath features (functions, axies, etc.) are not supported. +* Only well-formed HTML can be parsed (hopefully rustdoc doesn't output mismatched tags). From 72aa06dff1dcfd3516aded79399336932c290976 Mon Sep 17 00:00:00 2001 From: binarycat Date: Tue, 18 Mar 2025 13:19:33 -0500 Subject: [PATCH 040/107] rustdoc test suite: clean up wording and intro --- src/rustdoc-internals/htmldocck.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/rustdoc-internals/htmldocck.md b/src/rustdoc-internals/htmldocck.md index b37c1ac3..97c40c8a 100644 --- a/src/rustdoc-internals/htmldocck.md +++ b/src/rustdoc-internals/htmldocck.md @@ -1,8 +1,10 @@ -# `rustdoc` tests +# the `rustdoc` test suite This page is specifically about the `rustdoc` test suite, for other test suites used for testing rustdoc, see [Rustdoc § Tests](../rustdoc.md#tests). -`htmldocck.py` is a custom checker script that uses [XPath] to verify the HTML output of rustdoc. +The `rustdoc` test suite is specifically used to test the HTML output of rustdoc. + +This is achived by means of `htmldocck.py`, a custom checker script that leverages [XPath]. [XPath]: https://en.wikipedia.org/wiki/XPath @@ -14,7 +16,8 @@ In addition to the directives listed here, [compiletest directives](../tests/directives.html). All `PATH`s in directives are relative to the the rustdoc output directory (`build/TARGET/test/rustdoc/TESTNAME`), -so it is conventional to use a `#![crate_name = "foo"]` attribute to avoid writing paths. +so it is conventional to use a `#![crate_name = "foo"]` attribute to avoid +having to write a long crate name multiple times. To avoid repetion, `-` can be used in any `PATH` argument to re-use the previous `PATH` argument. All arguments take the form of quoted strings, From 174678da355231d76700710caca84b5439474c9c Mon Sep 17 00:00:00 2001 From: binarycat Date: Tue, 18 Mar 2025 13:21:15 -0500 Subject: [PATCH 041/107] rename htmldocck.md -> rustdoc-test-suite.md --- src/SUMMARY.md | 2 +- src/rustdoc-internals/{htmldocck.md => rustdoc-test-suite.md} | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename src/rustdoc-internals/{htmldocck.md => rustdoc-test-suite.md} (100%) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index fb468777..8ea5e86c 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -95,7 +95,7 @@ - [Parallel Compilation](./parallel-rustc.md) - [Rustdoc internals](./rustdoc-internals.md) - [Search](./rustdoc-internals/search.md) - - [`rustdoc` tests](./rustdoc-internals/htmldocck.md) + - [the `rustdoc` test suite](./rustdoc-internals/rustdoc-test-suite.md) # Source Code Representation - [Prologue](./part-3-intro.md) diff --git a/src/rustdoc-internals/htmldocck.md b/src/rustdoc-internals/rustdoc-test-suite.md similarity index 100% rename from src/rustdoc-internals/htmldocck.md rename to src/rustdoc-internals/rustdoc-test-suite.md From 2e1c4999c812e7c8f5af44a9f902d908c672dfe9 Mon Sep 17 00:00:00 2001 From: binarycat Date: Tue, 18 Mar 2025 13:23:37 -0500 Subject: [PATCH 042/107] clean up wording/grammar and mention double quotes --- src/SUMMARY.md | 2 +- src/rustdoc-internals/rustdoc-test-suite.md | 7 ++++--- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 8ea5e86c..075d5af4 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -95,7 +95,7 @@ - [Parallel Compilation](./parallel-rustc.md) - [Rustdoc internals](./rustdoc-internals.md) - [Search](./rustdoc-internals/search.md) - - [the `rustdoc` test suite](./rustdoc-internals/rustdoc-test-suite.md) + - [The `rustdoc` test suite](./rustdoc-internals/rustdoc-test-suite.md) # Source Code Representation - [Prologue](./part-3-intro.md) diff --git a/src/rustdoc-internals/rustdoc-test-suite.md b/src/rustdoc-internals/rustdoc-test-suite.md index 97c40c8a..fb4be51c 100644 --- a/src/rustdoc-internals/rustdoc-test-suite.md +++ b/src/rustdoc-internals/rustdoc-test-suite.md @@ -1,6 +1,6 @@ -# the `rustdoc` test suite +# The `rustdoc` test suite -This page is specifically about the `rustdoc` test suite, for other test suites used for testing rustdoc, see [Rustdoc § Tests](../rustdoc.md#tests). +This page is specifically about the test suite named `rustdoc`, for other test suites used for testing rustdoc, see [Rustdoc Tests](../rustdoc.md#tests). The `rustdoc` test suite is specifically used to test the HTML output of rustdoc. @@ -20,7 +20,8 @@ so it is conventional to use a `#![crate_name = "foo"]` attribute to avoid having to write a long crate name multiple times. To avoid repetion, `-` can be used in any `PATH` argument to re-use the previous `PATH` argument. -All arguments take the form of quoted strings, +All arguments take the form of quoted strings +(both single and double quotes are supported), with the exception of `COUNT` and the special `-` form of `PATH`. Directives are assertions that place constraints on the generated HTML. From 871280d6dfb5c9b6d0962749d9c3c19b839b7872 Mon Sep 17 00:00:00 2001 From: binarycat Date: Tue, 18 Mar 2025 13:25:39 -0500 Subject: [PATCH 043/107] normalize link titles --- src/rustdoc-internals/rustdoc-test-suite.md | 2 +- src/tests/compiletest.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/rustdoc-internals/rustdoc-test-suite.md b/src/rustdoc-internals/rustdoc-test-suite.md index fb4be51c..c2a2e649 100644 --- a/src/rustdoc-internals/rustdoc-test-suite.md +++ b/src/rustdoc-internals/rustdoc-test-suite.md @@ -1,6 +1,6 @@ # The `rustdoc` test suite -This page is specifically about the test suite named `rustdoc`, for other test suites used for testing rustdoc, see [Rustdoc Tests](../rustdoc.md#tests). +This page is specifically about the test suite named `rustdoc`, for other test suites used for testing rustdoc, see [Rustdoc tests](../rustdoc.md#tests). The `rustdoc` test suite is specifically used to test the HTML output of rustdoc. diff --git a/src/tests/compiletest.md b/src/tests/compiletest.md index ee66b787..2c35381e 100644 --- a/src/tests/compiletest.md +++ b/src/tests/compiletest.md @@ -78,7 +78,7 @@ The following test suites are available, with links for more information: ### Rustdoc test suites -See [Rustdoc § Tests](../rustdoc.md#tests) for more details. +See [Rustdoc tests](../rustdoc.md#tests) for more details. | Test suite | Purpose | |------------------|--------------------------------------------------------------------------| From 59f11cdfe58f0b42525e129320670bb725aaebc8 Mon Sep 17 00:00:00 2001 From: binarycat Date: Tue, 18 Mar 2025 13:29:14 -0500 Subject: [PATCH 044/107] update filename in link --- src/rustdoc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/rustdoc.md b/src/rustdoc.md index 5ac62aa2..320dc9d5 100644 --- a/src/rustdoc.md +++ b/src/rustdoc.md @@ -97,7 +97,7 @@ does is call the `main()` that's in this crate's `lib.rs`, though.) * The tests on the structure of rustdoc HTML output are located in `tests/rustdoc`, where they're handled by the test runner of bootstrap and the supplementary script `src/etc/htmldocck.py`. - [These tests have several extra directives available to them](./rustdoc-internals/htmldocck.md). + [These tests have several extra directives available to them](./rustdoc-internals/rustdoc-test-suite.md). [TypeScript-flavored JSDoc]: https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html From 531e5f9d35cea141f8205fb137e4135b5fb87709 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Wed, 19 Mar 2025 09:36:05 +0100 Subject: [PATCH 045/107] Set linkcheck in `ci.yml` --- .github/workflows/ci.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 22a4fb19..415d0dc3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -18,6 +18,7 @@ jobs: MDBOOK_LINKCHECK2_VERSION: 0.9.1 MDBOOK_MERMAID_VERSION: 0.12.6 MDBOOK_TOC_VERSION: 0.11.2 + MDBOOK_OUTPUT__LINKCHECK__FOLLOW_WEB_LINKS: ${{ github.event_name != 'pull_request' }} DEPLOY_DIR: book/html BASE_SHA: ${{ github.event.pull_request.base.sha }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} From 86aa63456e4e5747c9c57a70dc047e5bb5252dbd Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Wed, 19 Mar 2025 18:06:50 +0200 Subject: [PATCH 046/107] use correct code block markers This makes this command pass mdbook test --chapter "Remarks on perma-unstable features" --- src/rustc-driver/remarks-on-perma-unstable-features.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/src/rustc-driver/remarks-on-perma-unstable-features.md b/src/rustc-driver/remarks-on-perma-unstable-features.md index c307adc0..b434cfc9 100644 --- a/src/rustc-driver/remarks-on-perma-unstable-features.md +++ b/src/rustc-driver/remarks-on-perma-unstable-features.md @@ -1,4 +1,3 @@ - # Remarks on perma unstable features ## `rustc_private` @@ -18,7 +17,7 @@ When using the `rustc_private` feature with official Rust toolchains distributed Install both components using rustup: -```bash +```text rustup component add rustc-dev llvm-tools ``` @@ -26,7 +25,7 @@ rustup component add rustc-dev llvm-tools Without the `llvm-tools` component, you'll encounter linking errors like: -``` +```text error: linking with `cc` failed: exit status: 1 | = note: rust-lld: error: unable to find library -lLLVM-{version} @@ -45,7 +44,7 @@ For custom-built toolchains or environments not using rustup, additional configu 1. **Check LLVM installation**: Verify LLVM is installed and accessible 2. **Configure library paths**: You may need to set environment variables: - ```bash + ```text export LD_LIBRARY_PATH=/path/to/llvm/lib:$LD_LIBRARY_PATH ``` 3. **Check version compatibility**: Ensure your LLVM version is compatible with your Rust toolchain From 0da80bbea1399badc98b2734201d50b7ddee418d Mon Sep 17 00:00:00 2001 From: Jieyou Xu Date: Thu, 20 Mar 2025 12:40:51 +0800 Subject: [PATCH 047/107] Disambiguate between wg-llvm and icebreakers-llvm in rustc-dev-guide --- src/notification-groups/about.md | 4 ++-- src/notification-groups/llvm.md | 13 ++++++++----- 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/src/notification-groups/about.md b/src/notification-groups/about.md index 74629aa0..af305f01 100644 --- a/src/notification-groups/about.md +++ b/src/notification-groups/about.md @@ -23,7 +23,7 @@ Here's the list of the notification groups: - [ARM](./arm.md) - [Cleanup Crew](./cleanup-crew.md) - [Emscripten](./emscripten.md) -- [LLVM](./llvm.md) +- [LLVM Icebreakers](./llvm.md) - [RISC-V](./risc-v.md) - [WASI](./wasi.md) - [WebAssembly](./wasm.md) @@ -83,7 +83,7 @@ group. For example: @rustbot ping arm @rustbot ping cleanup-crew @rustbot ping emscripten -@rustbot ping llvm +@rustbot ping icebreakers-llvm @rustbot ping risc-v @rustbot ping wasi @rustbot ping wasm diff --git a/src/notification-groups/llvm.md b/src/notification-groups/llvm.md index 2eff6371..9d008728 100644 --- a/src/notification-groups/llvm.md +++ b/src/notification-groups/llvm.md @@ -1,13 +1,16 @@ -# LLVM Notification group +# LLVM Icebreakers Notification group **Github Label:** [A-LLVM]
-**Ping command:** `@rustbot ping llvm` +**Ping command:** `@rustbot ping icebreakers-llvm` [A-LLVM]: https://github.com/rust-lang/rust/labels/A-LLVM -The "LLVM Notification Group" are focused on bugs that center around LLVM. -These bugs often arise because of LLVM optimizations gone awry, or as -the result of an LLVM upgrade. The goal here is: +*Note*: this notification group is *not* the same as the LLVM working group +(WG-llvm). + +The "LLVM Icebreakers Notification Group" are focused on bugs that center around +LLVM. These bugs often arise because of LLVM optimizations gone awry, or as the +result of an LLVM upgrade. The goal here is: - to determine whether the bug is a result of us generating invalid LLVM IR, or LLVM misoptimizing; From b5522c1f58497500ccee7e40b9e945c1daf30fd2 Mon Sep 17 00:00:00 2001 From: Boxy Date: Thu, 20 Mar 2025 17:30:18 +0000 Subject: [PATCH 048/107] Update `ParamEnv` section for `TypingEnv` changes --- src/SUMMARY.md | 5 +- src/appendix/code-index.md | 2 +- src/const-eval.md | 2 +- src/param_env/param_env_acquisition.md | 43 ---- .../param_env_construction_internals.md | 83 ------- src/param_env/param_env_summary.md | 18 -- src/param_env/param_env_what_is_it.md | 59 ----- src/traits/caching.md | 2 +- src/traits/resolution.md | 2 +- src/typing_parameter_envs.md | 206 ++++++++++++++++++ 10 files changed, 211 insertions(+), 211 deletions(-) delete mode 100644 src/param_env/param_env_acquisition.md delete mode 100644 src/param_env/param_env_construction_internals.md delete mode 100644 src/param_env/param_env_summary.md delete mode 100644 src/param_env/param_env_what_is_it.md create mode 100644 src/typing_parameter_envs.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 075d5af4..c5d04601 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -146,10 +146,7 @@ - [ADTs and Generic Arguments](./ty_module/generic_arguments.md) - [Parameter types/consts/regions](./ty_module/param_ty_const_regions.md) - [`TypeFolder` and `TypeFoldable`](./ty-fold.md) -- [Parameter Environments](./param_env/param_env_summary.md) - - [What is it?](./param_env/param_env_what_is_it.md) - - [How are `ParamEnv`'s constructed internally](./param_env/param_env_construction_internals.md) - - [Which `ParamEnv` do I use?](./param_env/param_env_acquisition.md) +- [Typing/Param Envs](./typing_parameter_envs.md) - [Type inference](./type-inference.md) - [Trait solving](./traits/resolution.md) - [Higher-ranked trait bounds](./traits/hrtb.md) diff --git a/src/appendix/code-index.md b/src/appendix/code-index.md index b96ede68..65fbf752 100644 --- a/src/appendix/code-index.md +++ b/src/appendix/code-index.md @@ -40,5 +40,5 @@ Item | Kind | Short description | Chapter | [Emitting Diagnostics]: ../diagnostics.html [Macro expansion]: ../macro-expansion.html [Name resolution]: ../name-resolution.html -[Parameter Environment]: ../param_env/param_env_summary.html +[Parameter Environment]: ../typing_parameter_envs.html [Trait Solving: Goals and Clauses]: ../traits/goals-and-clauses.html#domain-goals diff --git a/src/const-eval.md b/src/const-eval.md index 69329a3e..ca6a35a5 100644 --- a/src/const-eval.md +++ b/src/const-eval.md @@ -35,7 +35,7 @@ They're the wrappers of the `const_eval` query. Statics are special; all other functions do not represent statics correctly and have thus assertions preventing their use on statics. -The `const_eval_*` functions use a [`ParamEnv`](./param_env/param_env_summary.html) of environment +The `const_eval_*` functions use a [`ParamEnv`](./typing_parameter_envs.html) of environment in which the constant is evaluated (e.g. the function within which the constant is used) and a [`GlobalId`]. The `GlobalId` is made up of an `Instance` referring to a constant or static or of an `Instance` of a function and an index into the function's `Promoted` table. diff --git a/src/param_env/param_env_acquisition.md b/src/param_env/param_env_acquisition.md deleted file mode 100644 index f6cff2d6..00000000 --- a/src/param_env/param_env_acquisition.md +++ /dev/null @@ -1,43 +0,0 @@ - -# Which `ParamEnv` do I use? - -When needing a [`ParamEnv`][pe] in the compiler there are a few options for obtaining one: -- The correct env is already in scope simply use it (or pass it down the call stack to where you are). -- The [`tcx.param_env(def_id)` query][param_env_query] -- Use [`ParamEnv::new`][param_env_new] to construct an env with an arbitrary set of where clauses. Then call [`traits::normalize_param_env_or_error`][normalize_env_or_error] which will handle normalizing and elaborating all the where clauses in the env for you. -- Creating an empty environment via [`ParamEnv::reveal_all`][env_reveal_all] or [`ParamEnv::empty`][env_empty] - -In the large majority of cases a `ParamEnv` when required already exists somewhere in scope or above in the call stack and should be passed down. A non exhaustive list of places where you might find an existing `ParamEnv`: -- During typeck `FnCtxt` has a [`param_env` field][fnctxt_param_env] -- When writing late lints the `LateContext` has a [`param_env` field][latectxt_param_env] -- During well formedness checking the `WfCheckingCtxt` has a [`param_env` field][wfckctxt_param_env] -- The `TypeChecker` used by Mir Typeck has a [`param_env` field][mirtypeck_param_env] -- In the next-gen trait solver all `Goal`s have a [`param_env` field][goal_param_env] specifying what environment to prove the goal in -- When editing an existing [`TypeRelation`][typerelation] if it implements `PredicateEmittingRelation` then a [`param_env` method][typerelation_param_env] will be available. - -Using the `param_env` query to obtain an env is generally done at the start of some kind of analysis and then passed everywhere that a `ParamEnv` is required. For example the type checker will create a `ParamEnv` for the item it is type checking and then pass it around everywhere. - -Creating an env from an arbitrary set of where clauses is usually unnecessary and should only be done if the environment you need does not correspond to an actual item in the source code (i.e. [`compare_method_predicate_entailment`][method_pred_entailment] as mentioned earlier). - -Creating an empty environment via `ParamEnv::empty` is almost always wrong. There are very few places where we actually know that the environment should be empty. One of the only places where we do actually know this is after monomorphization, however the `ParamEnv` there should be constructed via `ParamEnv::reveal_all` instead as at this point we should be able to determine the hidden type of opaque types. Codegen/Post-mono is one of the only places that should be using `ParamEnv::reveal_all`. - -An additional piece of complexity here is specifying the `Reveal` (see linked docs for explanation of what reveal does) used for the `ParamEnv`. When constructing a param env using the `param_env` query it will have `Reveal::UserFacing`, if `Reveal::All` is desired then the [`tcx.param_env_reveal_all_normalized`][env_reveal_all_normalized] query can be used instead. - -The `ParamEnv` type has a method [`ParamEnv::with_reveal_all_normalized`][with_reveal_all] which converts an existing `ParamEnv` into one with `Reveal::All` specified. Where possible the previously mentioned query should be preferred as it is more efficient. - -[param_env_new]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.new -[normalize_env_or_error]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/traits/fn.normalize_param_env_or_error.html -[fnctxt_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_typeck/fn_ctxt/struct.FnCtxt.html#structfield.param_env -[latectxt_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/context/struct.LateContext.html#structfield.param_env -[wfckctxt_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/wfcheck/struct.WfCheckingCtxt.html#structfield.param_env -[goal_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/infer/canonical/ir/solve/struct.Goal.html#structfield.param_env -[typerelation_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/infer/trait.PredicateEmittingRelation.html#tymethod.param_env -[typerelation]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/relate/trait.TypeRelation.html -[mirtypeck_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_borrowck/type_check/struct.TypeChecker.html#structfield.param_env -[env_reveal_all_normalized]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.param_env_reveal_all_normalized -[with_reveal_all]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.with_reveal_all_normalized -[env_reveal_all]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.reveal_all -[env_empty]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.empty -[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html -[param_env_query]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_typeck/fn_ctxt/struct.FnCtxt.html#structfield.param_env -[method_pred_entailment]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/compare_impl_item/fn.compare_method_predicate_entailment.html diff --git a/src/param_env/param_env_construction_internals.md b/src/param_env/param_env_construction_internals.md deleted file mode 100644 index 69a262a1..00000000 --- a/src/param_env/param_env_construction_internals.md +++ /dev/null @@ -1,83 +0,0 @@ - -# How are `ParamEnv`'s constructed internally? - -Creating a [`ParamEnv`][pe] is more complicated than simply using the list of where clauses defined on an item as written by the user. We need to both elaborate supertraits into the env and fully normalize all aliases. This logic is handled by [`traits::normalize_param_env_or_error`][normalize_env_or_error] (even though it does not mention anything about elaboration). - -## Elaborating supertraits - -When we have a function such as `fn foo()` we would like to be able to prove `T: Clone` inside of the function as the `Copy` trait has a `Clone` supertrait. Constructing a `ParamEnv` looks at all of the trait bounds in the env and explicitly adds new where clauses to the `ParamEnv` for any supertraits found on the traits. - -A concrete example would be the following function: -```rust -trait Trait: SuperTrait {} -trait SuperTrait: SuperSuperTrait {} - -// `bar`'s unelaborated `ParamEnv` would be: -// `[T: Sized, T: Copy, T: Trait]` -fn bar(a: T) { - requires_impl(a); -} - -fn requires_impl(a: T) {} -``` - -If we did not elaborate the env then the `requires_impl` call would fail to typecheck as we would not be able to prove `T: Clone` or `T: SuperSuperTrait`. In practice we elaborate the env which means that `bar`'s `ParamEnv` is actually: -`[T: Sized, T: Copy, T: Clone, T: Trait, T: SuperTrait, T: SuperSuperTrait]` -This allows us to prove `T: Clone` and `T: SuperSuperTrait` when type checking `bar`. - -The `Clone` trait has a `Sized` supertrait however we do not end up with two `T: Sized` bounds in the env (one for the supertrait and one for the implicitly added `T: Sized` bound). This is because the elaboration process (implemented via [`util::elaborate`][elaborate]) deduplicates the where clauses to avoid this. - -As a side effect this also means that even if no actual elaboration of supertraits takes place, the existing where clauses in the env are _also_ deduplicated. See the following example: -```rust -trait Trait {} -// The unelaborated `ParamEnv` would be: -// `[T: Sized, T: Trait, T: Trait]` -// but after elaboration it would be: -// `[T: Sized, T: Trait]` -fn foo() {} -``` - -The [next-gen trait solver][next-gen-solver] also requires this elaboration to take place. - -[elaborate]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/traits/util/fn.elaborate.html -[next-gen-solver]: ../solve/trait-solving.md - -## Normalizing all bounds - -In the old trait solver the where clauses stored in `ParamEnv` are required to be fully normalized or else the trait solver will not function correctly. A concrete example of needing to normalize the `ParamEnv` is the following: -```rust -trait Trait { - type Assoc; -} - -trait Other { - type Bar; -} - -impl Other for T { - type Bar = u32; -} - -// `foo`'s unnormalized `ParamEnv` would be: -// `[T: Sized, U: Sized, U: Trait]` -fn foo(a: U) -where - U: Trait<::Bar>, -{ - requires_impl(a); -} - -fn requires_impl>(_: U) {} -``` - -As humans we can tell that `::Bar` is equal to `u32` so the trait bound on `U` is equivalent to `U: Trait`. In practice trying to prove `U: Trait` in the old solver in this environment would fail as it is unable to determine that `::Bar` is equal to `u32`. - -To work around this we normalize `ParamEnv`'s after constructing them so that `foo`'s `ParamEnv` is actually: `[T: Sized, U: Sized, U: Trait]` which means the trait solver is now able to use the `U: Trait` in the `ParamEnv` to determine that the trait bound `U: Trait` holds. - -This workaround does not work in all cases as normalizing associated types requires a `ParamEnv` which introduces a bootstrapping problem. We need a normalized `ParamEnv` in order for normalization to give correct results, but we need to normalize to get that `ParamEnv`. Currently we normalize the `ParamEnv` once using the unnormalized param env and it tends to give okay results in practice even though there are some examples where this breaks ([example]). - -In the next-gen trait solver the requirement for all where clauses in the `ParamEnv` to be fully normalized is not present and so we do not normalize when constructing `ParamEnv`s. - -[example]: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=e6933265ea3e84eaa47019465739992c -[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html -[normalize_env_or_error]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/traits/fn.normalize_param_env_or_error.html diff --git a/src/param_env/param_env_summary.md b/src/param_env/param_env_summary.md deleted file mode 100644 index 0ff6d8fc..00000000 --- a/src/param_env/param_env_summary.md +++ /dev/null @@ -1,18 +0,0 @@ -# The `ParamEnv` type - -## Summary - -The [`ParamEnv`][pe] is used to store information about the environment that we are interacting with the type system from. For example the set of in-scope where-clauses is stored in `ParamEnv` as it differs between each item whereas the list of user written impls is not stored in the `ParamEnv` as this does not change for each item. - -This chapter of the dev guide covers: -- A high level summary of what a `ParamEnv` is and what it is used for -- Technical details about what the process of constructing a `ParamEnv` involves -- Guidance about how to acquire a `ParamEnv` when one is required - -## Bundling - -A useful API on `ParamEnv` is the [`and`][and] method which allows bundling a value with the `ParamEnv`. The `and` method produces a [`ParamEnvAnd`][pea] making it clearer that using the inner value is intended to be done in that specific environment. - -[and]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.and -[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html -[pea]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnvAnd.html \ No newline at end of file diff --git a/src/param_env/param_env_what_is_it.md b/src/param_env/param_env_what_is_it.md deleted file mode 100644 index 5c2f4d59..00000000 --- a/src/param_env/param_env_what_is_it.md +++ /dev/null @@ -1,59 +0,0 @@ - -# What is a `ParamEnv`? - -The type system relies on information in the environment in order for it to function correctly. This information is stored in the [`ParamEnv`][pe] type and it is important to use the correct `ParamEnv` when interacting with the type system. - -The information represented by `ParamEnv` is a list of in-scope where-clauses, and a `Reveal` (see linked docs for more information). A `ParamEnv` typically corresponds to a specific item's where clauses, some clauses are not explicitly written bounds and instead are implicitly added in [`predicates_of`][predicates_of] such as `ConstArgHasType` or some implied bounds. - -A `ParamEnv` can also be created with arbitrary data that is not derived from a specific item such as in [`compare_method_predicate_entailment`][method_pred_entailment] which creates a hybrid `ParamEnv` consisting of the impl's where clauses and the trait definition's function's where clauses. In most cases `ParamEnv`s are initially created via the [`param_env` query][query] which returns a `ParamEnv` derived from the provided item's where clauses. - -If we have a function such as: -```rust -// `foo` would have a `ParamEnv` of: -// `[T: Sized, T: Trait, ::Assoc: Clone]` -fn foo() -where - ::Assoc: Clone, -{} -``` -If we were conceptually inside of `foo` (for example, type-checking or linting it) we would use this `ParamEnv` everywhere that we interact with the type system. This would allow things such as normalization (TODO: write a chapter about normalization and link it), evaluating generic constants, and proving where clauses/goals, to rely on `T` being sized, implementing `Trait`, etc. - -A more concrete example: -```rust -// `foo` would have a `ParamEnv` of: -// `[T: Sized, T: Clone]` -fn foo(a: T) { - // when typechecking `foo` we require all the where clauses on `bar` - // to hold in order for it to be legal to call. This means we have to - // prove `T: Clone`. As we are type checking `foo` we use `foo`'s - // environment when trying to check that `T: Clone` holds. - // - // Trying to prove `T: Clone` with a `ParamEnv` of `[T: Sized, T: Clone]` - // will trivially succeed as bound we want to prove is in our environment. - requires_clone(a); -} -``` - -Or alternatively an example that would not compile: -```rust -// `foo2` would have a `ParamEnv` of: -// `[T: Sized]` -fn foo2(a: T) { - // When typechecking `foo2` we attempt to prove `T: Clone`. - // As we are type checking `foo2` we use `foo2`'s environment - // when trying to prove `T: Clone`. - // - // Trying to prove `T: Clone` with a `ParamEnv` of `[T: Sized]` will - // fail as there is nothing in the environment telling the trait solver - // that `T` implements `Clone` and there exists no user written impl - // that could apply. - requires_clone(a); -} -``` - -It's very important to use the correct `ParamEnv` when interacting with the type system as otherwise it can lead to ICEs or things compiling when they shouldn't (or vice versa). See [#82159](https://github.com/rust-lang/rust/pull/82159) and [#82067](https://github.com/rust-lang/rust/pull/82067) as examples of PRs that changed rustc to use the correct param env to avoid ICE. Determining how to acquire the correct `ParamEnv` is explained later in this chapter. - -[predicates_of]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/collect/predicates_of/fn.predicates_of.html -[method_pred_entailment]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/compare_impl_item/fn.compare_method_predicate_entailment.html -[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html -[query]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.param_env diff --git a/src/traits/caching.md b/src/traits/caching.md index a9f20969..c44722a1 100644 --- a/src/traits/caching.md +++ b/src/traits/caching.md @@ -61,7 +61,7 @@ to be pretty clearly safe and also still retains a very high hit rate **TODO**: it looks like `pick_candidate_cache` no longer exists. In general, is this section still accurate at all? -[`ParamEnv`]: ../param_env/param_env_summary.html +[`ParamEnv`]: ../typing_parameter_envs.html [`tcx`]: ../ty.html [#18290]: https://github.com/rust-lang/rust/issues/18290 [#22019]: https://github.com/rust-lang/rust/issues/22019 diff --git a/src/traits/resolution.md b/src/traits/resolution.md index 26eb7245..c62b0593 100644 --- a/src/traits/resolution.md +++ b/src/traits/resolution.md @@ -183,7 +183,7 @@ in that list. If so, it is considered satisfied. More precisely, we want to check whether there is a where-clause obligation that is for the same trait (or some subtrait) and which can match against the obligation. -[parameter environment]: ../param_env/param_env_summary.html +[parameter environment]: ../typing_parameter_envs.html Consider this simple example: diff --git a/src/typing_parameter_envs.md b/src/typing_parameter_envs.md new file mode 100644 index 00000000..757296d1 --- /dev/null +++ b/src/typing_parameter_envs.md @@ -0,0 +1,206 @@ +# Typing/Parameter Environments + + + +## Typing Environments + +When interacting with the type system there are a few variables to consider that can affect the results of trait solving. The the set of in-scope where clauses, and what phase of the compiler type system operations are being performed in (the [`ParamEnv`][penv] and [`TypingMode`][tmode] structs respectively). + +When an environment to perform type system operations in has not yet been created, the [`TypingEnv`][tenv] can be used to bundle all of the external context required into a single type. + +Once a context to perform type system operations in has been created (e.g. an [`ObligationCtxt`][ocx] or [`FnCtxt`][fnctxt]) a `TypingEnv` is typically not stored anywhere as only the `TypingMode` is a property of the whole environment, whereas different `ParamEnv`s can be used on a per-goal basis. + +[ocx]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/traits/struct.ObligationCtxt.html +[fnctxt]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_typeck/fn_ctxt/struct.FnCtxt.html + +## Parameter Environemnts + +### What is a `ParamEnv` + +The [`ParamEnv`][penv] is a list of in-scope where-clauses, it typically corresponds to a specific item's where clauses. Some clauses are not explicitly written but are instead are implicitly added in the [`predicates_of`][predicates_of] query, such as `ConstArgHasType` or (some) implied bounds. + +In most cases `ParamEnv`s are initially created via the [`param_env` query][query] which returns a `ParamEnv` derived from the provided item's where clauses. A `ParamEnv` can also be created with arbitrary sets of clauses that are not derived from a specific item, such as in [`compare_method_predicate_entailment`][method_pred_entailment] where we create a hybrid `ParamEnv` consisting of the impl's where clauses and the trait definition's function's where clauses. + +--- + +If we have a function such as: +```rust +// `foo` would have a `ParamEnv` of: +// `[T: Sized, T: Trait, ::Assoc: Clone]` +fn foo() +where + ::Assoc: Clone, +{} +``` +If we were conceptually inside of `foo` (for example, type-checking or linting it) we would use this `ParamEnv` everywhere that we interact with the type system. This would allow things such as normalization (TODO: write a chapter about normalization and link it), evaluating generic constants, and proving where clauses/goals, to rely on `T` being sized, implementing `Trait`, etc. + +A more concrete example: +```rust +// `foo` would have a `ParamEnv` of: +// `[T: Sized, T: Clone]` +fn foo(a: T) { + // when typechecking `foo` we require all the where clauses on `requires_clone` + // to hold in order for it to be legal to call. This means we have to + // prove `T: Clone`. As we are type checking `foo` we use `foo`'s + // environment when trying to check that `T: Clone` holds. + // + // Trying to prove `T: Clone` with a `ParamEnv` of `[T: Sized, T: Clone]` + // will trivially succeed as bound we want to prove is in our environment. + requires_clone(a); +} +``` + +Or alternatively an example that would not compile: +```rust +// `foo2` would have a `ParamEnv` of: +// `[T: Sized]` +fn foo2(a: T) { + // When typechecking `foo2` we attempt to prove `T: Clone`. + // As we are type checking `foo2` we use `foo2`'s environment + // when trying to prove `T: Clone`. + // + // Trying to prove `T: Clone` with a `ParamEnv` of `[T: Sized]` will + // fail as there is nothing in the environment telling the trait solver + // that `T` implements `Clone` and there exists no user written impl + // that could apply. + requires_clone(a); +} +``` + +[predicates_of]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/collect/predicates_of/fn.predicates_of.html +[method_pred_entailment]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/compare_impl_item/fn.compare_method_predicate_entailment.html +[query]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.param_env + +### Acquiring a `ParamEnv` + +Using the wrong [`ParamEnv`][penv] when interacting with the type system can lead to ICEs, illformed programs compiling, or erroing when we shouldn't. See [#82159](https://github.com/rust-lang/rust/pull/82159) and [#82067](https://github.com/rust-lang/rust/pull/82067) as examples of PRs that modified the compiler to use the correct param env and in the process fixed ICEs. + +In the large majority of cases, when a `ParamEnv` is required it either already exists somewhere in scope, or above in the call stack and should be passed down. A non exhaustive list of places where you might find an existing `ParamEnv`: +- During typeck `FnCtxt` has a [`param_env` field][fnctxt_param_env] +- When writing late lints the `LateContext` has a [`param_env` field][latectxt_param_env] +- During well formedness checking the `WfCheckingCtxt` has a [`param_env` field][wfckctxt_param_env] +- The `TypeChecker` used for MIR Typeck has a [`param_env` field][mirtypeck_param_env] +- In the next-gen trait solver all `Goal`s have a [`param_env` field][goal_param_env] specifying what environment to prove the goal in +- When editing an existing [`TypeRelation`][typerelation] if it implements [`PredicateEmittingRelation`][predicate_emitting_relation] then a [`param_env` method][typerelation_param_env] will be available. + +If you aren't sure if there's a `ParamEnv` in scope somewhere that can be used it can be worth opening a thread in the [`#t-compiler/help`][compiler_help] zulip stream where someone may be able to point out where a `ParamEnv` can be acquired from. + +Manually constructing a `ParamEnv` is typically only needed at the start of some kind of top level analysis (e.g. hir typeck or borrow checking). In such cases there are three ways it can be done: +- Calling the [`tcx.param_env(def_id)` query][param_env_query] which returns the environment associated with a given definition. +- Creating an empty environment with [`ParamEnv::empty`][env_empty]. +- Using [`ParamEnv::new`][param_env_new] to construct an env with an arbitrary set of where clauses. Then calling [`traits::normalize_param_env_or_error`][normalize_env_or_error] to handle normalizing and elaborating all the where clauses in the env. + +Using the `param_env` query is by far the most common way to construct a `ParamEnv` as most of the time the compiler is performing an analysis as part of some specific definition. + +Creating an empty environment with `ParamEnv::empty` is typically only done either in codegen (indirectly via [`TypingEnv::fully_monomorphized`][tenv_mono]), or as part of some analysis that do not expect to ever encounter generic parameters (e.g. various parts of coherence/orphan check). + +Creating an env from an arbitrary set of where clauses is usually unnecessary and should only be done if the environment you need does not correspond to an actual item in the source code (e.g. [`compare_method_predicate_entailment`][method_pred_entailment]). + +[param_env_new]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.new +[normalize_env_or_error]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/traits/fn.normalize_param_env_or_error.html +[fnctxt_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_typeck/fn_ctxt/struct.FnCtxt.html#structfield.param_env +[latectxt_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/context/struct.LateContext.html#structfield.param_env +[wfckctxt_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/wfcheck/struct.WfCheckingCtxt.html#structfield.param_env +[goal_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/infer/canonical/ir/solve/struct.Goal.html#structfield.param_env +[typerelation_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/infer/trait.PredicateEmittingRelation.html#tymethod.param_env +[typerelation]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/relate/trait.TypeRelation.html +[mirtypeck_param_env]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_borrowck/type_check/struct.TypeChecker.html#structfield.param_env +[env_empty]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html#method.empty +[param_env_query]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_typeck/fn_ctxt/struct.FnCtxt.html#structfield.param_env +[method_pred_entailment]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/compare_impl_item/fn.compare_method_predicate_entailment.html +[predicate_emitting_relation]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/relate/combine/trait.PredicateEmittingRelation.html +[tenv_mono]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TypingEnv.html#method.fully_monomorphized +[compiler_help]: https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp + +### How are `ParamEnv`s constructed + +Creating a [`ParamEnv`][pe] is more complicated than simply using the list of where clauses defined on an item as written by the user. We need to both elaborate supertraits into the env and fully normalize all aliases. This logic is handled by [`traits::normalize_param_env_or_error`][normalize_env_or_error] (even though it does not mention anything about elaboration). + +#### Elaborating supertraits + +When we have a function such as `fn foo()` we would like to be able to prove `T: Clone` inside of the function as the `Copy` trait has a `Clone` supertrait. Constructing a `ParamEnv` looks at all of the trait bounds in the env and explicitly adds new where clauses to the `ParamEnv` for any supertraits found on the traits. + +A concrete example would be the following function: +```rust +trait Trait: SuperTrait {} +trait SuperTrait: SuperSuperTrait {} + +// `bar`'s unelaborated `ParamEnv` would be: +// `[T: Sized, T: Copy, T: Trait]` +fn bar(a: T) { + requires_impl(a); +} + +fn requires_impl(a: T) {} +``` + +If we did not elaborate the env then the `requires_impl` call would fail to typecheck as we would not be able to prove `T: Clone` or `T: SuperSuperTrait`. In practice we elaborate the env which means that `bar`'s `ParamEnv` is actually: +`[T: Sized, T: Copy, T: Clone, T: Trait, T: SuperTrait, T: SuperSuperTrait]` +This allows us to prove `T: Clone` and `T: SuperSuperTrait` when type checking `bar`. + +The `Clone` trait has a `Sized` supertrait however we do not end up with two `T: Sized` bounds in the env (one for the supertrait and one for the implicitly added `T: Sized` bound) as the elaboration process (implemented via [`util::elaborate`][elaborate]) deduplicates where clauses. + +A side effect of this is that even if no actual elaboration of supertraits takes place, the existing where clauses in the env are _also_ deduplicated. See the following example: +```rust +trait Trait {} +// The unelaborated `ParamEnv` would be: +// `[T: Sized, T: Trait, T: Trait]` +// but after elaboration it would be: +// `[T: Sized, T: Trait]` +fn foo() {} +``` + +The [next-gen trait solver][next-gen-solver] also requires this elaboration to take place. + +[elaborate]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_infer/traits/util/fn.elaborate.html +[next-gen-solver]: ./solve/trait-solving.md + +#### Normalizing all bounds + +In the old trait solver the where clauses stored in `ParamEnv` are required to be fully normalized as otherwise the trait solver will not function correctly. A concrete example of needing to normalize the `ParamEnv` is the following: +```rust +trait Trait { + type Assoc; +} + +trait Other { + type Bar; +} + +impl Other for T { + type Bar = u32; +} + +// `foo`'s unnormalized `ParamEnv` would be: +// `[T: Sized, U: Sized, U: Trait]` +fn foo(a: U) +where + U: Trait<::Bar>, +{ + requires_impl(a); +} + +fn requires_impl>(_: U) {} +``` + +As humans we can tell that `::Bar` is equal to `u32` so the trait bound on `U` is equivalent to `U: Trait`. In practice trying to prove `U: Trait` in the old solver in this environment would fail as it is unable to determine that `::Bar` is equal to `u32`. + +To work around this we normalize `ParamEnv`'s after constructing them so that `foo`'s `ParamEnv` is actually: `[T: Sized, U: Sized, U: Trait]` which means the trait solver is now able to use the `U: Trait` in the `ParamEnv` to determine that the trait bound `U: Trait` holds. + +This workaround does not work in all cases as normalizing associated types requires a `ParamEnv` which introduces a bootstrapping problem. We need a normalized `ParamEnv` in order for normalization to give correct results, but we need to normalize to get that `ParamEnv`. Currently we normalize the `ParamEnv` once using the unnormalized param env and it tends to give okay results in practice even though there are some examples where this breaks ([example]). + +In the next-gen trait solver the requirement for all where clauses in the `ParamEnv` to be fully normalized is not present and so we do not normalize when constructing `ParamEnv`s. + +[example]: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=e6933265ea3e84eaa47019465739992c +[pe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html +[normalize_env_or_error]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_trait_selection/traits/fn.normalize_param_env_or_error.html + +## Typing Modes + +Depending on what context we are performing type system operations in, different behaviour may be required. For example during coherence there are stronger requirements about when we can consider goals to not hold or when we can consider types to be unequal. + +Tracking which "phase" of the compiler type system operations are being performed in is done by the [`TypingMode`][tenv] enum. The documentation on the `TypingMode` enum is quite good so instead of repeating it here verbatim we would recommend reading the API documentation directly. + +[penv]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.ParamEnv.html +[tenv]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_type_ir/infer_ctxt/enum.TypingMode.html +[tmode]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/type.TypingMode.html From 8c3302aa5f2bffecb6d983ea5afe652b38ab036b Mon Sep 17 00:00:00 2001 From: Jieyou Xu Date: Fri, 21 Mar 2025 16:18:43 +0800 Subject: [PATCH 049/107] Move Fuchsia and RfL under `ecosystem-test-jobs/` folder Includes redirects to avoid breaking existing links. --- book.toml | 4 +++- src/SUMMARY.md | 4 ++-- src/notification-groups/fuchsia.md | 2 +- src/tests/{ => ecosystem-test-jobs}/fuchsia.md | 4 ++-- src/tests/{ => ecosystem-test-jobs}/rust-for-linux.md | 2 +- src/tests/ecosystem.md | 4 ++-- 6 files changed, 11 insertions(+), 9 deletions(-) rename src/tests/{ => ecosystem-test-jobs}/fuchsia.md (98%) rename src/tests/{ => ecosystem-test-jobs}/rust-for-linux.md (97%) diff --git a/book.toml b/book.toml index 67069d99..eb2f6806 100644 --- a/book.toml +++ b/book.toml @@ -62,5 +62,7 @@ warning-policy = "error" "/diagnostics/sessiondiagnostic.html" = "diagnostic-structs.html" "/diagnostics/diagnostic-codes.html" = "error-codes.html" "/miri.html" = "const-eval/interpret.html" -"/tests/integration.html" = "ecosystem.html" +"/tests/fuchsia.html" = "ecosystem-test-jobs/fuchsia.html" "/tests/headers.html" = "directives.html" +"/tests/integration.html" = "ecosystem.html" +"/tests/rust-for-linux.html" = "ecosystem-test-jobs/rust-for-linux.html" diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 075d5af4..8fc4fd33 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -28,8 +28,8 @@ - [Minicore](./tests/minicore.md) - [Ecosystem testing](./tests/ecosystem.md) - [Crater](./tests/crater.md) - - [Fuchsia](./tests/fuchsia.md) - - [Rust for Linux](./tests/rust-for-linux.md) + - [Fuchsia](./tests/ecosystem-test-jobs/fuchsia.md) + - [Rust for Linux](./tests/ecosystem-test-jobs/rust-for-linux.md) - [Performance testing](./tests/perf.md) - [Suggest tests tool](./tests/suggest-tests.md) - [Misc info](./tests/misc.md) diff --git a/src/notification-groups/fuchsia.md b/src/notification-groups/fuchsia.md index a4658e0d..e3c1a714 100644 --- a/src/notification-groups/fuchsia.md +++ b/src/notification-groups/fuchsia.md @@ -9,4 +9,4 @@ This list will be used to notify [Fuchsia][fuchsia] maintainers when the compiler or the standard library changes in a way that would break the Fuchsia integration. -[fuchsia]: ../tests/fuchsia.md +[fuchsia]: ../tests/ecosystem-test-jobs/fuchsia.md diff --git a/src/tests/fuchsia.md b/src/tests/ecosystem-test-jobs/fuchsia.md similarity index 98% rename from src/tests/fuchsia.md rename to src/tests/ecosystem-test-jobs/fuchsia.md index 2766c362..b19d94d6 100644 --- a/src/tests/fuchsia.md +++ b/src/tests/ecosystem-test-jobs/fuchsia.md @@ -40,7 +40,7 @@ using your local Rust toolchain. src/ci/docker/run.sh x86_64-fuchsia ``` -See the [Testing with Docker](docker.md) chapter for more details on how to run +See the [Testing with Docker](../docker.md) chapter for more details on how to run and debug jobs with Docker. Note that a Fuchsia checkout is *large* – as of this writing, a checkout and @@ -170,7 +170,7 @@ rustc book][platform-support]. [`public_configs`]: https://gn.googlesource.com/gn/+/main/docs/reference.md#var_public_configs [`//build/config:compiler`]: https://cs.opensource.google/fuchsia/fuchsia/+/main:build/config/BUILD.gn;l=121;drc=c26c473bef93b33117ae417893118907a026fec7 [build system]: https://fuchsia.dev/fuchsia-src/development/build/build_system -[fuchsia-ping]: ../notification-groups/fuchsia.md +[fuchsia-ping]: ../../notification-groups/fuchsia.md [^loc]: As of June 2024, Fuchsia had about 2 million lines of first-party Rust code and a roughly equal amount of third-party code, as counted by tokei diff --git a/src/tests/rust-for-linux.md b/src/tests/ecosystem-test-jobs/rust-for-linux.md similarity index 97% rename from src/tests/rust-for-linux.md rename to src/tests/ecosystem-test-jobs/rust-for-linux.md index bdf32ffc..d549ec6f 100644 --- a/src/tests/rust-for-linux.md +++ b/src/tests/ecosystem-test-jobs/rust-for-linux.md @@ -48,4 +48,4 @@ line to your PR description: Then when you `@bors try` it will pick the job that builds the Rust for Linux integration. -[rfl-ping]: ../notification-groups/rust-for-linux.md +[rfl-ping]: ../../notification-groups/rust-for-linux.md diff --git a/src/tests/ecosystem.md b/src/tests/ecosystem.md index 08360140..f4b93492 100644 --- a/src/tests/ecosystem.md +++ b/src/tests/ecosystem.md @@ -24,5 +24,5 @@ there aren't any significant regressions. We have CI jobs that build large open-source Rust projects that are used as regression tests in CI. Our integration jobs build the following projects: -- [Fuchsia](fuchsia.md) -- [Rust for Linux](rust-for-linux.md) +- [Fuchsia](./ecosystem-test-jobs/fuchsia.md) +- [Rust for Linux](./ecosystem-test-jobs/rust-for-linux.md) From b7fc809e0bf0fd1942a8c59eab4f5a29950b62c7 Mon Sep 17 00:00:00 2001 From: Jieyou Xu Date: Fri, 21 Mar 2025 16:35:46 +0800 Subject: [PATCH 050/107] Stub out codegen backend test pages --- src/SUMMARY.md | 3 +++ src/tests/codegen-backend-tests/cg_clif.md | 3 +++ src/tests/codegen-backend-tests/cg_gcc.md | 3 +++ src/tests/codegen-backend-tests/intro.md | 13 +++++++++++++ src/tests/intro.md | 8 ++++++-- 5 files changed, 28 insertions(+), 2 deletions(-) create mode 100644 src/tests/codegen-backend-tests/cg_clif.md create mode 100644 src/tests/codegen-backend-tests/cg_gcc.md create mode 100644 src/tests/codegen-backend-tests/intro.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 8fc4fd33..29085a6e 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -30,6 +30,9 @@ - [Crater](./tests/crater.md) - [Fuchsia](./tests/ecosystem-test-jobs/fuchsia.md) - [Rust for Linux](./tests/ecosystem-test-jobs/rust-for-linux.md) + - [Codegen backend testing](./tests/codegen-backend-tests/intro.md) + - [Cranelift codegen backend](./tests/codegen-backend-tests/cg_clif.md) + - [GCC codegen backend](./tests/codegen-backend-tests/cg_gcc.md) - [Performance testing](./tests/perf.md) - [Suggest tests tool](./tests/suggest-tests.md) - [Misc info](./tests/misc.md) diff --git a/src/tests/codegen-backend-tests/cg_clif.md b/src/tests/codegen-backend-tests/cg_clif.md new file mode 100644 index 00000000..030ddd7d --- /dev/null +++ b/src/tests/codegen-backend-tests/cg_clif.md @@ -0,0 +1,3 @@ +# Cranelift codegen backend tests + +TODO: please add some more information to this page. diff --git a/src/tests/codegen-backend-tests/cg_gcc.md b/src/tests/codegen-backend-tests/cg_gcc.md new file mode 100644 index 00000000..4caf4c0e --- /dev/null +++ b/src/tests/codegen-backend-tests/cg_gcc.md @@ -0,0 +1,3 @@ +# GCC codegen backend tests + +TODO: please add some more information to this page. diff --git a/src/tests/codegen-backend-tests/intro.md b/src/tests/codegen-backend-tests/intro.md new file mode 100644 index 00000000..c4bac26a --- /dev/null +++ b/src/tests/codegen-backend-tests/intro.md @@ -0,0 +1,13 @@ +# Codegen backend testing + +See also the [Code generation](../../../src/backend/codegen.md) chapter. + +In addition to the primary LLVM codegen backend, the rust-lang/rust CI also runs tests of the [cranelift][cg_clif] and [GCC][cg_gcc] codegen backends in certain test jobs. + +For more details on the tests involved, see: + +- [Cranelift codegen backend tests](./cg_clif.md) +- [GCC codegen backend tests](./cg_gcc.md) + +[cg_clif]: https://github.com/rust-lang/rustc_codegen_cranelift +[cg_gcc]: https://github.com/rust-lang/rustc_codegen_gcc diff --git a/src/tests/intro.md b/src/tests/intro.md index ba44a969..7bf30b10 100644 --- a/src/tests/intro.md +++ b/src/tests/intro.md @@ -38,7 +38,7 @@ directory, and `x` will essentially run `cargo test` on that package. Examples: | Command | Description | -| ----------------------------------------- | ------------------------------------- | +|-------------------------------------------|---------------------------------------| | `./x test library/std` | Runs tests on `std` only | | `./x test library/core` | Runs tests on `core` only | | `./x test compiler/rustc_data_structures` | Runs tests on `rustc_data_structures` | @@ -86,7 +86,7 @@ above. Examples: | Command | Description | -| ----------------------- | ------------------------------------------------------------------ | +|-------------------------|--------------------------------------------------------------------| | `./x fmt --check` | Checks formatting and exits with an error if formatting is needed. | | `./x fmt` | Runs rustfmt across the entire codebase. | | `./x test tidy --bless` | First runs rustfmt to format the codebase, then runs tidy checks. | @@ -155,6 +155,10 @@ chapter](ecosystem.md) for more details. A separate infrastructure is used for testing and tracking performance of the compiler. See the [Performance testing chapter](perf.md) for more details. +### Codegen backend testing + +See [Codegen backend testing](./codegen-backend-tests/intro.md). + ## Miscellaneous information There are some other useful testing-related info at [Misc info](misc.md). From 6f9680da7a529d63167be8876bd5f431912f7f32 Mon Sep 17 00:00:00 2001 From: Chiichen Date: Sun, 23 Mar 2025 12:34:41 +0800 Subject: [PATCH 051/107] doc: fix reference to #create-a-configtoml --- src/building/prerequisites.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/building/prerequisites.md b/src/building/prerequisites.md index f49f6bb0..6761caba 100644 --- a/src/building/prerequisites.md +++ b/src/building/prerequisites.md @@ -38,4 +38,4 @@ incremental compilation ([see here][config]). This will make compilation take longer (especially after a rebase), but will save a ton of space from the incremental caches. -[config]: ./how-to-build-and-run.md#create-a-configtoml +[config]: ./how-to-build-and-run.md#create-a-bootstraptoml From 07e905fbe6e9eb4d5a9b6c6ea227a4f9cd87ffb3 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 24 Mar 2025 10:40:03 +0200 Subject: [PATCH 052/107] add needed break --- src/rustdoc-internals/rustdoc-test-suite.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/rustdoc-internals/rustdoc-test-suite.md b/src/rustdoc-internals/rustdoc-test-suite.md index c2a2e649..471dd29f 100644 --- a/src/rustdoc-internals/rustdoc-test-suite.md +++ b/src/rustdoc-internals/rustdoc-test-suite.md @@ -1,6 +1,7 @@ # The `rustdoc` test suite -This page is specifically about the test suite named `rustdoc`, for other test suites used for testing rustdoc, see [Rustdoc tests](../rustdoc.md#tests). +This page is specifically about the test suite named `rustdoc`. +For other test suites used for testing rustdoc, see [Rustdoc tests](../rustdoc.md#tests). The `rustdoc` test suite is specifically used to test the HTML output of rustdoc. From 55e47b61f5bf197654074c81e166cb515dbfd90a Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 24 Mar 2025 10:41:19 +0200 Subject: [PATCH 053/107] typo --- src/rustdoc-internals/rustdoc-test-suite.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/rustdoc-internals/rustdoc-test-suite.md b/src/rustdoc-internals/rustdoc-test-suite.md index 471dd29f..169b95a7 100644 --- a/src/rustdoc-internals/rustdoc-test-suite.md +++ b/src/rustdoc-internals/rustdoc-test-suite.md @@ -5,7 +5,7 @@ For other test suites used for testing rustdoc, see [Rustdoc tests](../rustdoc.m The `rustdoc` test suite is specifically used to test the HTML output of rustdoc. -This is achived by means of `htmldocck.py`, a custom checker script that leverages [XPath]. +This is achieved by means of `htmldocck.py`, a custom checker script that leverages [XPath]. [XPath]: https://en.wikipedia.org/wiki/XPath From 2e81955d9cf6660585d9567792f79d9214c1d293 Mon Sep 17 00:00:00 2001 From: Vadim Petrochenkov Date: Sun, 23 Mar 2025 15:50:51 +0300 Subject: [PATCH 054/107] compiletest: Support matching on diagnostics without a span --- src/tests/ui.md | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/src/tests/ui.md b/src/tests/ui.md index c8536b00..98bb9dee 100644 --- a/src/tests/ui.md +++ b/src/tests/ui.md @@ -202,6 +202,9 @@ several ways to match the message with the line (see the examples below): * `~|`: Associates the error level and message with the *same* line as the *previous comment*. This is more convenient than using multiple carets when there are multiple messages associated with the same line. +* `~?`: Used to match error levels and messages with errors not having line + information. These can be placed on any line in the test file, but are + conventionally placed at the end. Example: @@ -270,10 +273,23 @@ fn main() { //~| ERROR this pattern has 1 field, but the corresponding tuple struct has 3 fields [E0023] ``` +#### Error without line information + +Use `//~?` to match an error without line information. +`//~?` is precise and will not match errors if their line information is available. +It should be preferred to using `error-pattern`, which is imprecise and non-exhaustive. + +```rust,ignore +//@ compile-flags: --print yyyy + +//~? ERROR unknown print request: `yyyy` +``` + ### `error-pattern` -The `error-pattern` [directive](directives.md) can be used for messages that don't -have a specific span. +The `error-pattern` [directive](directives.md) can be used for runtime messages, which don't +have a specific span, or for compile time messages if imprecise matching is required due to +multi-line platform specific diagnostics. Let's think about this test: @@ -300,7 +316,9 @@ fn main() { } ``` -But for strict testing, try to use the `ERROR` annotation as much as possible. +But for strict testing, try to use the `ERROR` annotation as much as possible, +including `//~?` annotations for diagnostics without span. +For compile time diagnostics `error-pattern` should very rarely be necessary. ### Error levels @@ -353,7 +371,7 @@ would be a `.mir.stderr` and `.thir.stderr` file with the different outputs of the different revisions. > Note: cfg revisions also work inside the source code with `#[cfg]` attributes. -> +> > By convention, the `FALSE` cfg is used to have an always-false config. ## Controlling pass/fail expectations From 9745529523f09a7ad534b9524f9fd728f61c7fc9 Mon Sep 17 00:00:00 2001 From: mejrs <59372212+mejrs@users.noreply.github.com> Date: Thu, 27 Mar 2025 18:32:48 +0100 Subject: [PATCH 055/107] Delete from_method from rustc_on_unimplemented documentation --- src/diagnostics.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/src/diagnostics.md b/src/diagnostics.md index 972309b5..6f72ea90 100644 --- a/src/diagnostics.md +++ b/src/diagnostics.md @@ -954,9 +954,6 @@ application of these fields based on a variety of attributes when using `Self="std::iter::Iterator"`. This is needed because `Self` is a keyword which cannot appear in attributes. - `direct`: user-specified rather than derived obligation. - - `from_method`: usable both as boolean (whether the flag is present, like - `crate_local`) or matching against a particular method. Currently used - for `try`. - `from_desugaring`: usable both as boolean (whether the flag is present) or matching against a particular desugaring. The desugaring is identified with its variant name in the `DesugaringKind` enum. From a480687c6c430a5249b6294ab99f35b6782352ac Mon Sep 17 00:00:00 2001 From: Santiago Pastorino Date: Fri, 28 Mar 2025 12:38:32 -0300 Subject: [PATCH 056/107] Fix code generation link --- src/tests/codegen-backend-tests/intro.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tests/codegen-backend-tests/intro.md b/src/tests/codegen-backend-tests/intro.md index c4bac26a..6bf46ddc 100644 --- a/src/tests/codegen-backend-tests/intro.md +++ b/src/tests/codegen-backend-tests/intro.md @@ -1,6 +1,6 @@ # Codegen backend testing -See also the [Code generation](../../../src/backend/codegen.md) chapter. +See also the [Code generation](../../backend/codegen.md) chapter. In addition to the primary LLVM codegen backend, the rust-lang/rust CI also runs tests of the [cranelift][cg_clif] and [GCC][cg_gcc] codegen backends in certain test jobs. From f1a4e592db7edfe665ec66d5aec1c59d3cbd6d02 Mon Sep 17 00:00:00 2001 From: Samuel Tardieu Date: Fri, 28 Mar 2025 20:10:31 +0100 Subject: [PATCH 057/107] Fix trivial typo of `BoundVariableKind` --- src/ty_module/binders.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ty_module/binders.md b/src/ty_module/binders.md index defb7cde..71157eca 100644 --- a/src/ty_module/binders.md +++ b/src/ty_module/binders.md @@ -40,7 +40,7 @@ We did not always explicitly track the set of bound vars introduced by each `Bin ``` Binder( fn(&'^1_0 &'^1 T/#0), - &[BoundVariarbleKind::Region(...)], + &[BoundVariableKind::Region(...)], ) ``` This would cause all kinds of issues as the region `'^1_0` refers to a binder at a higher level than the outermost binder i.e. it is an escaping bound var. The `'^1` region (also writeable as `'^0_1`) is also ill formed as the binder it refers to does not introduce a second parameter. Modern day rustc will ICE when constructing this binder due to both of those regions, in the past we would have simply allowed this to work and then ran into issues in other parts of the codebase. From 628eb7064d7aaea62521e11d6cc6b7306f51c248 Mon Sep 17 00:00:00 2001 From: Vadim Petrochenkov Date: Sat, 29 Mar 2025 12:52:10 +0300 Subject: [PATCH 058/107] compiletest: Support matching diagnostics on lines below --- src/tests/ui.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/src/tests/ui.md b/src/tests/ui.md index 98bb9dee..1190c264 100644 --- a/src/tests/ui.md +++ b/src/tests/ui.md @@ -202,6 +202,9 @@ several ways to match the message with the line (see the examples below): * `~|`: Associates the error level and message with the *same* line as the *previous comment*. This is more convenient than using multiple carets when there are multiple messages associated with the same line. +* `~v`: Associates the error level and message with the *next* error + annotation line. Each symbol (`v`) that you add adds a line to this, so `~vvv` + is three lines below the error annotation line. * `~?`: Used to match error levels and messages with errors not having line information. These can be placed on any line in the test file, but are conventionally placed at the end. @@ -273,6 +276,18 @@ fn main() { //~| ERROR this pattern has 1 field, but the corresponding tuple struct has 3 fields [E0023] ``` +#### Positioned above error line + +Use the `//~v` idiom with number of v's in the string to indicate the number +of lines below. This is typically used in lexer or parser tests matching on errors like unclosed +delimiter or unclosed literal happening at the end of file. + +```rust,ignore +// ignore-tidy-trailing-newlines +//~v ERROR this file contains an unclosed delimiter +fn main((ؼ +``` + #### Error without line information Use `//~?` to match an error without line information. From 6c1077e05d5409dd25cd01daa106da20667fc76c Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 29 Mar 2025 20:02:01 +0200 Subject: [PATCH 059/107] mention that know-bug test directive takes arguments --- src/tests/ui.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/tests/ui.md b/src/tests/ui.md index c8536b00..728ea3de 100644 --- a/src/tests/ui.md +++ b/src/tests/ui.md @@ -415,6 +415,14 @@ reasons, including: can alert the developer so they know that the associated issue has been fixed and can possibly be closed. +This directive takes comma-separated issue numbers as arguments, or `"unknown"`: + +- `//@ known-bug: #123, #456` (when the issues are on rust-lang/rust) +- `//@ known-bug: rust-lang/chalk#123456` + (allows arbitrary text before the `#`, which is useful when the issue is on another repo) +- `//@ known-bug: unknown` + (when there is no known issue yet; preferrably open one if it does not already exist) + Do not include [error annotations](#error-annotations) in a test with `known-bug`. The test should still include other normal directives and stdout/stderr files. From 550dc9b31b901539ed2a521669ced52b31508544 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 29 Mar 2025 23:52:57 +0200 Subject: [PATCH 060/107] update rustc-{driver,interface} examples --- examples/rustc-driver-example.rs | 11 +++++------ examples/rustc-driver-interacting-with-the-ast.rs | 4 ++-- examples/rustc-interface-example.rs | 8 ++++---- examples/rustc-interface-getting-diagnostics.rs | 2 +- 4 files changed, 12 insertions(+), 13 deletions(-) diff --git a/examples/rustc-driver-example.rs b/examples/rustc-driver-example.rs index 35dd07dd..fb8b13ad 100644 --- a/examples/rustc-driver-example.rs +++ b/examples/rustc-driver-example.rs @@ -1,4 +1,4 @@ -// Tested with nightly-2025-03-08 +// Tested with nightly-2025-03-28 #![feature(rustc_private)] @@ -20,7 +20,7 @@ use std::path::Path; use std::sync::Arc; use rustc_ast_pretty::pprust::item_to_string; -use rustc_driver::{Compilation, run_compiler}; +use rustc_driver::{run_compiler, Compilation}; use rustc_interface::interface::{Compiler, Config}; use rustc_middle::ty::TyCtxt; @@ -71,13 +71,12 @@ impl rustc_driver::Callbacks for MyCallbacks { fn after_analysis(&mut self, _compiler: &Compiler, tcx: TyCtxt<'_>) -> Compilation { // Analyze the program and inspect the types of definitions. - for id in tcx.hir_free_items(){ + for id in tcx.hir_free_items() { let item = &tcx.hir_item(id); match item.kind { - rustc_hir::ItemKind::Static(_, _, _) | rustc_hir::ItemKind::Fn { .. } => { - let name = item.ident; + rustc_hir::ItemKind::Static(ident, ..) | rustc_hir::ItemKind::Fn { ident, .. } => { let ty = tcx.type_of(item.hir_id().owner.def_id); - println!("{name:?}:\t{ty:?}") + println!("{ident:?}:\t{ty:?}") } _ => (), } diff --git a/examples/rustc-driver-interacting-with-the-ast.rs b/examples/rustc-driver-interacting-with-the-ast.rs index 2a43ba47..b0c62d5b 100644 --- a/examples/rustc-driver-interacting-with-the-ast.rs +++ b/examples/rustc-driver-interacting-with-the-ast.rs @@ -1,4 +1,4 @@ -// Tested with nightly-2025-03-08 +// Tested with nightly-2025-03-28 #![feature(rustc_private)] @@ -71,7 +71,7 @@ impl rustc_driver::Callbacks for MyCallbacks { fn after_analysis(&mut self, _compiler: &Compiler, tcx: TyCtxt<'_>) -> Compilation { // Iterate over the top-level items in the crate, looking for the main function. - for id in tcx.hir_free_items(){ + for id in tcx.hir_free_items() { let item = &tcx.hir_item(id); // Use pattern-matching to find a specific node inside the main function. if let rustc_hir::ItemKind::Fn { body, .. } = item.kind { diff --git a/examples/rustc-interface-example.rs b/examples/rustc-interface-example.rs index 11125cae..360f70c8 100644 --- a/examples/rustc-interface-example.rs +++ b/examples/rustc-interface-example.rs @@ -1,4 +1,4 @@ -// Tested with nightly-2025-03-08 +// Tested with nightly-2025-03-28 #![feature(rustc_private)] @@ -67,10 +67,10 @@ fn main() { for id in tcx.hir_free_items() { let item = tcx.hir_item(id); match item.kind { - rustc_hir::ItemKind::Static(_, _, _) | rustc_hir::ItemKind::Fn { .. } => { - let name = item.ident; + rustc_hir::ItemKind::Static(ident, ..) + | rustc_hir::ItemKind::Fn { ident, .. } => { let ty = tcx.type_of(item.hir_id().owner.def_id); - println!("{name:?}:\t{ty:?}") + println!("{ident:?}:\t{ty:?}") } _ => (), } diff --git a/examples/rustc-interface-getting-diagnostics.rs b/examples/rustc-interface-getting-diagnostics.rs index 9bb93ab4..2512ba3c 100644 --- a/examples/rustc-interface-getting-diagnostics.rs +++ b/examples/rustc-interface-getting-diagnostics.rs @@ -1,4 +1,4 @@ -// Tested with nightly-2025-03-08 +// Tested with nightly-2025-03-28 #![feature(rustc_private)] From eb3d07befcfe964291baaf7cb3fee3cd040fcbc4 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sun, 30 Mar 2025 00:16:39 +0200 Subject: [PATCH 061/107] example assumes a static exists This was removed, likely by mistake, during a refactor. --- examples/rustc-driver-example.rs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/examples/rustc-driver-example.rs b/examples/rustc-driver-example.rs index fb8b13ad..0e345008 100644 --- a/examples/rustc-driver-example.rs +++ b/examples/rustc-driver-example.rs @@ -34,9 +34,9 @@ impl rustc_span::source_map::FileLoader for MyFileLoader { fn read_file(&self, path: &Path) -> io::Result { if path == Path::new("main.rs") { Ok(r#" +static MESSAGE: &str = "Hello, World!"; fn main() { - let message = "Hello, World!"; - println!("{message}"); + println!("{MESSAGE}"); } "# .to_string()) From ed9122f5343f64a192c2baffb0655f2599e79f3e Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sun, 30 Mar 2025 00:31:44 +0200 Subject: [PATCH 062/107] add rustfmt settings file --- ci/date-check/src/main.rs | 155 ++++-------------- examples/rustc-driver-example.rs | 2 +- .../rustc-driver-interacting-with-the-ast.rs | 2 +- rustfmt.toml | 7 + 4 files changed, 39 insertions(+), 127 deletions(-) create mode 100644 rustfmt.toml diff --git a/ci/date-check/src/main.rs b/ci/date-check/src/main.rs index 5ab3e6c8..9af69dbb 100644 --- a/ci/date-check/src/main.rs +++ b/ci/date-check/src/main.rs @@ -1,11 +1,8 @@ -use std::{ - collections::BTreeMap, - convert::TryInto as _, - env, fmt, fs, - path::{Path, PathBuf}, - process, - str::FromStr, -}; +use std::collections::BTreeMap; +use std::convert::TryInto as _; +use std::path::{Path, PathBuf}; +use std::str::FromStr; +use std::{env, fmt, fs, process}; use chrono::{Datelike as _, Month, TimeZone as _, Utc}; use glob::glob; @@ -19,19 +16,13 @@ struct Date { impl Date { fn months_since(self, other: Date) -> Option { - let self_chrono = Utc - .with_ymd_and_hms(self.year.try_into().unwrap(), self.month, 1, 0, 0, 0) - .unwrap(); - let other_chrono = Utc - .with_ymd_and_hms(other.year.try_into().unwrap(), other.month, 1, 0, 0, 0) - .unwrap(); + let self_chrono = + Utc.with_ymd_and_hms(self.year.try_into().unwrap(), self.month, 1, 0, 0, 0).unwrap(); + let other_chrono = + Utc.with_ymd_and_hms(other.year.try_into().unwrap(), other.month, 1, 0, 0, 0).unwrap(); let duration_since = self_chrono.signed_duration_since(other_chrono); let months_since = duration_since.num_days() / 30; - if months_since < 0 { - None - } else { - Some(months_since.try_into().unwrap()) - } + if months_since < 0 { None } else { Some(months_since.try_into().unwrap()) } } } @@ -66,26 +57,18 @@ fn collect_dates_from_file(date_regex: &Regex, text: &str) -> Vec<(usize, Date)> date_regex .captures_iter(text) .filter_map(|cap| { - if let (Some(month), Some(year), None, None) | (None, None, Some(month), Some(year)) = ( - cap.name("m1"), - cap.name("y1"), - cap.name("m2"), - cap.name("y2"), - ) { + if let (Some(month), Some(year), None, None) | (None, None, Some(month), Some(year)) = + (cap.name("m1"), cap.name("y1"), cap.name("m2"), cap.name("y2")) + { let year = year.as_str().parse().expect("year"); - let month = Month::from_str(month.as_str()) - .expect("month") - .number_from_month(); + let month = Month::from_str(month.as_str()).expect("month").number_from_month(); Some((cap.get(0).expect("all").range(), Date { year, month })) } else { None } }) .map(|(byte_range, date)| { - line += text[end_of_last_cap..byte_range.end] - .chars() - .filter(|c| *c == '\n') - .count(); + line += text[end_of_last_cap..byte_range.end].chars().filter(|c| *c == '\n').count(); end_of_last_cap = byte_range.end; (line, date) }) @@ -138,10 +121,7 @@ fn main() { let root_dir_path = Path::new(&root_dir); let glob_pat = format!("{}/**/*.md", root_dir); let today_chrono = Utc::now().date_naive(); - let current_month = Date { - year: today_chrono.year_ce().1, - month: today_chrono.month(), - }; + let current_month = Date { year: today_chrono.year_ce().1, month: today_chrono.month() }; let dates_by_file = collect_dates(glob(&glob_pat).unwrap().map(Result::unwrap)); let dates_by_file: BTreeMap<_, _> = @@ -173,10 +153,7 @@ fn main() { println!(); for (path, dates) in dates_by_file { - println!( - "- {}", - path.strip_prefix(&root_dir_path).unwrap_or(&path).display(), - ); + println!("- {}", path.strip_prefix(&root_dir_path).unwrap_or(&path).display(),); for (line, date) in dates { println!(" - [ ] line {}: {}", line, date); } @@ -191,14 +168,8 @@ mod tests { #[test] fn test_months_since() { - let date1 = Date { - year: 2020, - month: 3, - }; - let date2 = Date { - year: 2021, - month: 1, - }; + let date1 = Date { year: 2020, month: 3 }; + let date2 = Date { year: 2021, month: 1 }; assert_eq!(date2.months_since(date1), Some(10)); } @@ -273,83 +244,17 @@ Test8 assert_eq!( collect_dates_from_file(&make_date_regex(), text), vec![ - ( - 3, - Date { - year: 2021, - month: 1, - } - ), - ( - 6, - Date { - year: 2021, - month: 2, - } - ), - ( - 9, - Date { - year: 2021, - month: 3, - } - ), - ( - 11, - Date { - year: 2021, - month: 4, - } - ), - ( - 17, - Date { - year: 2021, - month: 5, - } - ), - ( - 20, - Date { - year: 2021, - month: 1, - } - ), - ( - 23, - Date { - year: 2021, - month: 2, - } - ), - ( - 26, - Date { - year: 2021, - month: 3, - } - ), - ( - 28, - Date { - year: 2021, - month: 4, - } - ), - ( - 34, - Date { - year: 2021, - month: 5, - } - ), - ( - 38, - Date { - year: 2021, - month: 6, - } - ), + (3, Date { year: 2021, month: 1 }), + (6, Date { year: 2021, month: 2 }), + (9, Date { year: 2021, month: 3 }), + (11, Date { year: 2021, month: 4 }), + (17, Date { year: 2021, month: 5 }), + (20, Date { year: 2021, month: 1 }), + (23, Date { year: 2021, month: 2 }), + (26, Date { year: 2021, month: 3 }), + (28, Date { year: 2021, month: 4 }), + (34, Date { year: 2021, month: 5 }), + (38, Date { year: 2021, month: 6 }), ], ); } diff --git a/examples/rustc-driver-example.rs b/examples/rustc-driver-example.rs index 0e345008..db6ac185 100644 --- a/examples/rustc-driver-example.rs +++ b/examples/rustc-driver-example.rs @@ -20,7 +20,7 @@ use std::path::Path; use std::sync::Arc; use rustc_ast_pretty::pprust::item_to_string; -use rustc_driver::{run_compiler, Compilation}; +use rustc_driver::{Compilation, run_compiler}; use rustc_interface::interface::{Compiler, Config}; use rustc_middle::ty::TyCtxt; diff --git a/examples/rustc-driver-interacting-with-the-ast.rs b/examples/rustc-driver-interacting-with-the-ast.rs index b0c62d5b..c0d7f977 100644 --- a/examples/rustc-driver-interacting-with-the-ast.rs +++ b/examples/rustc-driver-interacting-with-the-ast.rs @@ -20,7 +20,7 @@ use std::path::Path; use std::sync::Arc; use rustc_ast_pretty::pprust::item_to_string; -use rustc_driver::{run_compiler, Compilation}; +use rustc_driver::{Compilation, run_compiler}; use rustc_interface::interface::{Compiler, Config}; use rustc_middle::ty::TyCtxt; diff --git a/rustfmt.toml b/rustfmt.toml new file mode 100644 index 00000000..b285329c --- /dev/null +++ b/rustfmt.toml @@ -0,0 +1,7 @@ +# matches that of rust-lang/rust +style_edition = "2024" +use_small_heuristics = "Max" +merge_derives = false +group_imports = "StdExternalCrate" +imports_granularity = "Module" +use_field_init_shorthand = true From fcb637040666808d123c5e702015c9524b12920d Mon Sep 17 00:00:00 2001 From: clubby789 Date: Sun, 30 Mar 2025 13:21:01 +0100 Subject: [PATCH 063/107] Fix partial clone link --- src/building/how-to-build-and-run.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/building/how-to-build-and-run.md b/src/building/how-to-build-and-run.md index 067e2871..c3c1c41e 100644 --- a/src/building/how-to-build-and-run.md +++ b/src/building/how-to-build-and-run.md @@ -63,7 +63,7 @@ cd rust > **NOTE**: A shallow clone limits which `git` commands can be run. > If you intend to work on and contribute to the compiler, it is > generally recommended to fully clone the repository [as shown above](#get-the-source-code), -> or to perform a [partial clone](#shallow-clone-the-repository) instead. +> or to perform a [partial clone](#partial-clone-the-repository) instead. > > For example, `git bisect` and `git blame` require access to the commit history, > so they don't work if the repository was cloned with `--depth 1`. From b67b4357e49c0026578803d0f14f45e454ee70bf Mon Sep 17 00:00:00 2001 From: Jieyou Xu Date: Wed, 2 Apr 2025 23:26:26 +0800 Subject: [PATCH 064/107] Preparing for merge from rustc --- rust-version | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust-version b/rust-version index 6baf4339..d7c20d8c 100644 --- a/rust-version +++ b/rust-version @@ -1 +1 @@ -493c38ba371929579fe136df26eccd9516347c7a +ae9173d7dd4a31806c950c90dcc331f1508b4d17 From a90cb7416c5755995f488c70e012068ea432e8b6 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Thu, 3 Apr 2025 02:04:49 +0200 Subject: [PATCH 065/107] test directive can appear anywhere in the file --- src/tests/directives.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/tests/directives.md b/src/tests/directives.md index 81aa35f1..8e4a7101 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -6,7 +6,8 @@ FIXME(jieyouxu) completely revise this chapter. --> -Directives are special comments that tell compiletest how to build and interpret a test. They must appear before the Rust source in the test. They may also appear in `rmake.rs` [run-make tests](compiletest.md#run-make-tests). +Directives are special comments that tell compiletest how to build and interpret a test. +They may also appear in `rmake.rs` [run-make tests](compiletest.md#run-make-tests). They are normally put after the short comment that explains the point of this test. Compiletest test suites use `//@` to signal that a comment is a directive. From 8f4357e7a2575b3109521476768d72db5ac14203 Mon Sep 17 00:00:00 2001 From: Vadim Petrochenkov Date: Sat, 29 Mar 2025 02:41:32 +0300 Subject: [PATCH 066/107] compiletest: Require `//~` annotations even if `error-pattern` is specified --- src/tests/ui.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/tests/ui.md b/src/tests/ui.md index 1190c264..6f412a7a 100644 --- a/src/tests/ui.md +++ b/src/tests/ui.md @@ -335,6 +335,9 @@ But for strict testing, try to use the `ERROR` annotation as much as possible, including `//~?` annotations for diagnostics without span. For compile time diagnostics `error-pattern` should very rarely be necessary. +Per-line annotations (`//~`) are still checked in tests using `error-pattern`, +to opt out of these checks in exceptional cases use `//@ compile-flags: --error-format=human`. + ### Error levels The error levels that you can have are: From 7fda61b1166c4e9e835684915007c33dbe41d18f Mon Sep 17 00:00:00 2001 From: binarycat Date: Thu, 3 Apr 2025 15:04:32 -0500 Subject: [PATCH 067/107] add some links about the rustdoc-gui test suite --- src/rustdoc.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/rustdoc.md b/src/rustdoc.md index 320dc9d5..e36d6a38 100644 --- a/src/rustdoc.md +++ b/src/rustdoc.md @@ -90,7 +90,9 @@ does is call the `main()` that's in this crate's `lib.rs`, though.) are in `tests/rustdoc-gui`. These use a [NodeJS tool called browser-UI-test](https://github.com/GuillaumeGomez/browser-UI-test/) that uses puppeteer to run tests in a headless browser and check rendering and - interactivity. + interactivity. For information on how to write this form of test, + see [`tests/rustdoc-gui/README.md`][rustdoc-gui-readme] + as well as [the description of the `.goml` format][goml-script] * Additionally, JavaScript type annotations are written using [TypeScript-flavored JSDoc] comments and an external d.ts file. The code itself is plain, valid JavaScript; we only use tsc as a linter. @@ -100,6 +102,8 @@ does is call the `main()` that's in this crate's `lib.rs`, though.) [These tests have several extra directives available to them](./rustdoc-internals/rustdoc-test-suite.md). [TypeScript-flavored JSDoc]: https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html +[rustdoc-gui-readme]: https://github.com/rust-lang/rust/blob/master/tests/rustdoc-gui/README.md +[goml-script]: https://github.com/GuillaumeGomez/browser-UI-test/blob/master/goml-script.md ## Constraints From 02aa6b48523a61a673e1740b55266fb794f4d0ca Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?G=C3=A1bor=20Szab=C3=B3?= Date: Fri, 4 Apr 2025 08:34:08 +0300 Subject: [PATCH 068/107] Update book.toml fix the authors field See https://rust-lang.github.io/mdBook/format/configuration/general.html#general-metadata --- book.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/book.toml b/book.toml index eb2f6806..b84b1e75 100644 --- a/book.toml +++ b/book.toml @@ -1,6 +1,6 @@ [book] title = "Rust Compiler Development Guide" -author = "The Rust Project Developers" +authors = ["The Rust Project Developers"] description = "A guide to developing the Rust compiler (rustc)" [build] From 4188afe2e760410545db90696fe4ef7e1e52688e Mon Sep 17 00:00:00 2001 From: Levi Zim Date: Sun, 6 Apr 2025 20:05:03 +0800 Subject: [PATCH 069/107] Fix deadlink in libs-and-metadata.md --- src/backend/libs-and-metadata.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/backend/libs-and-metadata.md b/src/backend/libs-and-metadata.md index 556b3fdf..513df165 100644 --- a/src/backend/libs-and-metadata.md +++ b/src/backend/libs-and-metadata.md @@ -110,7 +110,7 @@ See [`compute_hir_hash`] for where the hash is actually computed. [SVH]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_data_structures/svh/struct.Svh.html [incremental compilation]: ../queries/incremental-compilation.md -[`compute_hir_hash`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast_lowering/struct.LoweringContext.html#method.compute_hir_hash +[`compute_hir_hash`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast_lowering/fn.compute_hir_hash.html ### Stable Crate Id From e6458031c06a0b7050d331d7110f16b4be9999d7 Mon Sep 17 00:00:00 2001 From: The rustc-dev-guide Cronjob Bot Date: Mon, 7 Apr 2025 04:06:33 +0000 Subject: [PATCH 070/107] Preparing for merge from rustc --- rust-version | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust-version b/rust-version index d7c20d8c..a6f29510 100644 --- a/rust-version +++ b/rust-version @@ -1 +1 @@ -ae9173d7dd4a31806c950c90dcc331f1508b4d17 +25a615bf829b9f6d6f22da537e3851043f92e5f2 From e76ab78d1dba25ccc1ab0583b29572f706deb2da Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 7 Apr 2025 06:42:37 +0200 Subject: [PATCH 071/107] improve flow --- src/tests/ui.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/tests/ui.md b/src/tests/ui.md index 407862d4..e862a07c 100644 --- a/src/tests/ui.md +++ b/src/tests/ui.md @@ -335,8 +335,9 @@ But for strict testing, try to use the `ERROR` annotation as much as possible, including `//~?` annotations for diagnostics without span. For compile time diagnostics `error-pattern` should very rarely be necessary. -Per-line annotations (`//~`) are still checked in tests using `error-pattern`, -to opt out of these checks in exceptional cases use `//@ compile-flags: --error-format=human`. +Per-line annotations (`//~`) are still checked in tests using `error-pattern`. +To opt out of these checks, use `//@ compile-flags: --error-format=human`. +Do that only in exceptional cases. ### Error levels From f34fa18b015a1d9b563ff72eb9605c87208d84e9 Mon Sep 17 00:00:00 2001 From: timesince Date: Wed, 9 Apr 2025 18:46:50 +0800 Subject: [PATCH 072/107] Remove redundant words --- src/solve/opaque-types.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/solve/opaque-types.md b/src/solve/opaque-types.md index 672aab77..509c34a4 100644 --- a/src/solve/opaque-types.md +++ b/src/solve/opaque-types.md @@ -33,7 +33,7 @@ For opaque types in the defining scope and in the implicit-negative coherence mo always done in two steps. Outside of the defining scope `normalizes-to` for opaques always returns `Err(NoSolution)`. -We start by trying to to assign the expected type as a hidden type. +We start by trying to assign the expected type as a hidden type. In the implicit-negative coherence mode, this currently always results in ambiguity without interacting with the opaque types storage. We could instead add allow 'defining' all opaque types, From 5fb6b8e48faffaf2570e39a5ae262d4ffcdbf7d9 Mon Sep 17 00:00:00 2001 From: Jieyou Xu Date: Mon, 7 Apr 2025 13:18:03 +0800 Subject: [PATCH 073/107] rustc-dev-guide: document `needs-crate-type` --- src/tests/directives.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/tests/directives.md b/src/tests/directives.md index 8e4a7101..7ed583c1 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -191,8 +191,13 @@ settings: specified atomic widths, e.g. the test with `//@ needs-target-has-atomic: 8, 16, ptr` will only run if it supports the comma-separated list of atomic widths. -- `needs-dynamic-linking` - ignores if target does not support dynamic linking +- `needs-dynamic-linking` — ignores if target does not support dynamic linking (which is orthogonal to it being unable to create `dylib` and `cdylib` crate types) +- `needs-crate-type` — ignores if target platform does not support one or more + of the comma-delimited list of specified crate types. For example, + `//@ needs-crate-type: cdylib, proc-macro` will cause the test to be ignored + on `wasm32-unknown-unknown` target because the target does not support the + `proc-macro` crate type. The following directives will check LLVM support: From d7631d20aa877d810c7026112c8a8c34a16a21f2 Mon Sep 17 00:00:00 2001 From: Pietro Albini Date: Thu, 10 Apr 2025 12:34:57 +0200 Subject: [PATCH 074/107] mention --edition restrictions in rustc-dev-guide --- src/tests/directives.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/src/tests/directives.md b/src/tests/directives.md index 8e4a7101..f97f08e3 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -229,14 +229,14 @@ ignoring debuggers. ### Affecting how tests are built -| Directive | Explanation | Supported test suites | Possible values | -|---------------------|----------------------------------------------------------------------------------------------|---------------------------|------------------------------------------------------------------------------| -| `compile-flags` | Flags passed to `rustc` when building the test or aux file | All except for `run-make` | Any valid `rustc` flags, e.g. `-Awarnings -Dfoo`. Cannot be `-Cincremental`. | -| `edition` | Alias for `compile-flags: --edition=xxx` | All except for `run-make` | Any valid `--edition` value | -| `rustc-env` | Env var to set when running `rustc` | All except for `run-make` | `=` | -| `unset-rustc-env` | Env var to unset when running `rustc` | All except for `run-make` | Any env var name | -| `incremental` | Proper incremental support for tests outside of incremental test suite | `ui`, `crashes` | N/A | -| `no-prefer-dynamic` | Don't use `-C prefer-dynamic`, don't build as a dylib via a `--crate-type=dylib` preset flag | `ui`, `crashes` | N/A | +| Directive | Explanation | Supported test suites | Possible values | +|---------------------|----------------------------------------------------------------------------------------------|---------------------------|--------------------------------------------------------------------------------------------| +| `compile-flags` | Flags passed to `rustc` when building the test or aux file | All except for `run-make` | Any valid `rustc` flags, e.g. `-Awarnings -Dfoo`. Cannot be `-Cincremental` or `--edition` | +| `edition` | The edition used to build the test | All except for `run-make` | Any valid `--edition` value | +| `rustc-env` | Env var to set when running `rustc` | All except for `run-make` | `=` | +| `unset-rustc-env` | Env var to unset when running `rustc` | All except for `run-make` | Any env var name | +| `incremental` | Proper incremental support for tests outside of incremental test suite | `ui`, `crashes` | N/A | +| `no-prefer-dynamic` | Don't use `-C prefer-dynamic`, don't build as a dylib via a `--crate-type=dylib` preset flag | `ui`, `crashes` | N/A |
Tests (outside of `run-make`) that want to use incremental tests not in the From 0423c7b2df61391b126d5a4f4f883fac40056497 Mon Sep 17 00:00:00 2001 From: Vadim Petrochenkov Date: Thu, 10 Apr 2025 19:33:04 +0300 Subject: [PATCH 075/107] dev-guide: Document `dont-require-annotations` and its use cases in more detail --- src/tests/directives.md | 1 + src/tests/ui.md | 55 ++++++++++++++++++++++++++++------------- 2 files changed, 39 insertions(+), 17 deletions(-) diff --git a/src/tests/directives.md b/src/tests/directives.md index 8e4a7101..2ebd98f6 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -101,6 +101,7 @@ for more details. | `normalize-stdout` | Normalize actual stdout with a rule `"" -> ""` before comparing against snapshot | `ui`, `incremental` | `"" -> ""`, ``/`` is regex capture and replace syntax | | `dont-check-compiler-stderr` | Don't check actual compiler stderr vs stderr snapshot | `ui` | N/A | | `dont-check-compiler-stdout` | Don't check actual compiler stdout vs stdout snapshot | `ui` | N/A | +| `dont-require-annotations` | Don't require line annotations for the given diagnostic kind (`//~ KIND`) to be exhaustive | `ui`, `incremental` | `ERROR`, `WARN`, `NOTE`, `HELP`, `SUGGESTION` | | `run-rustfix` | Apply all suggestions via `rustfix`, snapshot fixed output, and check fixed output builds | `ui` | N/A | | `rustfix-only-machine-applicable` | `run-rustfix` but only machine-applicable suggestions | `ui` | N/A | | `exec-env` | Env var to set when executing a test | `ui`, `crashes` | `=` | diff --git a/src/tests/ui.md b/src/tests/ui.md index e862a07c..3243a353 100644 --- a/src/tests/ui.md +++ b/src/tests/ui.md @@ -303,8 +303,7 @@ It should be preferred to using `error-pattern`, which is imprecise and non-exha ### `error-pattern` The `error-pattern` [directive](directives.md) can be used for runtime messages, which don't -have a specific span, or for compile time messages if imprecise matching is required due to -multi-line platform specific diagnostics. +have a specific span, or in exceptional cases for compile time messages. Let's think about this test: @@ -318,7 +317,7 @@ fn main() { ``` We want to ensure this shows "index out of bounds" but we cannot use the `ERROR` -annotation since the error doesn't have any span. Then it's time to use the +annotation since the runtime error doesn't have any span. Then it's time to use the `error-pattern` directive: ```rust,ignore @@ -331,29 +330,51 @@ fn main() { } ``` -But for strict testing, try to use the `ERROR` annotation as much as possible, -including `//~?` annotations for diagnostics without span. -For compile time diagnostics `error-pattern` should very rarely be necessary. +Use of `error-pattern` is not recommended in general. -Per-line annotations (`//~`) are still checked in tests using `error-pattern`. -To opt out of these checks, use `//@ compile-flags: --error-format=human`. -Do that only in exceptional cases. +For strict testing of compile time output, try to use the line annotations `//~` as much as +possible, including `//~?` annotations for diagnostics without span. -### Error levels +If the compile time output is target dependent or too verbose, use directive +`//@ dont-require-annotations: ` to make the line annotation checking +non-exhaustive, some of the compiler messages can stay uncovered by annotations in this mode. -The error levels that you can have are: +For checking runtime output `//@ check-run-results` may be preferable. + +Only use `error-pattern` if none of the above works. + +Line annotations `//~` are still checked in tests using `error-pattern`. +In exceptional cases use `//@ compile-flags: --error-format=human` to opt out of these checks. + +### Diagnostic kinds (error levels) + +The diagnostic kinds that you can have are: - `ERROR` -- `WARN` or `WARNING` +- `WARN` (or `WARNING`) - `NOTE` -- `HELP` and `SUGGESTION` +- `HELP` +- `SUGGESTION` -You are allowed to not include a level, but you should include it at least for -the primary message. - -The `SUGGESTION` level is used for specifying what the expected replacement text +The `SUGGESTION` kind is used for specifying what the expected replacement text should be for a diagnostic suggestion. +`ERROR` and `WARN` kinds are required to be exhaustively covered by line annotations +`//~` by default. + +Other kinds only need to be line-annotated if at least one annotation of that kind appears +in the test file. For example, one `//~ NOTE` will also require all other `//~ NOTE`s in the file +to be written out explicitly. + +Use directive `//@ dont-require-annotations` to opt out of exhaustive annotations. +E.g. use `//@ dont-require-annotations: NOTE` to annotate notes selectively. +Avoid using this directive for `ERROR`s and `WARN`ings, unless there's a serious reason, like +target-dependent compiler output. + +Missing diagnostic kinds (`//~ message`) are currently accepted, but are being phased away. +They will match any compiler output kind, but will not force exhaustive annotations for that kind. +Prefer explicit kind and `//@ dont-require-annotations` to achieve the same effect. + UI tests use the `-A unused` flag by default to ignore all unused warnings, as unused warnings are usually not the focus of a test. However, simple code samples often have unused warnings. If the test is specifically testing an From ca410e88e47731b63c9f9f1ad626289be7e7315f Mon Sep 17 00:00:00 2001 From: Stan Manilov Date: Fri, 11 Apr 2025 12:12:46 +0300 Subject: [PATCH 076/107] Fix link to rustc_* TEST attributes in ui.md --- src/tests/ui.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tests/ui.md b/src/tests/ui.md index e862a07c..41dc9299 100644 --- a/src/tests/ui.md +++ b/src/tests/ui.md @@ -575,4 +575,4 @@ with "user-facing" Rust alone. Indeed, one could say that this slightly abuses the term "UI" (*user* interface) and turns such UI tests from black-box tests into white-box ones. Use them carefully and sparingly. -[compiler debugging]: ../compiler-debugging.md#rustc_test-attributes +[compiler debugging]: ../compiler-debugging.md#rustc_-test-attributes From 281f106cde5ab8556c52908a28cb31c861433894 Mon Sep 17 00:00:00 2001 From: Stan Manilov Date: Tue, 8 Apr 2025 11:34:55 +0300 Subject: [PATCH 077/107] Update table of contents in about-this-guide.md 1. added two new parts: Bootstrapping and Supporting Infrastructure; 2. touched up names of pre-existing parts, to match actual names in sidebar; 3. syntactic nits (start description of Analysis with a capital letter); and 4. make numbered list use only 1. Co-authored-by: Tshepang Mbambo --- src/about-this-guide.md | 36 ++++++++++++++++++++++-------------- 1 file changed, 22 insertions(+), 14 deletions(-) diff --git a/src/about-this-guide.md b/src/about-this-guide.md index 781a5c51..057e4a4c 100644 --- a/src/about-this-guide.md +++ b/src/about-this-guide.md @@ -3,33 +3,41 @@ This guide is meant to help document how rustc – the Rust compiler – works, as well as to help new contributors get involved in rustc development. -There are seven parts to this guide: +There are several parts to this guide: -1. [Building `rustc`][p1]: +1. [Building and debugging `rustc`][p1]: Contains information that should be useful no matter how you are contributing, about building, debugging, profiling, etc. -2. [Contributing to `rustc`][p2]: +1. [Contributing to Rust][p2]: Contains information that should be useful no matter how you are contributing, about procedures for contribution, using git and Github, stabilizing features, etc. -3. [High-Level Compiler Architecture][p3]: +1. [Bootstrapping][p3]: + Describes how the Rust compiler builds itself using previous versions, including + an introduction to the bootstrap process and debugging methods. +1. [High-level Compiler Architecture][p4]: Discusses the high-level architecture of the compiler and stages of the compile process. -4. [Source Code Representation][p4]: +1. [Source Code Representation][p5]: Describes the process of taking raw source code from the user and transforming it into various forms that the compiler can work with easily. -5. [Analysis][p5]: - discusses the analyses that the compiler uses to check various properties of the code +1. [Supporting Infrastructure][p6]: + Covers command-line argument conventions, compiler entry points like rustc_driver and + rustc_interface, and the design and implementation of errors and lints. +1. [Analysis][p7]: + Discusses the analyses that the compiler uses to check various properties of the code and inform later stages of the compile process (e.g., type checking). -6. [From MIR to Binaries][p6]: How linked executable machine code is generated. -7. [Appendices][p7] at the end with useful reference information. +1. [MIR to Binaries][p8]: How linked executable machine code is generated. +1. [Appendices][p9] at the end with useful reference information. There are a few of these with different information, including a glossary. [p1]: ./building/how-to-build-and-run.html [p2]: ./contributing.md -[p3]: ./part-2-intro.md -[p4]: ./part-3-intro.md -[p5]: ./part-4-intro.md -[p6]: ./part-5-intro.md -[p7]: ./appendix/background.md +[p3]: ./building/bootstrapping/intro.md +[p4]: ./part-2-intro.md +[p5]: ./part-3-intro.md +[p6]: ./cli.md +[p7]: ./part-4-intro.md +[p8]: ./part-5-intro.md +[p9]: ./appendix/background.md ### Constant change From 35acfcc46b7f2fbbff57c6e27bf18c6bdaec907c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Fri, 11 Apr 2025 21:30:10 +0200 Subject: [PATCH 078/107] Document that `opt-dist` requires metrics to be enabled --- src/building/optimized-build.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/src/building/optimized-build.md b/src/building/optimized-build.md index 0849464e..62dfaca8 100644 --- a/src/building/optimized-build.md +++ b/src/building/optimized-build.md @@ -109,11 +109,16 @@ like Python or LLVM. Here is an example of how can `opt-dist` be used locally (outside of CI): -1. Build the tool with the following command: +1. Enable metrics in your `bootstrap.toml` file, because `opt-dist` expects it to be enabled: + ```toml + [build] + metrics = true + ``` +2. Build the tool with the following command: ```bash ./x build tools/opt-dist ``` -2. Run the tool with the `local` mode and provide necessary parameters: +3. Run the tool with the `local` mode and provide necessary parameters: ```bash ./build/host/stage0-tools-bin/opt-dist local \ --target-triple \ # select target, e.g. "x86_64-unknown-linux-gnu" From 54b7da853ae269cf23c8469d89d7d8b4d6454a34 Mon Sep 17 00:00:00 2001 From: Freya Arbjerg Date: Sat, 12 Apr 2025 00:05:26 +0200 Subject: [PATCH 079/107] Update "crater" link to actually point to crater.md --- src/walkthrough.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/walkthrough.md b/src/walkthrough.md index 6e07ceb7..48b3f8bb 100644 --- a/src/walkthrough.md +++ b/src/walkthrough.md @@ -221,7 +221,7 @@ There are a couple of things that may happen for some PRs during the review proc some merge conflicts with other PRs that happen to get merged first. You should fix these merge conflicts using the normal git procedures. -[crater]: ./tests/intro.html#crater +[crater]: ./tests/crater.html If you are not doing a new feature or something like that (e.g. if you are fixing a bug), then that's it! Thanks for your contribution :) From 8d316037215b8665df30ca303cf0cff23279cf9d Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 12 Apr 2025 06:14:37 +0200 Subject: [PATCH 080/107] ease copy-paste --- src/tests/ecosystem.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/tests/ecosystem.md b/src/tests/ecosystem.md index f4b93492..eee07dd0 100644 --- a/src/tests/ecosystem.md +++ b/src/tests/ecosystem.md @@ -15,9 +15,11 @@ CI. See the [Crater chapter](crater.md) for more details. `cargotest` is a small tool which runs `cargo test` on a few sample projects (such as `servo`, `ripgrep`, `tokei`, etc.). This runs as part of CI and ensures -there aren't any significant regressions. +there aren't any significant regressions: -> Example: `./x test src/tools/cargotest` +```console +./x test src/tools/cargotest +``` ### Large OSS Project builders From 5c08026fe23aa29a1f78080e17308213b3475b3e Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 12 Apr 2025 09:39:13 +0200 Subject: [PATCH 081/107] use more simple language Not obvious what "sort by" means --- src/contributing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/contributing.md b/src/contributing.md index 09a7f912..1dcda962 100644 --- a/src/contributing.md +++ b/src/contributing.md @@ -346,7 +346,7 @@ function in the same way as other pull requests. [`src/doc`]: https://github.com/rust-lang/rust/tree/master/src/doc [std-root]: https://github.com/rust-lang/rust/blob/master/library/std/src/lib.rs#L1 -To find documentation-related issues, sort by the [A-docs label]. +To find documentation-related issues, use the [A-docs label]. You can find documentation style guidelines in [RFC 1574]. From 962701f4088dbc43654d41940507d10863ebb746 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 12 Apr 2025 10:21:10 +0200 Subject: [PATCH 082/107] remove implied text "how much to trust" implies the opposite --- src/contributing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/contributing.md b/src/contributing.md index 09a7f912..579a2439 100644 --- a/src/contributing.md +++ b/src/contributing.md @@ -373,7 +373,7 @@ Just a few things to keep in mind: There is no strict limit on line lengths; let the sentence or part of the sentence flow to its proper end on the same line. - When contributing text to the guide, please contextualize the information with some time period - and/or a reason so that the reader knows how much to trust or mistrust the information. + and/or a reason so that the reader knows how much to trust the information. Aim to provide a reasonable amount of context, possibly including but not limited to: - A reason for why the data may be out of date other than "change", From 8721d56e32c6fb3f2751baccce86166494859519 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 12 Apr 2025 10:24:31 +0200 Subject: [PATCH 083/107] add missing word --- src/contributing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/contributing.md b/src/contributing.md index 09a7f912..9e75d9d8 100644 --- a/src/contributing.md +++ b/src/contributing.md @@ -388,7 +388,7 @@ Just a few things to keep in mind: - january 2021 There is a CI action (in `~/.github/workflows/date-check.yml`) - that generates a monthly showing those that are over 6 months old + that generates a monthly report showing those that are over 6 months old ([example](https://github.com/rust-lang/rustc-dev-guide/issues/2052)). For the action to pick the date, From a0a83bc7c83c316ddf032e7375d0902e2313736f Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 12 Apr 2025 10:28:15 +0200 Subject: [PATCH 084/107] date-check rdg contribution section --- src/contributing.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/contributing.md b/src/contributing.md index 73006de6..aeea4504 100644 --- a/src/contributing.md +++ b/src/contributing.md @@ -395,20 +395,20 @@ Just a few things to keep in mind: add a special annotation before specifying the date: ```md - Sep 2024 + Apr 2025 ``` Example: ```md - As of Sep 2024, the foo did the bar. + As of Apr 2025, the foo did the bar. ``` For cases where the date should not be part of the visible rendered output, use the following instead: ```md - + ``` - A link to a relevant WG, tracking issue, `rustc` rustdoc page, or similar, that may provide From e4fab61c10cd56daa7d4f81cb5fc3c9fb1d319e5 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 12 Apr 2025 10:41:48 +0200 Subject: [PATCH 085/107] fix path --- src/contributing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/contributing.md b/src/contributing.md index 7946c444..0e622fff 100644 --- a/src/contributing.md +++ b/src/contributing.md @@ -387,7 +387,7 @@ Just a few things to keep in mind: - jan 2021 - january 2021 - There is a CI action (in `~/.github/workflows/date-check.yml`) + There is a CI action (in `.github/workflows/date-check.yml`) that generates a monthly report showing those that are over 6 months old ([example](https://github.com/rust-lang/rustc-dev-guide/issues/2052)). From 4a71db13fae335dd52ca21015d4222da9e88fb1e Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 12 Apr 2025 13:01:16 +0200 Subject: [PATCH 086/107] use consistent title capitalization --- src/SUMMARY.md | 40 +++++++++---------- src/ast-validation.md | 2 +- src/bug-fix-procedure.md | 2 +- src/building/suggested.md | 2 +- src/coherence.md | 1 - src/contributing.md | 2 +- src/crates-io.md | 2 +- src/diagnostics.md | 4 +- src/feature-gates.md | 2 +- src/incrcomp-debugging.md | 2 +- src/memory.md | 2 +- src/panic-implementation.md | 2 +- src/parallel-rustc.md | 10 ++--- .../incremental-compilation-in-detail.md | 2 +- .../query-evaluation-model-in-detail.md | 2 +- src/serialization.md | 2 +- src/test-implementation.md | 4 +- src/the-parser.md | 4 +- src/unsafety-checking.md | 2 +- 19 files changed, 44 insertions(+), 45 deletions(-) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 95a3cd7c..05c43c83 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -10,9 +10,9 @@ - [How to build and run the compiler](./building/how-to-build-and-run.md) - [Quickstart](./building/quickstart.md) - [Prerequisites](./building/prerequisites.md) - - [Suggested Workflows](./building/suggested.md) + - [Suggested workflows](./building/suggested.md) - [Distribution artifacts](./building/build-install-distribution-artifacts.md) - - [Building Documentation](./building/compiler-documenting.md) + - [Building documentation](./building/compiler-documenting.md) - [Rustdoc overview](./rustdoc.md) - [Adding a new target](./building/new-target.md) - [Optimized build](./building/optimized-build.md) @@ -42,11 +42,11 @@ - [with the linux perf tool](./profiling/with_perf.md) - [with Windows Performance Analyzer](./profiling/wpa_profiling.md) - [with the Rust benchmark suite](./profiling/with_rustc_perf.md) -- [crates.io Dependencies](./crates-io.md) +- [crates.io dependencies](./crates-io.md) # Contributing to Rust -- [Contribution Procedures](./contributing.md) +- [Contribution procedures](./contributing.md) - [About the compiler team](./compiler-team.md) - [Using Git](./git.md) - [Mastering @rustbot](./rustbot.md) @@ -56,7 +56,7 @@ - [Stabilizing Features](./stabilization_guide.md) - [Feature Gates](./feature-gates.md) - [Coding conventions](./conventions.md) -- [Procedures for Breaking Changes](./bug-fix-procedure.md) +- [Procedures for breaking changes](./bug-fix-procedure.md) - [Using external repositories](./external-repos.md) - [Fuzzing](./fuzzing.md) - [Notification groups](notification-groups/about.md) @@ -88,14 +88,14 @@ - [Overview of the compiler](./overview.md) - [The compiler source code](./compiler-src.md) - [Queries: demand-driven compilation](./query.md) - - [The Query Evaluation Model in Detail](./queries/query-evaluation-model-in-detail.md) + - [The Query Evaluation Model in detail](./queries/query-evaluation-model-in-detail.md) - [Incremental compilation](./queries/incremental-compilation.md) - - [Incremental compilation In Detail](./queries/incremental-compilation-in-detail.md) - - [Debugging and Testing](./incrcomp-debugging.md) + - [Incremental compilation in detail](./queries/incremental-compilation-in-detail.md) + - [Debugging and testing](./incrcomp-debugging.md) - [Salsa](./queries/salsa.md) -- [Memory Management in Rustc](./memory.md) -- [Serialization in Rustc](./serialization.md) -- [Parallel Compilation](./parallel-rustc.md) +- [Memory management in rustc](./memory.md) +- [Serialization in rustc](./serialization.md) +- [Parallel compilation](./parallel-rustc.md) - [Rustdoc internals](./rustdoc-internals.md) - [Search](./rustdoc-internals/search.md) - [The `rustdoc` test suite](./rustdoc-internals/rustdoc-test-suite.md) @@ -103,14 +103,14 @@ - [Prologue](./part-3-intro.md) - [Syntax and the AST](./syntax-intro.md) - - [Lexing and Parsing](./the-parser.md) + - [Lexing and parsing](./the-parser.md) - [Macro expansion](./macro-expansion.md) - [Name resolution](./name-resolution.md) - [Attributes](./attributes.md) - - [`#[test]` Implementation](./test-implementation.md) - - [Panic Implementation](./panic-implementation.md) - - [AST Validation](./ast-validation.md) - - [Feature Gate Checking](./feature-gate-ck.md) + - [`#[test]` implementation](./test-implementation.md) + - [Panic implementation](./panic-implementation.md) + - [AST validation](./ast-validation.md) + - [Feature gate checking](./feature-gate-ck.md) - [Lang Items](./lang-items.md) - [The HIR (High-level IR)](./hir.md) - [Lowering AST to HIR](./ast-lowering.md) @@ -129,7 +129,7 @@ - [Example: Type checking](./rustc-driver/interacting-with-the-ast.md) - [Example: Getting diagnostics](./rustc-driver/getting-diagnostics.md) - [Remarks on perma-unstable features](./rustc-driver/remarks-on-perma-unstable-features.md) -- [Errors and Lints](diagnostics.md) +- [Errors and lints](diagnostics.md) - [Diagnostic and subdiagnostic structs](./diagnostics/diagnostic-structs.md) - [Translation](./diagnostics/translation.md) - [`LintStore`](./diagnostics/lintstore.md) @@ -175,14 +175,14 @@ - [Type checking](./type-checking.md) - [Method Lookup](./method-lookup.md) - [Variance](./variance.md) - - [Coherence Checking](./coherence.md) - - [Opaque Types](./opaque-types-type-alias-impl-trait.md) + - [Coherence checking](./coherence.md) + - [Opaque types](./opaque-types-type-alias-impl-trait.md) - [Inference details](./opaque-types-impl-trait-inference.md) - [Return Position Impl Trait In Trait](./return-position-impl-trait-in-trait.md) - [Region inference restrictions][opaque-infer] - [Const condition checking](./effects.md) - [Pattern and Exhaustiveness Checking](./pat-exhaustive-checking.md) -- [Unsafety Checking](./unsafety-checking.md) +- [Unsafety checking](./unsafety-checking.md) - [MIR dataflow](./mir/dataflow.md) - [Drop elaboration](./mir/drop-elaboration.md) - [The borrow checker](./borrow_check.md) diff --git a/src/ast-validation.md b/src/ast-validation.md index fa0f1d95..8f10bbec 100644 --- a/src/ast-validation.md +++ b/src/ast-validation.md @@ -1,4 +1,4 @@ -# AST Validation +# AST validation _AST validation_ is a separate AST pass that visits each item in the tree and performs simple checks. This pass diff --git a/src/bug-fix-procedure.md b/src/bug-fix-procedure.md index e6a16df6..8e6725c5 100644 --- a/src/bug-fix-procedure.md +++ b/src/bug-fix-procedure.md @@ -1,4 +1,4 @@ -# Procedures for Breaking Changes +# Procedures for breaking changes diff --git a/src/building/suggested.md b/src/building/suggested.md index 43ff2ba7..a70d9d88 100644 --- a/src/building/suggested.md +++ b/src/building/suggested.md @@ -1,4 +1,4 @@ -# Suggested Workflows +# Suggested workflows The full bootstrapping process takes quite a while. Here are some suggestions to make your life easier. diff --git a/src/coherence.md b/src/coherence.md index b3af101f..73f9213b 100644 --- a/src/coherence.md +++ b/src/coherence.md @@ -1,4 +1,3 @@ - # Coherence > NOTE: this is based on [notes by @lcnr](https://github.com/rust-lang/rust/pull/121848) diff --git a/src/contributing.md b/src/contributing.md index 0e622fff..cc68c011 100644 --- a/src/contributing.md +++ b/src/contributing.md @@ -1,4 +1,4 @@ -# Contribution Procedures +# Contribution procedures diff --git a/src/crates-io.md b/src/crates-io.md index 403d61a8..4431585a 100644 --- a/src/crates-io.md +++ b/src/crates-io.md @@ -1,4 +1,4 @@ -# crates.io Dependencies +# crates.io dependencies The Rust compiler supports building with some dependencies from `crates.io`. Examples are `log` and `env_logger`. diff --git a/src/diagnostics.md b/src/diagnostics.md index 6f72ea90..2f8f4b0a 100644 --- a/src/diagnostics.md +++ b/src/diagnostics.md @@ -1,4 +1,4 @@ -# Errors and Lints +# Errors and lints @@ -772,7 +772,7 @@ store.register_renamed("single_use_lifetime", "single_use_lifetimes"); [`store.register_removed`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/struct.LintStore.html#method.register_removed [`rustc_lint::register_builtins`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/fn.register_builtins.html -### Lint Groups +### Lint groups Lints can be turned on in groups. These groups are declared in the [`register_builtins`][rbuiltins] function in [`rustc_lint::lib`][builtin]. The diff --git a/src/feature-gates.md b/src/feature-gates.md index 24ce9bb7..9806f73c 100644 --- a/src/feature-gates.md +++ b/src/feature-gates.md @@ -1,4 +1,4 @@ -# Feature Gates +# Feature gates This chapter is intended to provide basic help for adding, removing, and modifying feature gates. diff --git a/src/incrcomp-debugging.md b/src/incrcomp-debugging.md index 7045d3fa..a548215c 100644 --- a/src/incrcomp-debugging.md +++ b/src/incrcomp-debugging.md @@ -1,4 +1,4 @@ -# Debugging and Testing Dependencies +# Debugging and testing dependencies ## Testing the dependency graph diff --git a/src/memory.md b/src/memory.md index 1e030ff4..eeb4a813 100644 --- a/src/memory.md +++ b/src/memory.md @@ -1,4 +1,4 @@ -# Memory Management in Rustc +# Memory management in rustc Generally rustc tries to be pretty careful how it manages memory. The compiler allocates _a lot_ of data structures throughout compilation, diff --git a/src/panic-implementation.md b/src/panic-implementation.md index f3587428..468190ff 100644 --- a/src/panic-implementation.md +++ b/src/panic-implementation.md @@ -1,4 +1,4 @@ -# Panicking in rust +# Panicking in Rust diff --git a/src/parallel-rustc.md b/src/parallel-rustc.md index 690fb19c..ce69b66c 100644 --- a/src/parallel-rustc.md +++ b/src/parallel-rustc.md @@ -1,4 +1,4 @@ -# Parallel Compilation +# Parallel compilation
As of November 2024, @@ -28,7 +28,7 @@ The following sections are kept for now but are quite outdated. [codegen]: backend/codegen.md -## Code Generation +## Code generation During monomorphization the compiler splits up all the code to be generated into smaller chunks called _codegen units_. These are then generated by @@ -38,7 +38,7 @@ occurs in the [`rustc_codegen_ssa::base`] module. [`rustc_codegen_ssa::base`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/base/index.html -## Data Structures +## Data structures The underlying thread-safe data-structures used in the parallel compiler can be found in the [`rustc_data_structures::sync`] module. These data structures @@ -83,7 +83,7 @@ can be accessed directly through `Deref::deref`. [`rustc_data_structures::sync::worker_local`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_data_structures/sync/worker_local/index.html [`WorkerLocal`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_data_structures/sync/worker_local/struct.WorkerLocal.html -## Parallel Iterator +## Parallel iterator The parallel iterators provided by the [`rayon`] crate are easy ways to implement parallelism. In the current implementation of the parallel compiler @@ -124,7 +124,7 @@ the parallel iterator function has been used are as follows: There are still many loops that have the potential to use parallel iterators. -## Query System +## Query system The query model has some properties that make it actually feasible to evaluate multiple queries in parallel without too much effort: diff --git a/src/queries/incremental-compilation-in-detail.md b/src/queries/incremental-compilation-in-detail.md index 4133b196..03c822d4 100644 --- a/src/queries/incremental-compilation-in-detail.md +++ b/src/queries/incremental-compilation-in-detail.md @@ -1,4 +1,4 @@ -# Incremental Compilation In Detail +# Incremental Compilation in detail diff --git a/src/queries/query-evaluation-model-in-detail.md b/src/queries/query-evaluation-model-in-detail.md index f7f204bf..444e20bc 100644 --- a/src/queries/query-evaluation-model-in-detail.md +++ b/src/queries/query-evaluation-model-in-detail.md @@ -1,4 +1,4 @@ -# The Query Evaluation Model in Detail +# The Query Evaluation Model in detail diff --git a/src/serialization.md b/src/serialization.md index 0ff04990..670a37ff 100644 --- a/src/serialization.md +++ b/src/serialization.md @@ -1,4 +1,4 @@ -# Serialization in Rustc +# Serialization in rustc rustc has to [serialize] and deserialize various data during compilation. Specifically: diff --git a/src/test-implementation.md b/src/test-implementation.md index bee783c0..e906dd29 100644 --- a/src/test-implementation.md +++ b/src/test-implementation.md @@ -83,7 +83,7 @@ with your hand-written one, it will not share a [Symbol][Symbol]. This technique prevents name collision during code generation and is the foundation of Rust's [`macro`] hygiene. -## Step 2: Harness Generation +## Step 2: Harness generation Now that our tests are accessible from the root of our crate, we need to do something with them using [`rustc_ast`][ast] generates a module like so: @@ -106,7 +106,7 @@ called [`test`][test] that is part of Rust core, that implements all of the runtime for testing. [`test`][test]'s interface is unstable, so the only stable way to interact with it is through the `#[test]` macro. -## Step 3: Test Object Generation +## Step 3: Test object generation If you've written tests in Rust before, you may be familiar with some of the optional attributes available on test functions. For example, a test can be diff --git a/src/the-parser.md b/src/the-parser.md index 60a71ae3..601a81e2 100644 --- a/src/the-parser.md +++ b/src/the-parser.md @@ -1,4 +1,4 @@ -# Lexing and Parsing +# Lexing and parsing The very first thing the compiler does is take the program (in UTF-8 Unicode text) and turn it into a data format the compiler can work with more conveniently than strings. @@ -59,7 +59,7 @@ Note that while parsing, we may encounter macro definitions or invocations. We set these aside to be expanded (see [Macro Expansion](./macro-expansion.md)). Expansion itself may require parsing the output of a macro, which may reveal more macros to be expanded, and so on. -## More on Lexical Analysis +## More on lexical analysis Code for lexical analysis is split between two crates: diff --git a/src/unsafety-checking.md b/src/unsafety-checking.md index 11308789..fbc19d89 100644 --- a/src/unsafety-checking.md +++ b/src/unsafety-checking.md @@ -1,4 +1,4 @@ -# Unsafety Checking +# Unsafety checking Certain expressions in Rust can violate memory safety and as such need to be inside an `unsafe` block or function. The compiler will also warn if an unsafe From 031d694d696091f649415d4547aaffea2b7ec5a4 Mon Sep 17 00:00:00 2001 From: Urgau <3616612+Urgau@users.noreply.github.com> Date: Sat, 12 Apr 2025 18:24:59 +0200 Subject: [PATCH 087/107] Enable [canonicalize-issue-links] and [no-mentions] in triagebot.toml --- triagebot.toml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/triagebot.toml b/triagebot.toml index 12aa0b7b..6232dbf0 100644 --- a/triagebot.toml +++ b/triagebot.toml @@ -7,5 +7,9 @@ allow-unauthenticated = [ "blocked", ] +[no-mentions] + +[canonicalize-issue-links] + # Automatically close and reopen PRs made by bots to run CI on them [bot-pull-requests] From 79af744efe165cb9627b23b7a4e2f3dcee6d9ebb Mon Sep 17 00:00:00 2001 From: Jieyou Xu Date: Thu, 10 Apr 2025 07:14:39 +0800 Subject: [PATCH 088/107] tests: document `-A {unused,internal_features}` ui test mode presets --- src/tests/ui.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/src/tests/ui.md b/src/tests/ui.md index 41dc9299..e10bc2da 100644 --- a/src/tests/ui.md +++ b/src/tests/ui.md @@ -576,3 +576,26 @@ the term "UI" (*user* interface) and turns such UI tests from black-box tests into white-box ones. Use them carefully and sparingly. [compiler debugging]: ../compiler-debugging.md#rustc_-test-attributes + +## UI test mode preset lint levels + +By default, test suites under UI test mode (`tests/ui`, `tests/ui-fulldeps`, +but not `tests/rustdoc-ui`) will specify + +- `-A unused` +- `-A internal_features` + +If: + +- The ui test's pass mode is below `run` (i.e. check or build). +- No compare modes are specified. + +Since they can be very noisy in ui tests. + +You can override them with `compile-flags` lint level flags or +in-source lint level attributes as required. + +Note that the `rustfix` version will *not* have `-A unused` passed, +meaning that you may have to `#[allow(unused)]` to suppress `unused` +lints on the rustfix'd file (because we might be testing rustfix +on `unused` lints themselves). From b557b97e1ec1f35a5254b28c821eb888f564cb82 Mon Sep 17 00:00:00 2001 From: Nicholas Nethercote Date: Wed, 2 Apr 2025 13:00:45 +1100 Subject: [PATCH 089/107] Documentation fixes. Remove old references to the HIR map. --- src/appendix/glossary.md | 1 - src/hir.md | 43 ++++++++++++++++++---------------------- 2 files changed, 19 insertions(+), 25 deletions(-) diff --git a/src/appendix/glossary.md b/src/appendix/glossary.md index a7c3236d..1837b59e 100644 --- a/src/appendix/glossary.md +++ b/src/appendix/glossary.md @@ -31,7 +31,6 @@ Term | Meaning generics | The list of generic parameters defined on an item. There are three kinds of generic parameters: Type, lifetime and const parameters. HIR | The _high-level [IR](#ir)_, created by lowering and desugaring the AST. ([see more](../hir.md)) `HirId` | Identifies a particular node in the HIR by combining a def-id with an "intra-definition offset". See [the HIR chapter for more](../hir.md#identifiers-in-the-hir). -HIR map | The HIR map, accessible via `tcx.hir()`, allows you to quickly navigate the HIR and convert between various forms of identifiers. ICE | Short for _internal compiler error_, this is when the compiler crashes. ICH | Short for _incremental compilation hash_, these are used as fingerprints for things such as HIR and crate metadata, to check if changes have been made. This is useful in incremental compilation to see if part of a crate has changed and should be recompiled. `infcx` | The type inference context (`InferCtxt`). (see `rustc_middle::infer`) diff --git a/src/hir.md b/src/hir.md index 75f5a9e2..65779f31 100644 --- a/src/hir.md +++ b/src/hir.md @@ -100,7 +100,7 @@ The HIR uses a bunch of different identifiers that coexist and serve different p a wrapper around a [`HirId`]. For more info about HIR bodies, please refer to the [HIR chapter][hir-bodies]. -These identifiers can be converted into one another through the [HIR map][map]. +These identifiers can be converted into one another through the `TyCtxt`. [`DefId`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/def_id/struct.DefId.html [`LocalDefId`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/def_id/struct.LocalDefId.html @@ -110,30 +110,24 @@ These identifiers can be converted into one another through the [HIR map][map]. [`CrateNum`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/def_id/struct.CrateNum.html [`DefIndex`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/def_id/struct.DefIndex.html [`Body`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/struct.Body.html -[hir-map]: ./hir.md#the-hir-map [hir-bodies]: ./hir.md#hir-bodies -[map]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html -## The HIR Map +## HIR Operations Most of the time when you are working with the HIR, you will do so via -the **HIR Map**, accessible in the tcx via [`tcx.hir()`] (and defined in -the [`hir::map`] module). The [HIR map] contains a [number of methods] to -convert between IDs of various kinds and to lookup data associated -with a HIR node. +`TyCtxt`. It contains a number of methods, defined in the `hir::map` module and +mostly prefixed with `hir_`, to convert between IDs of various kinds and to +lookup data associated with a HIR node. -[`tcx.hir()`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.hir -[`hir::map`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/index.html -[HIR map]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html -[number of methods]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html#methods +[`TyCtxt`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html For example, if you have a [`LocalDefId`], and you would like to convert it -to a [`HirId`], you can use [`tcx.hir().local_def_id_to_hir_id(def_id)`][local_def_id_to_hir_id]. +to a [`HirId`], you can use [`tcx.local_def_id_to_hir_id(def_id)`][local_def_id_to_hir_id]. You need a `LocalDefId`, rather than a `DefId`, since only local items have HIR nodes. -[local_def_id_to_hir_id]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html#method.local_def_id_to_hir_id +[local_def_id_to_hir_id]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.local_def_id_to_hir_id -Similarly, you can use [`tcx.hir().find(n)`][find] to lookup the node for a +Similarly, you can use [`tcx.hir_node(n)`][hir_node] to lookup the node for a [`HirId`]. This returns a `Option>`, where [`Node`] is an enum defined in the map. By matching on this, you can find out what sort of node the `HirId` referred to and also get a pointer to the data @@ -142,15 +136,16 @@ that `n` must be some HIR expression, you can do [`tcx.hir_expect_expr(n)`][expect_expr], which will extract and return the [`&hir::Expr`][Expr], panicking if `n` is not in fact an expression. -[find]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html#method.find +[hir_node]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.hir_node [`Node`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/enum.Node.html [expect_expr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.expect_expr [Expr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/struct.Expr.html -Finally, you can use the HIR map to find the parents of nodes, via -calls like [`tcx.hir().get_parent(n)`][get_parent]. +Finally, you can find the parents of nodes, via +calls like [`tcx.parent_hir_node(n)`][parent_hir_node]. + +[get_parent_item]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.parent_hir_node -[get_parent]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html#method.get_parent ## HIR Bodies @@ -158,10 +153,10 @@ A [`rustc_hir::Body`] represents some kind of executable code, such as the body of a function/closure or the definition of a constant. Bodies are associated with an **owner**, which is typically some kind of item (e.g. an `fn()` or `const`), but could also be a closure expression -(e.g. `|x, y| x + y`). You can use the HIR map to find the body -associated with a given def-id ([`maybe_body_owned_by`]) or to find -the owner of a body ([`body_owner_def_id`]). +(e.g. `|x, y| x + y`). You can use the `TyCtxt` to find the body +associated with a given def-id ([`hir_maybe_body_owned_by`]) or to find +the owner of a body ([`hir_body_owner_def_id`]). [`rustc_hir::Body`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/struct.Body.html -[`maybe_body_owned_by`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html#method.maybe_body_owned_by -[`body_owner_def_id`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html#method.body_owner_def_id +[`hir_maybe_body_owned_by`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.hir_maybe_body_owned_by +[`hir_body_owner_def_id`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.hir_body_owner_def_id From 07d3c696375905175631b923d9c6e4815c1662a1 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Mon, 14 Apr 2025 08:41:22 +0200 Subject: [PATCH 090/107] clean "Coding conventions" chapter - use correct code block markers - add missing title - rustfmt can now use edition setting in its config ... and this is set in Rust repo - reduce visual noise - needless repetition - convention is to start sentence with upper case - sembr - whitespace - semi-heading not adding much value - fix grammar --- src/conventions.md | 65 +++++++++++++++++++++++----------------------- 1 file changed, 33 insertions(+), 32 deletions(-) diff --git a/src/conventions.md b/src/conventions.md index 0e624a45..4356cf24 100644 --- a/src/conventions.md +++ b/src/conventions.md @@ -1,3 +1,5 @@ +# Coding conventions + This file offers some tips on the coding conventions for rustc. This chapter covers [formatting](#formatting), [coding for correctness](#cc), [using crates from crates.io](#cio), and some tips on @@ -5,7 +7,7 @@ chapter covers [formatting](#formatting), [coding for correctness](#cc), -# Formatting and the tidy script +## Formatting and the tidy script rustc is moving towards the [Rust standard coding style][fmt]. @@ -20,44 +22,42 @@ Formatting is checked by the `tidy` script. It runs automatically when you do `./x test` and can be run in isolation with `./x fmt --check`. If you want to use format-on-save in your editor, the pinned version of -`rustfmt` is built under `build//stage0/bin/rustfmt`. You'll have to -pass the `--edition=2021` argument yourself when calling -`rustfmt` directly. +`rustfmt` is built under `build//stage0/bin/rustfmt`. [fmt]: https://github.com/rust-dev-tools/fmt-rfcs - [`rustfmt`]:https://github.com/rust-lang/rustfmt -## Formatting C++ code +### Formatting C++ code The compiler contains some C++ code for interfacing with parts of LLVM that don't have a stable C API. When modifying that code, use this command to format it: -```sh -./x test tidy --extra-checks=cpp:fmt --bless +```console +./x test tidy --extra-checks cpp:fmt --bless ``` This uses a pinned version of `clang-format`, to avoid relying on the local environment. -## Formatting and linting Python code +### Formatting and linting Python code The Rust repository contains quite a lot of Python code. We try to keep -it both linted and formatted by the [ruff][ruff] tool. +it both linted and formatted by the [ruff] tool. When modifying Python code, use this command to format it: -```sh -./x test tidy --extra-checks=py:fmt --bless + +```console +./x test tidy --extra-checks py:fmt --bless ``` -and the following command to run lints: -```sh -./x test tidy --extra-checks=py:lint +And, the following command to run lints: + +```console +./x test tidy --extra-checks py:lint ``` -This uses a pinned version of `ruff`, to avoid relying on the local -environment. +These use a pinned version of `ruff`, to avoid relying on the local environment. [ruff]: https://github.com/astral-sh/ruff @@ -65,7 +65,7 @@ environment. -## Copyright notice +### Copyright notice In the past, files began with a copyright and license notice. Please **omit** @@ -75,41 +75,42 @@ MIT/Apache-2.0). All of the copyright notices should be gone by now, but if you come across one in the rust-lang/rust repo, feel free to open a PR to remove it. -## Line length +### Line length Lines should be at most 100 characters. It's even better if you can keep things to 80. -**Ignoring the line length limit.** Sometimes – in particular for -tests – it can be necessary to exempt yourself from this limit. In -that case, you can add a comment towards the top of the file like so: +Sometimes, and particularly for tests, it can be necessary to exempt yourself from this limit. +In that case, you can add a comment towards the top of the file like so: ```rust // ignore-tidy-linelength ``` -## Tabs vs spaces +### Tabs vs spaces -Prefer 4-space indent. +Prefer 4-space indents. -# Coding for correctness +## Coding for correctness Beyond formatting, there are a few other tips that are worth following. -## Prefer exhaustive matches +### Prefer exhaustive matches Using `_` in a match is convenient, but it means that when new variants are added to the enum, they may not get handled correctly. Ask yourself: if a new variant were added to this enum, what's the chance that it would want to use the `_` code, versus having some other treatment? Unless the answer is "low", then prefer an -exhaustive match. (The same advice applies to `if let` and `while -let`, which are effectively tests for a single variant.) +exhaustive match. -## Use "TODO" comments for things you don't want to forget +The same advice applies to `if let` and `while let`, +which are effectively tests for a single variant. + +### Use "TODO" comments for things you don't want to forget As a useful tool to yourself, you can insert a `// TODO` comment for something that you want to get back to before you land your PR: @@ -136,13 +137,13 @@ if foo { -# Using crates from crates.io +## Using crates from crates.io See the [crates.io dependencies][crates] section. -# How to structure your PR +## How to structure your PR How you prepare the commits in your PR can make a big difference for the reviewer. Here are some tips. @@ -172,7 +173,7 @@ require that every intermediate commit successfully builds – we only expect to be able to bisect at a PR level. However, if you *can* make individual commits build, that is always helpful. -# Naming conventions +## Naming conventions Apart from normal Rust style/naming conventions, there are also some specific to the compiler. From b4426b7c4b6da4d1a8054fbbac4abda5c4684b94 Mon Sep 17 00:00:00 2001 From: onur-ozkan Date: Tue, 25 Mar 2025 20:17:32 +0300 Subject: [PATCH 091/107] document config extensions Signed-off-by: onur-ozkan --- src/building/suggested.md | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/src/building/suggested.md b/src/building/suggested.md index 43ff2ba7..5d37a78a 100644 --- a/src/building/suggested.md +++ b/src/building/suggested.md @@ -20,6 +20,42 @@ your `.git/hooks` folder as `pre-push` (without the `.sh` extension!). You can also install the hook as a step of running `./x setup`! +## Config extensions + +When working on different tasks, you might need to switch between different bootstrap configurations. +Sometimes you may want to keep an old configuration for future use. But saving raw config values in +random files and manually copying and pasting them can quickly become messy, especially if you have a +long history of different configurations. + +To simplify managing multiple configurations, you can create config extensions. + +For example, you can create a simple config file named `cross.toml`: + +```toml +[build] +build = "x86_64-unknown-linux-gnu" +host = ["i686-unknown-linux-gnu"] +target = ["i686-unknown-linux-gnu"] + + +[llvm] +download-ci-llvm = false + +[target.x86_64-unknown-linux-gnu] +llvm-config = "/path/to/llvm-19/bin/llvm-config" +``` + +Then, include this in your `bootstrap.toml`: + +```toml +include = ["cross.toml"] +``` + +You can also include extensions within extensions recursively. + +**Note:** In the `include` field, the overriding logic follows a right-to-left order. Also, the outer +extension/config always overrides the inner ones. + ## Configuring `rust-analyzer` for `rustc` ### Project-local rust-analyzer setup From 3ed7cc887e32fe09a710ec00e0089bf19ecb3210 Mon Sep 17 00:00:00 2001 From: onur-ozkan Date: Tue, 1 Apr 2025 11:53:32 +0300 Subject: [PATCH 092/107] document `include` in `bootstrap.example.toml` Signed-off-by: onur-ozkan --- src/building/suggested.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/building/suggested.md b/src/building/suggested.md index 5d37a78a..b2e258be 100644 --- a/src/building/suggested.md +++ b/src/building/suggested.md @@ -53,8 +53,9 @@ include = ["cross.toml"] You can also include extensions within extensions recursively. -**Note:** In the `include` field, the overriding logic follows a right-to-left order. Also, the outer -extension/config always overrides the inner ones. +**Note:** In the `include` field, the overriding logic follows a right-to-left order. For example, +in `include = ["a.toml", "b.toml"]`, extension `b.toml` overrides `a.toml`. Also, parent extensions +always overrides the inner ones. ## Configuring `rust-analyzer` for `rustc` From 220d208efdd5b4c1b9219af51da9670e08ccf0fe Mon Sep 17 00:00:00 2001 From: Nicholas Nethercote Date: Thu, 27 Mar 2025 15:52:33 +1100 Subject: [PATCH 093/107] Rename `LifetimeName` as `LifetimeKind`. It's a much better name, more consistent with how we name such things. Also rename `Lifetime::res` as `Lifetime::kind` to match. I suspect this field used to have the type `LifetimeRes` and then the type was changed but the field name remained the same. --- src/ty.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ty.md b/src/ty.md index b33d5403..ce6cffec 100644 --- a/src/ty.md +++ b/src/ty.md @@ -61,11 +61,11 @@ Here is a summary: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Describe the *syntax* of a type: what the user wrote (with some desugaring). | Describe the *semantics* of a type: the meaning of what the user wrote. | | Each `rustc_hir::Ty` has its own spans corresponding to the appropriate place in the program. | Doesn’t correspond to a single place in the user’s program. | -| `rustc_hir::Ty` has generics and lifetimes; however, some of those lifetimes are special markers like [`LifetimeName::Implicit`][implicit]. | `ty::Ty` has the full type, including generics and lifetimes, even if the user left them out | +| `rustc_hir::Ty` has generics and lifetimes; however, some of those lifetimes are special markers like [`LifetimeKind::Implicit`][implicit]. | `ty::Ty` has the full type, including generics and lifetimes, even if the user left them out | | `fn foo(x: u32) → u32 { }` - Two `rustc_hir::Ty` representing each usage of `u32`, each has its own `Span`s, and `rustc_hir::Ty` doesn’t tell us that both are the same type | `fn foo(x: u32) → u32 { }` - One `ty::Ty` for all instances of `u32` throughout the program, and `ty::Ty` tells us that both usages of `u32` mean the same type. | -| `fn foo(x: &u32) -> &u32)` - Two `rustc_hir::Ty` again. Lifetimes for the references show up in the `rustc_hir::Ty`s using a special marker, [`LifetimeName::Implicit`][implicit]. | `fn foo(x: &u32) -> &u32)`- A single `ty::Ty`. The `ty::Ty` has the hidden lifetime param. | +| `fn foo(x: &u32) -> &u32)` - Two `rustc_hir::Ty` again. Lifetimes for the references show up in the `rustc_hir::Ty`s using a special marker, [`LifetimeKind::Implicit`][implicit]. | `fn foo(x: &u32) -> &u32)`- A single `ty::Ty`. The `ty::Ty` has the hidden lifetime param. | -[implicit]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/enum.LifetimeName.html#variant.Implicit +[implicit]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/enum.LifetimeKind.html#variant.Implicit **Order** @@ -323,4 +323,4 @@ When looking at the debug output of `Ty` or simply talking about different types - Generic parameters: `{name}/#{index}` e.g. `T/#0`, where `index` corresponds to its position in the list of generic parameters - Inference variables: `?{id}` e.g. `?x`/`?0`, where `id` identifies the inference variable - Variables from binders: `^{binder}_{index}` e.g. `^0_x`/`^0_2`, where `binder` and `index` identify which variable from which binder is being referred to -- Placeholders: `!{id}` or `!{id}_{universe}` e.g. `!x`/`!0`/`!x_2`/`!0_2`, representing some unique type in the specified universe. The universe is often elided when it is `0` \ No newline at end of file +- Placeholders: `!{id}` or `!{id}_{universe}` e.g. `!x`/`!0`/`!x_2`/`!0_2`, representing some unique type in the specified universe. The universe is often elided when it is `0` From 299077b6c33c40da5de093ee48b743ffbe9d57aa Mon Sep 17 00:00:00 2001 From: Nicholas Nethercote Date: Wed, 16 Apr 2025 08:57:15 +1000 Subject: [PATCH 094/107] Improve `borrowck_graphviz_*` documentation. In particular, `borrowck_graphviz_preflow` no longer exists. --- src/compiler-debugging.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/compiler-debugging.md b/src/compiler-debugging.md index 47f39762..102e2020 100644 --- a/src/compiler-debugging.md +++ b/src/compiler-debugging.md @@ -301,7 +301,8 @@ Right below you can find elaborate explainers on a selected few. Some compiler options for debugging specific features yield graphviz graphs - e.g. the `#[rustc_mir(borrowck_graphviz_postflow="suffix.dot")]` attribute -dumps various borrow-checker dataflow graphs. +on a function dumps various borrow-checker dataflow graphs in conjunction with +`-Zdump-mir-dataflow`. These all produce `.dot` files. To view these files, install graphviz (e.g. `apt-get install graphviz`) and then run the following commands: From 523f2a571af427c74da9dbb6b10f1df3f8ca2a7d Mon Sep 17 00:00:00 2001 From: xizheyin Date: Wed, 16 Apr 2025 13:09:29 +0800 Subject: [PATCH 095/107] Remind to update dev branch while behind too many commits Signed-off-by: xizheyin --- src/contributing.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/contributing.md b/src/contributing.md index 09a7f912..8c59ac6e 100644 --- a/src/contributing.md +++ b/src/contributing.md @@ -150,6 +150,20 @@ when contributing to Rust under [the git section](./git.md). [t-compiler]: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler [triagebot]: https://github.com/rust-lang/rust/blob/master/triagebot.toml +### Keeping your branch up-to-date + +The CI in rust-lang/rust applies your patches directly against the current master, +not against the commit your branch is based on. This can lead to unexpected failures +if your branch is outdated, even when there are no explicit merge conflicts. + +Before submitting or updating a PR, make sure to update your branch +as mentioned [here](git.md#keeping-things-up-to-date) if it's significantly +behind the master branch (e.g., more than 100 commits behind). +This fetches the latest master branch and rebases your changes on top of it, +ensuring your PR is tested against the latest code. + +After rebasing, it's recommended to [run the relevant tests locally](tests/intro.md) to catch any issues before CI runs. + ### r? All pull requests are reviewed by another person. We have a bot, From 5cedd19842f4f9fc0fc4dd6776ab9348ead9d354 Mon Sep 17 00:00:00 2001 From: Manuel Drehwald Date: Wed, 16 Apr 2025 20:03:05 -0400 Subject: [PATCH 096/107] add a first version of autodiff docs --- src/SUMMARY.md | 5 ++ src/autodiff/debugging.md | 113 ++++++++++++++++++++++++++++++++++++ src/autodiff/flags.md | 42 ++++++++++++++ src/autodiff/internals.md | 27 +++++++++ src/autodiff/limitations.md | 27 +++++++++ 5 files changed, 214 insertions(+) create mode 100644 src/autodiff/debugging.md create mode 100644 src/autodiff/flags.md create mode 100644 src/autodiff/internals.md create mode 100644 src/autodiff/limitations.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 95a3cd7c..d8c7243f 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -99,6 +99,11 @@ - [Rustdoc internals](./rustdoc-internals.md) - [Search](./rustdoc-internals/search.md) - [The `rustdoc` test suite](./rustdoc-internals/rustdoc-test-suite.md) +- [Autodiff internals](./autodiff/internals.md) + - [How to debug](./autodiff/debugging.md) + - [Autodiff flags](./autodiff/flags.md) + - [Current limitations](./autodiff/limitations.md) + # Source Code Representation - [Prologue](./part-3-intro.md) diff --git a/src/autodiff/debugging.md b/src/autodiff/debugging.md new file mode 100644 index 00000000..bd46a66f --- /dev/null +++ b/src/autodiff/debugging.md @@ -0,0 +1,113 @@ +# Reporting backend crashes + +If after a compilation failure you are greeted by a large amount of llvm-ir code, then our enzyme backend likely failed to compile your code. These cases are harder to debug, so your help is highly appreciated. Please also keep in mind that release builds are usually much more likely to work at the moment. + +The final goal here is to reproduce your bug in the enzyme [compiler explorer](https://enzyme.mit.edu/explorer/), in order to create a bug report in the [Enzyme](https://github.com/enzymead/enzyme/issues) repository. + +We have an `autodiff` flag which you can pass to `rustflags` to help with this. it will print the whole llvm-ir module, along with some `__enzyme_fwddiff` or `__enzyme_autodiff` calls. A potential workflow on linux could look like: + +## Controlling llvm-ir generation + +Before generating the llvm-ir, keep in mind two techniques that can help ensure the relevant rust code is visible for debugging: + +- **`std::hint::black_box`**: wrap rust variables or expressions in `std::hint::black_box()` to prevent rust and llvm from optimizing them away. This is useful when you need to inspect or manually manipulate specific values in the llvm-ir. +- **`extern "rust"` or `extern "c"`**: if you want to see how a specific function declaration is lowered to llvm-ir, you can declare it as `extern "rust"` or `extern "c"`. You can also look for existing `__enzyme_autodiff` or similar declarations within the generated module for examples. + +## 1) Generate an llvm-ir reproducer + +```sh +rustflags="-z autodiff=enable,printmodbefore" cargo +enzyme build --release &> out.ll +``` + +This also captures a few warnings and info messages above and below your module. open out.ll and remove every line above `; moduleid = `. Now look at the end of the file and remove everything that's not part of llvm-ir, i.e. remove errors and warnings. The last line of your llvm-ir should now start with `! = `, i.e. `!40831 = !{i32 0, i32 1037508, i32 1037538, i32 1037559}` or `!43760 = !dilocation(line: 297, column: 5, scope: !43746)`. + +The actual numbers will depend on your code. + +## 2) Check your llvm-ir reproducer + +To confirm that your previous step worked, we will use llvm's `opt` tool. find your path to the opt binary, with a path similar to `/rust/build//build/bin/opt`. also find `llvmenzyme-19.` path, similar to `/rust/build/target-tripple/enzyme/build/enzyme/llvmenzyme-19`. Please keep in mind that llvm frequently updates it's llvm backend, so the version number might be higher (20, 21, ...). Once you have both, run the following command: + +```sh + out.ll -load-pass-plugin=/path/to/llvmenzyme-19.so -passes="enzyme" -s +``` + +If the previous step succeeded, you are going to see the same error that you saw when compiling your rust code with cargo. + +If you fail to get the same error, please open an issue in the rust repository. If you succeed, congrats! the file is still huge, so let's automatically minimize it. + +## 3) Minimize your llvm-ir reproducer + +First find your `llvm-extract` binary, it's in the same folder as your opt binary. then run: + +```sh + -s --func= --recursive --rfunc="enzyme_autodiff*" --rfunc="enzyme_fwddiff*" --rfunc= out.ll -o mwe.ll +``` + +This command creates `mwe.ll`, a minimal working example. + +Please adjust the name passed with the last `--func` flag. You can either apply the `#[no_mangle]` attribute to the function you differentiate, then you can replace it with the rust name. otherwise you will need to look up the mangled function name. To do that, open `out.ll` and search for `__enzyme_fwddiff` or `__enzyme_autodiff`. the first string in that function call is the name of your function. example: + +```llvm-ir +define double @enzyme_opt_helper_0(ptr %0, i64 %1, double %2) { + %4 = call double (...) @__enzyme_fwddiff(ptr @_zn2ad3_f217h3b3b1800bd39fde3e, metadata !"enzyme_const", ptr %0, metadata !"enzyme_const", i64 %1, metadata !"enzyme_dup", double %2, double %2) + ret double %4 +} +``` + +Here, `_zn2ad3_f217h3b3b1800bd39fde3e` is the correct name. make sure to not copy the leading `@`. redo step 2) by running the `opt` command again, but this time passing `mwe.ll` as the input file instead of `out.ll`. Check if this minimized example still reproduces the crash. + +## 4) (Optional) Minimize your llvm-ir reproducer further. + +After the previous step you should have an `mwe.ll` file with ~5k loc. let's try to get it down to 50. find your `llvm-reduce` binary next to `opt` and `llvm-extract`. Copy the first line of your error message, an example could be: + +```sh +opt: /home/manuel/prog/rust/src/llvm-project/llvm/lib/ir/instructions.cpp:686: void llvm::callinst::init(llvm::functiontype*, llvm::value*, llvm::arrayref, llvm::arrayref >, const llvm::twine&): assertion `(args.size() == fty->getnumparams() || (fty->isvararg() && args.size() > fty->getnumparams())) && "calling a function with bad signature!"' failed. +``` + +If you just get a `segfault` there is no sensible error message and not much to do automatically, so continue to 5). +otherwise, create a `script.sh` file containing + +```sh +#!/bin/bash + $1 -load-pass-plugin=/path/to/llvmenzyme-19.so -passes="enzyme" \ + |& grep "/some/path.cpp:686: void llvm::callinst::init" +``` + +Experiment a bit with which error message you pass to grep. it should be long enough to make sure that the error is unique. However, for longer errors including `(` or `)` you will need to escape them correctly which can become annoying. Run + +```sh + --test=script.sh mwe.ll +``` + +If you see `input isn't interesting! verify interesting-ness test`, you got the error message in script.sh wrong, you need to make sure that grep matches your actual error. If all works out, you will see a lot of iterations, ending with a new `reduced.ll` file. Verify with `opt` that you still get the same error. + +### Advanced debugging: manual llvm-ir investigation + +Once you have a minimized reproducer (`mwe.ll` or `reduced.ll`), you can delve deeper: + +- **manual editing:** try manually rewriting the llvm-ir. for certain issues, like those involving indirect calls, you might investigate enzyme-specific intrinsics like `__enzyme_virtualreverse`. Understanding how to use these might require consulting enzyme's documentation or source code. +- **enzyme test cases:** look for relevant test cases within the [enzyme repository](https://github.com/enzymead/enzyme/tree/main/enzyme/test) that might demonstrate the correct usage of features or intrinsics related to your problem. + +## 5) Report your bug. + +Afterwards, you should be able to copy and paste your `mwe.ll` (or `reduced.ll`) example into our [compiler explorer](https://enzyme.mit.edu/explorer/). + +- Select `llvm ir` as language and `opt 20` as compiler. +- Replace the field to the right of your compiler with `-passes="enzyme"`, if it is not already set. +- Hopefully, you will see once again your now familiar error. +- Please use the share button to copy links to them. +- Please create an issue on [https://github.com/enzymead/enzyme/issues](https://github.com/enzymead/enzyme/issues) and share `mwe.ll` and (if you have it) `reduced.ll`, as well as links to the compiler explorer. Please feel free to also add your rust code or a link to it. + +#### Documenting findings + +some enzyme errors, like `"attempting to call an indirect active function whose runtime value is inactive"`, have historically caused confusion. If you investigate such an issue, even if you don't find a complete solution, please consider documenting your findings. If the insights are general to enzyme and not specific to its rust usage, contributing them to the main [enzyme documentation](https://github.com/enzymead/www) is often the best first step. You can also mention your findings in the relevant enzyme github issue or propose updates to these docs if appropriate. This helps prevent others from starting from scratch. + +With a clear reproducer and documentation, hopefully an enzyme developer will be able to fix your bug. Once that happens, the enzyme submodule inside the rust compiler will be updated, which should allow you to differentiate your rust code. Thanks for helping us to improve rust-ad. + +# Minimize rust code + +Beyond having a minimal llvm-ir reproducer, it is also helpful to have a minimal rust reproducer without dependencies. This allows us to add it as a test case to ci once we fix it, which avoids regressions for the future. + +There are a few solutions to help you with minimizing the rust reproducer. This is probably the most simple automated approach: [cargo-minimize](https://github.com/nilstrieb/cargo-minimize). + +Otherwise we have various alternatives, including [`treereduce`](https://github.com/langston-barrett/treereduce), [`halfempty`](https://github.com/googleprojectzero/halfempty), or [`picireny`](https://github.com/renatahodovan/picireny), potentially also [`creduce`](https://github.com/csmith-project/creduce). diff --git a/src/autodiff/flags.md b/src/autodiff/flags.md new file mode 100644 index 00000000..946ae1d0 --- /dev/null +++ b/src/autodiff/flags.md @@ -0,0 +1,42 @@ +# Supported `RUSTFLAGS` + +To support you while debugging or profiling, we have added support for an experimental `-Z autodiff` rustc flag (which can be passed to cargo via `RUSTFLAGS`), which allow changing the behaviour of Enzyme, without recompiling rustc. We currently support the following values for `autodiff`. + +### Debug Flags + +```text +PrintTA // Print TypeAnalysis information +PrintAA // Print ActivityAnalysis information +Print // Print differentiated functions while they are being generated and optimized +PrintPerf // Print AD related Performance warnings +PrintModBefore // Print the whole LLVM-IR module directly before running AD +PrintModAfter // Print the whole LLVM-IR module after running AD, before optimizations +PrintModFinal // Print the whole LLVM-IR module after running optimizations and AD +LooseTypes // Risk incorrect derivatives instead of aborting when missing Type Info +``` + +
+`LooseTypes` is often helpful to get rid of Enzyme errors stating `Can not deduce type of ` and to be able to run some code. But please keep in mind that this flag absolutely has the chance to cause incorrect gradients. Even worse, the gradients might be correct for certain input values, but not for others. So please create issues about such bugs and only use this flag temporarily while you wait for your bug to be fixed. +
+ +### Benchmark flags + +For performance experiments and benchmarking we also support + +```text +NoPostopt // We won't optimize the LLVM-IR Module after AD +RuntimeActivity // Enables the runtime activity feature from Enzyme +Inline // Instructs Enzyme to maximize inlining as far as possible, beyond LLVM's default +``` + +You can combine multiple `autodiff` values using a comma as separator: + +```bash +RUSTFLAGS="-Z autodiff=Enable,LooseTypes,PrintPerf" cargo +enzyme build +``` + +Using `-Zautodiff=Enable` will allow using autodiff and update your normal rustc compilation pipeline: + +1. Run your selected compilation pipeline. If you selected a release build, we will disable vectorization and loop unrolling. +2. Differentiate your functions. +3. Run your selected compilation pipeline again on the whole module. This time we do not disable vectorization or loop unrolling. diff --git a/src/autodiff/internals.md b/src/autodiff/internals.md new file mode 100644 index 00000000..0093ef04 --- /dev/null +++ b/src/autodiff/internals.md @@ -0,0 +1,27 @@ +The `std::autodiff` module in Rust allows differentiable programming: + +```rust +#![feature(autodiff)] +use std::autodiff::autodiff; + +// f(x) = x * x, f'(x) = 2.0 * x +// bar therefore returns (x * x, 2.0 * x) +#[autodiff(bar, Reverse, Active, Active)] +fn foo(x: f32) -> f32 { x * x } + +fn main() { + assert_eq!(bar(3.0, 1.0), (9.0, 6.0)); + assert_eq!(bar(4.0, 1.0), (16.0, 8.0)); +} +``` + +The detailed documentation for the `std::autodiff` module is available at [std::autodiff](https://doc.rust-lang.org/std/autodiff/index.html). + +Differentiable programing is used in various fields like numerical computing, [solid mechanics][ratel], [computational chemistry][molpipx], [fluid dynamics][waterlily] or for Neural Network training via Backpropagation, [ODE solver][diffsol], [differentiable rendering][libigl], [quantum computing][catalyst], and climate simulations. + +[ratel]: https://gitlab.com/micromorph/ratel +[molpipx]: https://arxiv.org/abs/2411.17011v +[waterlily]: https://github.com/WaterLily-jl/WaterLily.jl +[diffsol]: https://github.com/martinjrobins/diffsol +[libigl]: https://github.com/alecjacobson/libigl-enzyme-example?tab=readme-ov-file#run +[catalyst]: https://github.com/PennyLaneAI/catalyst diff --git a/src/autodiff/limitations.md b/src/autodiff/limitations.md new file mode 100644 index 00000000..90afbd51 --- /dev/null +++ b/src/autodiff/limitations.md @@ -0,0 +1,27 @@ +# Current limitations + +## Safety and Soundness + +Enzyme currently assumes that the user passes shadow arguments (`dx`, `dy`, ...) of appropriate size. Under Reverse Mode, we additionally assume that shadow arguments are mutable. In Reverse Mode we adjust the outermost pointer or reference to be mutable. Therefore `&f32` will receive the shadow type `&mut f32`. However, we do not check length for other types than slices (e.g. enums, Vec). We also do not enforce mutability of inner references, but will warn if we recognize them. We do intend to add additional checks over time. + +## ABI adjustments + +In some cases, a function parameter might get lowered in a way that we currently don't handle correctly, leading to a compile time type mismatch in the `rustc_codegen_llvm` backend. Here are some [examples](https://github.com/EnzymeAD/rust/issues/105). + +## Compile Times + +Enzyme will often achieve excellent runtime performance, but might increase your compile time by a large factor. For Rust, we already have made significant improvements and have a list of further improvements planed - please reach out if you have time to help here. + +### Type Analysis + +Most of the times, Type Analysis (TA) is the reason of large (>5x) compile time increases when using Enzyme. This poster explains why we need to run Type Analysis in the bottom left part: [Poster Link](https://c.wsmoses.com/posters/Enzyme-llvmdev.pdf). + +We intend to increase the number of locations where we pass down Type information based on Rust types, which in turn will reduce the number of locations where Enzyme has to run Type Analysis, which will help compile times. + +### Duplicated Optimizations + +The key reason for Enzyme offering often excellent performance is that Enzyme differentiates already optimized LLVM-IR. However, we also (have to) run LLVM's optimization pipeline after differentiating, to make sure that the code which Enzyme generates is optimized properly. As a result you should have excellent runtime performance (please fill an issue if not), but at a compile time cost for running optimizations twice. + +### Fat-LTO + +The usage of `#[autodiff(...)]` currently requires compiling your project with Fat-LTO. We technically only need LTO if the function being differentiated calls functions in other compilation units. Therefore, other solutions are possible, but this is the most simple one to get started. From bde086e3fa6e5d19340b9379936d008968cf932c Mon Sep 17 00:00:00 2001 From: Jieyou Xu Date: Thu, 17 Apr 2025 17:04:30 +0800 Subject: [PATCH 097/107] rustc-dev-guide: document `//@ ignore-auxiliary` --- src/tests/best-practices.md | 2 ++ src/tests/directives.md | 3 +++ 2 files changed, 5 insertions(+) diff --git a/src/tests/best-practices.md b/src/tests/best-practices.md index 6905ee13..2bdc7f3a 100644 --- a/src/tests/best-practices.md +++ b/src/tests/best-practices.md @@ -175,6 +175,8 @@ See [compiletest directives] for a listing of directives. - For `ignore-*`/`needs-*`/`only-*` directives, unless extremely obvious, provide a brief remark on why the directive is needed. E.g. `"//@ ignore-wasi (wasi codegens the main symbol differently)"`. +- When using `//@ ignore-auxiliary`, specify the corresponding main test files, + e.g. ``//@ ignore-auxiliary (used by `./foo.rs`)``. ## FileCheck best practices diff --git a/src/tests/directives.md b/src/tests/directives.md index 0aad8be9..dae659e6 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -124,6 +124,9 @@ means the test won't be compiled or run. * `ignore-X` where `X` is a target detail or other criteria on which to ignore the test (see below) * `only-X` is like `ignore-X`, but will *only* run the test on that target or stage +* `ignore-auxiliary` is intended for files that *participate* in one or more other + main test files but that `compiletest` should not try to build the file itself. + Please backlink to which main test is actually using the auxiliary file. * `ignore-test` always ignores the test. This can be used to temporarily disable a test if it is currently not working, but you want to keep it in tree to re-enable it later. From 23c4b36751ea7450950854b47ffd269a96db69d9 Mon Sep 17 00:00:00 2001 From: Manuel Drehwald Date: Thu, 17 Apr 2025 20:52:07 -0400 Subject: [PATCH 098/107] upstream autodiff build instructions --- src/SUMMARY.md | 1 + src/autodiff/installation.md | 86 ++++++++++++++++++++++++++++++++++++ 2 files changed, 87 insertions(+) create mode 100644 src/autodiff/installation.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 6526e2f0..90fcde07 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -100,6 +100,7 @@ - [Search](./rustdoc-internals/search.md) - [The `rustdoc` test suite](./rustdoc-internals/rustdoc-test-suite.md) - [Autodiff internals](./autodiff/internals.md) + - [Installation](./autodiff/installation.md) - [How to debug](./autodiff/debugging.md) - [Autodiff flags](./autodiff/flags.md) - [Current limitations](./autodiff/limitations.md) diff --git a/src/autodiff/installation.md b/src/autodiff/installation.md new file mode 100644 index 00000000..dbea9dbe --- /dev/null +++ b/src/autodiff/installation.md @@ -0,0 +1,86 @@ +# Installation + +In the near future, `std::autodiff` should become available in nightly builds for users. As a contribute however, you will still need to build rustc from source. Please be aware that the msvc target is not supported at the moment, all other tier 1 targets should work. Please open an issue if you encounter any problems on a supported tier 1 target, or if you succesfully build this project on a tier2/tier3 target. + +## Build instructions + +First you need to clone and configure the Rust repository: +```bash +git clone --depth=1 git@github.com:rust-lang/rust.git +cd rust +./configure --enable-llvm-link-shared --enable-llvm-plugins --enable-llvm-enzyme --release-channel=nightly --enable-llvm-assertions --enable-clang --enable-lld --enable-option-checking --enable-ninja --disable-docs +``` + +Afterwards you can build rustc using: +```bash +./x.py build --stage 1 library +``` + +Afterwards rustc toolchain link will allow you to use it through cargo: +``` +rustup toolchain link enzyme build/host/stage1 +rustup toolchain install nightly # enables -Z unstable-options +``` + +You can then run our test cases: + +```bash +./x.py test --stage 1 library tests/ui/autodiff +./x.py test --stage 1 library tests/codegen/autodiff +./x.py test --stage 1 library tests/pretty/autodiff* +``` + +Autodiff is still experimental, so if you want to use it in your own projects, you will need to add `lto="fat"` to your Cargo.toml +and use `RUSTFLAGS="-Zautodiff=Enable" cargo +enzyme` instead of `cargo` or `cargo +nightly`. + +## Compiler Explorer and dist builds + +Our compiler explorer instance can be updated to a newer rustc in a similar way. First, prepare a docker instance. +```bash +docker run -it ubuntu:22.04 +export CC=clang CXX=clang++ +apt update +apt install wget vim python3 git curl libssl-dev pkg-config lld ninja-build cmake clang build-essential +``` +Then build rustc in a slightly altered way: +```bash +git clone --depth=1 https://github.com/EnzymeAD/rust.git +cd rust +./configure --enable-llvm-link-shared --enable-llvm-plugins --enable-llvm-enzyme --release-channel=nightly --enable-llvm-assertions --enable-clang --enable-lld --enable-option-checking --enable-ninja --disable-docs +./x dist +``` +We then copy the tarball to our host. The dockerid is the newest entry under `docker ps -a`. +```bash +docker cp :/rust/build/dist/rust-nightly-x86_64-unknown-linux-gnu.tar.gz rust-nightly-x86_64-unknown-linux-gnu.tar.gz +``` +Afterwards we can create a new (pre-release) tag on the EnzymeAD/rust repository and make a PR against the EnzymeAD/enzyme-explorer repository to update the tag. +Remember to ping `tgymnich` on the PR to run his update script. + + +## Build instruction for Enzyme itself + +Following the Rust build instruction above will build LLVMEnzyme, LLDEnzyme, and ClangEnzyme along with the Rust compiler. +We recommend that approach, if you just want to use any of them and have no experience with cmake. +However, if you prefer to just build Enzyme without Rust, then these instructions might help. + +```bash +git clone --depth=1 git@github.com:llvm/llvm-project.git +cd llvm-project +mkdir build +cd build +cmake -G Ninja ../llvm -DLLVM_TARGETS_TO_BUILD="host" -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_PROJECTS="clang;lld" -DLLVM_ENABLE_RUNTIMES="openmp" -DLLVM_ENABLE_PLUGINS=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=. +ninja +ninja install +``` +This gives you a working LLVM build, now we can continue with building Enzyme. +Leave the `llvm-project` folder, and execute the following commands: +```bash +git clone git@github.com:EnzymeAD/Enzyme.git +cd Enzyme/enzyme +mkdir build +cd build +cmake .. -G Ninja -DLLVM_DIR=/llvm-project/build/lib/cmake/llvm/ -DLLVM_EXTERNAL_LIT=/llvm-project/llvm/utils/lit/lit.py -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXPORT_COMPILE_COMMANDS=YES -DBUILD_SHARED_LIBS=ON +ninja +``` +This will build Enzyme, and you can find it in `Enzyme/enzyme/build/lib/Enzyme.so`. (Endings might differ based on your OS). + From 1bc859ed76451c8f36165941713c46f09defda75 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 19 Apr 2025 13:08:43 +0200 Subject: [PATCH 099/107] readme: be copy-paste friendly --- README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 6a25a91f..9f468b93 100644 --- a/README.md +++ b/README.md @@ -43,13 +43,13 @@ rustdocs][rustdocs]. To build a local static HTML site, install [`mdbook`](https://github.com/rust-lang/mdBook) with: ``` -> cargo install mdbook mdbook-linkcheck2 mdbook-toc mdbook-mermaid +cargo install mdbook mdbook-linkcheck2 mdbook-toc mdbook-mermaid ``` and execute the following command in the root of the repository: ``` -> mdbook build --open +mdbook build --open ``` The build files are found in the `book/html` directory. @@ -61,8 +61,8 @@ checking is **not** run by default locally, though it is in CI. To enable it locally, set the environment variable `ENABLE_LINKCHECK=1` like in the following example. -```console -$ ENABLE_LINKCHECK=1 mdbook serve +``` +ENABLE_LINKCHECK=1 mdbook serve ``` ### Table of Contents @@ -86,14 +86,14 @@ Older versions of `josh-proxy` may not round trip commits losslessly so it is im 1) Checkout a new branch that will be used to create a PR into `rust-lang/rustc-dev-guide` 2) Run the pull command ``` - $ cargo run --manifest-path josh-sync/Cargo.toml rustc-pull + cargo run --manifest-path josh-sync/Cargo.toml rustc-pull ``` 3) Push the branch to your fork and create a PR into `rustc-dev-guide` ### Push changes from this repository into `rust-lang/rust` 1) Run the push command to create a branch named `` in a `rustc` fork under the `` account ``` - $ cargo run --manifest-path josh-sync/Cargo.toml rustc-push + cargo run --manifest-path josh-sync/Cargo.toml rustc-push ``` 2) Create a PR from `` into `rust-lang/rust` @@ -106,5 +106,5 @@ You may observe "Nothing to pull" even if you *know* rustc-pull has something to To minimize the likelihood of this happening, you may wish to keep a separate *minimal* git config that *only* has `[user]` entries from global git config, then repoint system git to use the minimal git config instead. E.g. ``` -$ GIT_CONFIG_GLOBAL=/path/to/minimal/gitconfig GIT_CONFIG_SYSTEM='' cargo +stable run --manifest-path josh-sync/Cargo.toml -- rustc-pull +GIT_CONFIG_GLOBAL=/path/to/minimal/gitconfig GIT_CONFIG_SYSTEM='' cargo +stable run --manifest-path josh-sync/Cargo.toml -- rustc-pull ``` From 85a22b893cea20cb863dfae8be20661b1e3f43cf Mon Sep 17 00:00:00 2001 From: The rustc-dev-guide Cronjob Bot Date: Sat, 19 Apr 2025 13:53:05 +0000 Subject: [PATCH 100/107] Preparing for merge from rustc --- rust-version | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust-version b/rust-version index a6f29510..67de4a98 100644 --- a/rust-version +++ b/rust-version @@ -1 +1 @@ -25a615bf829b9f6d6f22da537e3851043f92e5f2 +a7c39b68616668a45f0afd62849a1da7c8ad2516 From 3a23ec25298731cabfca2df71521f59d9e071fca Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 19 Apr 2025 16:04:22 +0200 Subject: [PATCH 101/107] fix broken link --- src/hir.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/hir.md b/src/hir.md index 65779f31..0c1c9941 100644 --- a/src/hir.md +++ b/src/hir.md @@ -144,7 +144,7 @@ that `n` must be some HIR expression, you can do Finally, you can find the parents of nodes, via calls like [`tcx.parent_hir_node(n)`][parent_hir_node]. -[get_parent_item]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.parent_hir_node +[parent_hir_node]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/struct.TyCtxt.html#method.parent_hir_node ## HIR Bodies From 4aa47cdf024528e8e2f14176d3897db9a0fe8264 Mon Sep 17 00:00:00 2001 From: Folkert de Vries Date: Sat, 19 Apr 2025 15:12:11 +0200 Subject: [PATCH 102/107] document `#[cfg(bootstrap)]` in dependencies --- src/SUMMARY.md | 1 + .../bootstrap-in-dependencies.md | 53 +++++++++++++++++++ 2 files changed, 54 insertions(+) create mode 100644 src/building/bootstrapping/bootstrap-in-dependencies.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 90fcde07..68112d06 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -81,6 +81,7 @@ - [How Bootstrap does it](./building/bootstrapping/how-bootstrap-does-it.md) - [Writing tools in Bootstrap](./building/bootstrapping/writing-tools-in-bootstrap.md) - [Debugging bootstrap](./building/bootstrapping/debugging-bootstrap.md) +- [cfg(bootstrap) in dependencies](./building/bootstrapping/bootstrap-in-dependencies.md) # High-level Compiler Architecture diff --git a/src/building/bootstrapping/bootstrap-in-dependencies.md b/src/building/bootstrapping/bootstrap-in-dependencies.md new file mode 100644 index 00000000..68c5c238 --- /dev/null +++ b/src/building/bootstrapping/bootstrap-in-dependencies.md @@ -0,0 +1,53 @@ +# `cfg(bootstrap)` in compiler dependencies + +The rust compiler uses some external crates that can run into cyclic dependencies with the compiler itself: the compiler needs an updated crate to build, but the crate needs an updated compiler. This page describes how `#[cfg(bootstrap)]` can be used to break this cycle. + +## Enabling `#[cfg(bootstrap)]` + +Usually the use of `#[cfg(bootstrap)]` in an external crate causes a warning: + +``` +warning: unexpected `cfg` condition name: `bootstrap` + --> src/main.rs:1:7 + | +1 | #[cfg(bootstrap)] + | ^^^^^^^^^ + | + = help: expected names are: `docsrs`, `feature`, and `test` and 31 more + = help: consider using a Cargo feature instead + = help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint: + [lints.rust] + unexpected_cfgs = { level = "warn", check-cfg = ['cfg(bootstrap)'] } + = help: or consider adding `println!("cargo::rustc-check-cfg=cfg(bootstrap)");` to the top of the `build.rs` + = note: see for more information about checking conditional configuration + = note: `#[warn(unexpected_cfgs)]` on by default +``` + +This warning can be silenced by adding these lines to the project's `Cargo.toml`: + +```toml +[lints.rust] +unexpected_cfgs = { level = "warn", check-cfg = ['cfg(bootstrap)'] } +``` + +Now `#[cfg(bootstrap)]` can be used in the crate just like it can be in the compiler: when the bootstrap compiler is used, code annotated with `#[cfg(bootstrap)]` is compiled, otherwise code annotated with `#[cfg(not(bootstrap))]` is compiled. + +## The update dance + +As a concrete example we'll use a change where the `#[naked]` attribute was made into an unsafe attribute, which caused a cyclic dependency with the `compiler-builtins` crate. + +### Step 1: accept the new behavior in the compiler ([#139797](https://github.com/rust-lang/rust/pull/139797)) + +In this example it is possible to accept both the old and new behavior at the same time by disabling an error. + +### Step 2: update the crate ([#821](https://github.com/rust-lang/compiler-builtins/pull/821)) + +Now in the crate, use `#[cfg(bootstrap)]` to use the old behavior, or `#[cfg(not(bootstrap))]` to use the new behavior. + +### Step 3: update the crate version used by the compiler ([#139934](https://github.com/rust-lang/rust/pull/139934)) + +For `compiler-builtins` this meant a version bump, in other cases it may be a git submodule update. + +### Step 4: remove the old behavior from the compiler ([#139753](https://github.com/rust-lang/rust/pull/139753)) + +The updated crate can now be used. In this example that meant that the old behavior could be removed. From 58d5664e4e65b44726114c2218f5cded00a7193a Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 19 Apr 2025 17:51:41 +0200 Subject: [PATCH 103/107] needed a stronger pause --- src/tests/ui.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/tests/ui.md b/src/tests/ui.md index 7c4e2a0f..f7857ae6 100644 --- a/src/tests/ui.md +++ b/src/tests/ui.md @@ -337,7 +337,8 @@ possible, including `//~?` annotations for diagnostics without span. If the compile time output is target dependent or too verbose, use directive `//@ dont-require-annotations: ` to make the line annotation checking -non-exhaustive, some of the compiler messages can stay uncovered by annotations in this mode. +non-exhaustive. +Some of the compiler messages can stay uncovered by annotations in this mode. For checking runtime output `//@ check-run-results` may be preferable. From 0e5847e7c8e5ce5051ca1584db394557316eae94 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 19 Apr 2025 17:57:56 +0200 Subject: [PATCH 104/107] fix grammar --- src/tests/ui.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tests/ui.md b/src/tests/ui.md index f7857ae6..06b13a32 100644 --- a/src/tests/ui.md +++ b/src/tests/ui.md @@ -333,7 +333,7 @@ fn main() { Use of `error-pattern` is not recommended in general. For strict testing of compile time output, try to use the line annotations `//~` as much as -possible, including `//~?` annotations for diagnostics without span. +possible, including `//~?` annotations for diagnostics without spans. If the compile time output is target dependent or too verbose, use directive `//@ dont-require-annotations: ` to make the line annotation checking From 43a4aedbbe3332a3490aed3018808aaa2f7e4535 Mon Sep 17 00:00:00 2001 From: Tshepang Mbambo Date: Sat, 19 Apr 2025 17:56:23 +0200 Subject: [PATCH 105/107] improve readability by adding pauses --- src/tests/ui.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/tests/ui.md b/src/tests/ui.md index 06b13a32..6232c8bc 100644 --- a/src/tests/ui.md +++ b/src/tests/ui.md @@ -303,7 +303,7 @@ It should be preferred to using `error-pattern`, which is imprecise and non-exha ### `error-pattern` The `error-pattern` [directive](directives.md) can be used for runtime messages, which don't -have a specific span, or in exceptional cases for compile time messages. +have a specific span, or in exceptional cases, for compile time messages. Let's think about this test: @@ -316,7 +316,7 @@ fn main() { } ``` -We want to ensure this shows "index out of bounds" but we cannot use the `ERROR` +We want to ensure this shows "index out of bounds", but we cannot use the `ERROR` annotation since the runtime error doesn't have any span. Then it's time to use the `error-pattern` directive: @@ -340,12 +340,12 @@ If the compile time output is target dependent or too verbose, use directive non-exhaustive. Some of the compiler messages can stay uncovered by annotations in this mode. -For checking runtime output `//@ check-run-results` may be preferable. +For checking runtime output, `//@ check-run-results` may be preferable. Only use `error-pattern` if none of the above works. Line annotations `//~` are still checked in tests using `error-pattern`. -In exceptional cases use `//@ compile-flags: --error-format=human` to opt out of these checks. +In exceptional cases, use `//@ compile-flags: --error-format=human` to opt out of these checks. ### Diagnostic kinds (error levels) From d5cf5a36511bf129d011abb0ad247f3d36ec653c Mon Sep 17 00:00:00 2001 From: The rustc-dev-guide Cronjob Bot Date: Mon, 21 Apr 2025 04:03:02 +0000 Subject: [PATCH 106/107] Preparing for merge from rustc --- rust-version | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust-version b/rust-version index 67de4a98..dc52e092 100644 --- a/rust-version +++ b/rust-version @@ -1 +1 @@ -a7c39b68616668a45f0afd62849a1da7c8ad2516 +b8005bff3248cfc6e327faf4fa631ac49bb49ba9 From dd639a47fe6f75dfc4dbf7de04f0bdbe049000ab Mon Sep 17 00:00:00 2001 From: Jieyou Xu Date: Wed, 23 Apr 2025 17:18:20 +0800 Subject: [PATCH 107/107] rustc-dev-guide: document that `//@ add-core-stubs` imply `-Cforce-unwind-tables=yes` --- src/tests/minicore.md | 38 +++++++++++++++++++++++++------------- 1 file changed, 25 insertions(+), 13 deletions(-) diff --git a/src/tests/minicore.md b/src/tests/minicore.md index e4853b6d..507b259e 100644 --- a/src/tests/minicore.md +++ b/src/tests/minicore.md @@ -6,25 +6,37 @@ ui/codegen/assembly test suites. It provides `core` stubs for tests that need to build for cross-compiled targets but do not need/want to run. -A test can use [`minicore`] by specifying the `//@ add-core-stubs` directive. -Then, mark the test with `#![feature(no_core)]` + `#![no_std]` + `#![no_core]`. -Due to Edition 2015 extern prelude rules, you will probably need to declare -`minicore` as an extern crate. - -Due to the `no_std` + `no_core` nature of these tests, `//@ add-core-stubs` -implies and requires that the test will be built with `-C panic=abort`. -Unwinding panics are not supported. - -If you find a `core` item to be missing from the [`minicore`] stub, consider -adding it to the test auxiliary if it's likely to be used or is already needed -by more than one test. -
Please note that [`minicore`] is only intended for `core` items, and explicitly **not** `std` or `alloc` items because `core` items are applicable to a wider range of tests.
+A test can use [`minicore`] by specifying the `//@ add-core-stubs` directive. +Then, mark the test with `#![feature(no_core)]` + `#![no_std]` + `#![no_core]`. +Due to Edition 2015 extern prelude rules, you will probably need to declare +`minicore` as an extern crate. + +## Implied compiler flags + +Due to the `no_std` + `no_core` nature of these tests, `//@ add-core-stubs` +implies and requires that the test will be built with `-C panic=abort`. +**Unwinding panics are not supported.** + +Tests will also be built with `-C force-unwind-tables=yes` to preserve CFI +directives in assembly tests. + +TL;DR: `//@ add-core-stubs` implies two compiler flags: + +1. `-C panic=abort` +2. `-C force-unwind-tables=yes` + +## Adding more `core` stubs + +If you find a `core` item to be missing from the [`minicore`] stub, consider +adding it to the test auxiliary if it's likely to be used or is already needed +by more than one test. + ## Example codegen test that uses `minicore` ```rust,no_run