initial instructions for gpu offload

rust-version

@@ -1 +1 @@
-14346303d760027e53214e705109a62c0f00b214
+99e7c15e81385b38a8186b51edc4577d5d7b5bdd

src/SUMMARY.md

@@ -63,8 +63,10 @@
 - [Notification groups](notification-groups/about.md)
     - [Apple](notification-groups/apple.md)
     - [ARM](notification-groups/arm.md)
+    - [Cleanup Crew](notification-groups/cleanup-crew.md)
     - [Emscripten](notification-groups/emscripten.md)
     - [Fuchsia](notification-groups/fuchsia.md)
+    - [LLVM](notification-groups/llvm.md)
     - [RISC-V](notification-groups/risc-v.md)
     - [Rust for Linux](notification-groups/rust-for-linux.md)
     - [WASI](notification-groups/wasi.md)
@@ -99,10 +101,6 @@
 - [Rustdoc internals](./rustdoc-internals.md)
     - [Search](./rustdoc-internals/search.md)
 	- [The `rustdoc` test suite](./rustdoc-internals/rustdoc-test-suite.md)
-	- [The `rustdoc-gui` test suite](./rustdoc-internals/rustdoc-gui-test-suite.md)
-	- [The `rustdoc-json` test suite](./rustdoc-internals/rustdoc-json-test-suite.md)
-- [GPU offload internals](./offload/internals.md)
-    - [Installation](./offload/installation.md)
 - [Autodiff internals](./autodiff/internals.md)
     - [Installation](./autodiff/installation.md)
     - [How to debug](./autodiff/debugging.md)
@@ -123,9 +121,8 @@
     - [Feature gate checking](./feature-gate-ck.md)
     - [Lang Items](./lang-items.md)
 - [The HIR (High-level IR)](./hir.md)
-    - [Lowering AST to HIR](./hir/lowering.md)
-    - [Ambig/Unambig Types and Consts](./hir/ambig-unambig-ty-and-consts.md)
-    - [Debugging](./hir/debugging.md)
+    - [Lowering AST to HIR](./ast-lowering.md)
+    - [Debugging](./hir-debugging.md)
 - [The THIR (Typed High-level IR)](./thir.md)
 - [The MIR (Mid-level IR)](./mir/index.md)
     - [MIR construction](./mir/construction.md)
@@ -184,7 +181,7 @@
         - [Significant changes and quirks](./solve/significant-changes.md)
     - [`Unsize` and `CoerceUnsized` traits](./traits/unsize.md)
 - [Type checking](./type-checking.md)
-    - [Method lookup](./method-lookup.md)
+    - [Method Lookup](./method-lookup.md)
     - [Variance](./variance.md)
     - [Coherence checking](./coherence.md)
     - [Opaque types](./opaque-types-type-alias-impl-trait.md)
@@ -192,7 +189,7 @@
         - [Return Position Impl Trait In Trait](./return-position-impl-trait-in-trait.md)
         - [Region inference restrictions][opaque-infer]
 - [Const condition checking](./effects.md)
-- [Pattern and exhaustiveness checking](./pat-exhaustive-checking.md)
+- [Pattern and Exhaustiveness Checking](./pat-exhaustive-checking.md)
 - [Unsafety checking](./unsafety-checking.md)
 - [MIR dataflow](./mir/dataflow.md)
 - [Drop elaboration](./mir/drop-elaboration.md)
@@ -212,7 +209,7 @@
 - [Closure capture inference](./closure.md)
 - [Async closures/"coroutine-closures"](coroutine-closures.md)
 
-# MIR to binaries
+# MIR to Binaries
 
 - [Prologue](./part-5-intro.md)
 - [MIR optimizations](./mir/optimizations.md)
@@ -221,15 +218,15 @@
     - [Interpreter](./const-eval/interpret.md)
 - [Monomorphization](./backend/monomorph.md)
 - [Lowering MIR](./backend/lowering-mir.md)
-- [Code generation](./backend/codegen.md)
+- [Code Generation](./backend/codegen.md)
     - [Updating LLVM](./backend/updating-llvm.md)
     - [Debugging LLVM](./backend/debugging.md)
     - [Backend Agnostic Codegen](./backend/backend-agnostic.md)
-    - [Implicit caller location](./backend/implicit-caller-location.md)
-- [Libraries and metadata](./backend/libs-and-metadata.md)
-- [Profile-guided optimization](./profile-guided-optimization.md)
-- [LLVM source-based code coverage](./llvm-coverage-instrumentation.md)
-- [Sanitizers support](./sanitizers.md)
+    - [Implicit Caller Location](./backend/implicit-caller-location.md)
+- [Libraries and Metadata](./backend/libs-and-metadata.md)
+- [Profile-guided Optimization](./profile-guided-optimization.md)
+- [LLVM Source-Based Code Coverage](./llvm-coverage-instrumentation.md)
+- [Sanitizers Support](./sanitizers.md)
 - [Debugging support in the Rust compiler](./debugging-support-in-rustc.md)
 
 ---

src/ast-lowering.md

@@ -1,6 +1,6 @@
 # AST lowering
 
-The AST lowering step converts AST to [HIR](../hir.md).
+The AST lowering step converts AST to [HIR](hir.html).
 This means many structures are removed if they are irrelevant
 for type analysis or similar syntax agnostic analyses. Examples
 of such structures include but are not limited to

src/backend/implicit-caller-location.md

@@ -1,4 +1,4 @@
-# Implicit caller location
+# Implicit Caller Location
 
 <!-- toc -->
 
@@ -8,7 +8,7 @@ adds the [`#[track_caller]`][attr-reference] attribute for functions, the
 [`caller_location`][intrinsic] intrinsic, and the stabilization-friendly
 [`core::panic::Location::caller`][wrapper] wrapper.
 
-## Motivating example
+## Motivating Example
 
 Take this example program:
 
@@ -39,7 +39,7 @@ These error messages are achieved through a combination of changes to `panic!` i
 of `core::panic::Location::caller` and a number of `#[track_caller]` annotations in the standard
 library which propagate caller information.
 
-## Reading caller location
+## Reading Caller Location
 
 Previously, `panic!` made use of the `file!()`, `line!()`, and `column!()` macros to construct a
 [`Location`] pointing to where the panic occurred. These macros couldn't be given an overridden
@@ -51,7 +51,7 @@ was expanded. This function is itself annotated with `#[track_caller]` and wraps
 [`caller_location`][intrinsic] compiler intrinsic implemented by rustc. This intrinsic is easiest
 explained in terms of how it works in a `const` context.
 
-## Caller location in `const`
+## Caller Location in `const`
 
 There are two main phases to returning the caller location in a const context: walking up the stack
 to find the right location and allocating a const value to return.
@@ -138,7 +138,7 @@ fn main() {
 }
 ```
 
-### Dynamic dispatch
+### Dynamic Dispatch
 
 In codegen contexts we have to modify the callee ABI to pass this information down the stack, but
 the attribute expressly does *not* modify the type of the function. The ABI change must be
@@ -156,7 +156,7 @@ probably the best we can do without modifying fully-stabilized type signatures.
 > whether we'll be called in a const context (safe to ignore shim) or in a codegen context (unsafe
 > to ignore shim). Even if we did know, the results from const and codegen contexts must agree.
 
-## The attribute
+## The Attribute
 
 The `#[track_caller]` attribute is checked alongside other codegen attributes to ensure the
 function:

src/backend/libs-and-metadata.md

@@ -1,4 +1,4 @@
-# Libraries and metadata
+# Libraries and Metadata
 
 When the compiler sees a reference to an external crate, it needs to load some
 information about that crate. This chapter gives an overview of that process,

src/building/bootstrapping/debugging-bootstrap.md

@@ -55,7 +55,7 @@ Bootstrap will conditionally build `tracing` support and enable `tracing` output
 
 Example basic usage[^just-trace]:
 
-[^just-trace]: It is not recommended to use *just* `BOOTSTRAP_TRACING=TRACE` because that will dump *everything* at `TRACE` level, including logs intentionally gated behind custom targets as they are too verbose even for `TRACE` level by default.
+[^just-trace]: It is not recommend to use *just* `BOOTSTRAP_TRACING=TRACE` because that will dump *everything* at `TRACE` level, including logs intentionally gated behind custom targets as they are too verbose even for `TRACE` level by default.
 
 ```bash
 $ BOOTSTRAP_TRACING=bootstrap=TRACE ./x build library --stage 1

src/building/bootstrapping/what-bootstrapping-does.md

@@ -45,13 +45,13 @@ compiler.
 
 ```mermaid
 graph TD
-    s0c["stage0 compiler (1.86.0-beta.1)"]:::downloaded -->|A| s0l("stage0 std (1.86.0-beta.1)"):::downloaded;
+    s0c["stage0 compiler (1.63)"]:::downloaded -->|A| s0l("stage0 std (1.64)"):::with-s0c;
     s0c & s0l --- stepb[ ]:::empty;
-    stepb -->|B| s0ca["stage0 compiler artifacts (1.87.0-dev)"]:::with-s0c;
-    s0ca -->|copy| s1c["stage1 compiler (1.87.0-dev)"]:::with-s0c;
-    s1c -->|C| s1l("stage1 std (1.87.0-dev)"):::with-s1c;
+    stepb -->|B| s0ca["stage0 compiler artifacts (1.64)"]:::with-s0c;
+    s0ca -->|copy| s1c["stage1 compiler (1.64)"]:::with-s0c;
+    s1c -->|C| s1l("stage1 std (1.64)"):::with-s1c;
     s1c & s1l --- stepd[ ]:::empty;
-    stepd -->|D| s1ca["stage1 compiler artifacts (1.87.0-dev)"]:::with-s1c;
+    stepd -->|D| s1ca["stage1 compiler artifacts (1.64)"]:::with-s1c;
     s1ca -->|copy| s2c["stage2 compiler"]:::with-s1c;
 
     classDef empty width:0px,height:0px;
@@ -62,21 +62,19 @@ graph TD
 
 ### Stage 0: the pre-compiled compiler
 
-The stage0 compiler is by default the very recent _beta_ `rustc` compiler and its
+The stage0 compiler is usually the current _beta_ `rustc` compiler and its
 associated dynamic libraries, which `./x.py` will download for you. (You can
-also configure `./x.py` to change stage0 to something else.)
+also configure `./x.py` to use something else.)
 
-The precompiled stage0 compiler is then used only to compile [`src/bootstrap`] and [`compiler/rustc`]
-with precompiled stage0 std.
-
-Note that to build the stage1 compiler we use the precompiled stage0 compiler and std.
-Therefore, to use a compiler with a std that is freshly built from the tree, you need to
-build the stage2 compiler.
-
-There are two concepts at play here: a compiler (with its set of dependencies) and its
-'target' or 'object' libraries (`std` and `rustc`). Both are staged, but in a staggered manner.
+The stage0 compiler is then used only to compile [`src/bootstrap`],
+[`library/std`], and [`compiler/rustc`]. When assembling the libraries and
+binaries that will become the stage1 `rustc` compiler, the freshly compiled
+`std` and `rustc` are used. There are two concepts at play here: a compiler
+(with its set of dependencies) and its 'target' or 'object' libraries (`std` and
+`rustc`). Both are staged, but in a staggered manner.
 
 [`compiler/rustc`]: https://github.com/rust-lang/rust/tree/master/compiler/rustc
+[`library/std`]: https://github.com/rust-lang/rust/tree/master/library/std
 [`src/bootstrap`]: https://github.com/rust-lang/rust/tree/master/src/bootstrap
 
 ### Stage 1: from current code, by an earlier compiler
@@ -86,14 +84,16 @@ The rustc source code is then compiled with the `stage0` compiler to produce the
 
 ### Stage 2: the truly current compiler
 
-We then rebuild the compiler using `stage1` compiler with in-tree std to produce the `stage2`
+We then rebuild our `stage1` compiler with itself to produce the `stage2`
 compiler.
 
-The `stage1` compiler itself was built by precompiled `stage0` compiler and std
-and hence not by the source in your working directory. This means that the ABI
-generated by the `stage0` compiler may not match the ABI that would have been made
-by the `stage1` compiler, which can cause problems for dynamic libraries, tests
-and tools using `rustc_private`.
+In theory, the `stage1` compiler is functionally identical to the `stage2`
+compiler, but in practice there are subtle differences. In particular, the
+`stage1` compiler itself was built by `stage0` and hence not by the source in
+your working directory. This means that the ABI generated by the `stage0`
+compiler may not match the ABI that would have been made by the `stage1`
+compiler, which can cause problems for dynamic libraries, tests, and tools using
+`rustc_private`.
 
 Note that the `proc_macro` crate avoids this issue with a `C` FFI layer called
 `proc_macro::bridge`, allowing it to be used with `stage1`.
@@ -101,10 +101,9 @@ Note that the `proc_macro` crate avoids this issue with a `C` FFI layer called
 The `stage2` compiler is the one distributed with `rustup` and all other install
 methods. However, it takes a very long time to build because one must first
 build the new compiler with an older compiler and then use that to build the new
-compiler with itself.
-
-For development, you usually only want to use `--stage 1` flag to build things.
-See [Building the compiler](../how-to-build-and-run.html#building-the-compiler).
+compiler with itself. For development, you usually only want the `stage1`
+compiler, which you can build with `./x build library`. See [Building the
+compiler](../how-to-build-and-run.html#building-the-compiler).
 
 ### Stage 3: the same-result test
 
@@ -115,11 +114,10 @@ something has broken.
 ### Building the stages
 
 The script [`./x`] tries to be helpful and pick the stage you most likely meant
-for each subcommand. Here are some `x` commands with their default stages:
+for each subcommand. These defaults are as follows:
 
-- `check`: `--stage 1`
-- `clippy`: `--stage 1`
-- `doc`: `--stage 1`
+- `check`: `--stage 0`
+- `doc`: `--stage 0`
 - `build`: `--stage 1`
 - `test`: `--stage 1`
 - `dist`: `--stage 2`
@@ -193,8 +191,8 @@ include, but are not limited to:
   without building `rustc` from source ('build with `stage0`, then test the
   artifacts'). If you're working on the standard library, this is normally the
   test command you want.
-- `./x build --stage 0` means to build with the stage0 `rustc`.
-- `./x doc --stage 0` means to document using the stage0 `rustdoc`.
+- `./x build --stage 0` means to build with the beta `rustc`.
+- `./x doc --stage 0` means to document using the beta `rustdoc`.
 
 #### Examples of what *not* to do
 
@@ -210,6 +208,9 @@ include, but are not limited to:
 
 ### Building vs. running
 
+Note that `build --stage N compiler/rustc` **does not** build the stage N
+compiler: instead it builds the stage N+1 compiler _using_ the stage N compiler.
+
 In short, _stage 0 uses the `stage0` compiler to create `stage0` artifacts which
 will later be uplifted to be the stage1 compiler_.
 
@@ -267,6 +268,23 @@ However, when cross-compiling, `stage1` `std` will only run on the host. So the
 
 (See in the table how `stage2` only builds non-host `std` targets).
 
+### Why does only libstd use `cfg(bootstrap)`?
+
+For docs on `cfg(bootstrap)` itself, see [Complications of
+Bootstrapping](#complications-of-bootstrapping).
+
+The `rustc` generated by the `stage0` compiler is linked to the freshly-built
+`std`, which means that for the most part only `std` needs to be `cfg`-gated, so
+that `rustc` can use features added to `std` immediately after their addition,
+without need for them to get into the downloaded `beta` compiler.
+
+Note this is different from any other Rust program: `stage1` `rustc` is built by
+the _beta_ compiler, but using the _master_ version of `libstd`!
+
+The only time `rustc` uses `cfg(bootstrap)` is when it adds internal lints that
+use diagnostic items, or when it uses unstable library features that were
+recently changed.
+
 ### What is a 'sysroot'?
 
 When you build a project with `cargo`, the build artifacts for dependencies are
@@ -441,6 +459,7 @@ compiler itself uses to run. These aren't actually used by artifacts the new
 compiler generates. This step also copies the `rustc` and `rustdoc` binaries we
 generated into `build/$HOST/stage/bin`.
 
-The `stage1/bin/rustc` is a fully functional compiler built with stage0 (precompiled) compiler and std.
-To use a compiler built entirely from source with the in-tree compiler and std, you need to build the
-stage2 compiler, which is compiled using the stage1 (in-tree) compiler and std.
+The `stage1/bin/rustc` is a fully functional compiler, but it doesn't yet have
+any libraries to link built binaries or libraries to. The next 3 steps will
+provide those libraries for it; they are mostly equivalent to constructing the
+`stage1/bin` compiler so we don't go through them individually here.

src/building/how-to-build-and-run.md

@@ -217,6 +217,7 @@ probably the best "go to" command for building a local compiler:
 This may *look* like it only builds the standard library, but that is not the case.
 What this command does is the following:
 
+- Build `std` using the stage0 compiler
 - Build `rustc` using the stage0 compiler
   - This produces the stage1 compiler
 - Build `std` using the stage1 compiler
@@ -240,7 +241,8 @@ build. The **full** `rustc` build (what you get with `./x build
 --stage 2 compiler/rustc`) has quite a few more steps:
 
 - Build `rustc` with the stage1 compiler.
-  - The resulting compiler here is called the "stage2" compiler, which uses stage1 std from the previous command.
+  - The resulting compiler here is called the "stage2" compiler.
+- Build `std` with stage2 compiler.
 - Build `librustdoc` and a bunch of other things with the stage2 compiler.
 
 You almost never need to do this.
@@ -248,14 +250,14 @@ You almost never need to do this.
 ### Build specific components
 
 If you are working on the standard library, you probably don't need to build
-every other default component. Instead, you can build a specific component by
-providing its name, like this:
+the compiler unless you are planning to use a recently added nightly feature.
+Instead, you can just build using the bootstrap compiler.
 
 ```bash
-./x build --stage 1 library
+./x build --stage 0 library
 ```
 
-If you choose the `library` profile when running `x setup`, you can omit `--stage 1` (it's the
+If you choose the `library` profile when running `x setup`, you can omit `--stage 0` (it's the
 default).
 
 ## Creating a rustup toolchain
@@ -269,6 +271,7 @@ you will likely need to build at some point; for example, if you want
 to run the entire test suite).
 
 ```bash
+rustup toolchain link stage0 build/host/stage0-sysroot # beta compiler + stage0 std
 rustup toolchain link stage1 build/host/stage1
 rustup toolchain link stage2 build/host/stage2
 ```

src/building/new-target.md

@@ -85,7 +85,7 @@ Look for existing targets to use as examples.
 After adding your target to the `rustc_target` crate you may want to add
 `core`, `std`, ... with support for your new target. In that case you will
 probably need access to some `target_*` cfg. Unfortunately when building with
-stage0 (a precompiled compiler), you'll get an error that the target cfg is
+stage0 (the beta compiler), you'll get an error that the target cfg is
 unexpected because stage0 doesn't know about the new target specification and
 we pass `--check-cfg` in order to tell it to check.
 
@@ -174,8 +174,8 @@ compiler, you can use it instead of the JSON file for both arguments.
 ## Promoting a target from tier 2 (target) to tier 2 (host)
 
 There are two levels of tier 2 targets:
-- Targets that are only cross-compiled (`rustup target add`)
-- Targets that [have a native toolchain][tier2-native] (`rustup toolchain install`)
+  a) Targets that are only cross-compiled (`rustup target add`)
+  b) Targets that [have a native toolchain][tier2-native] (`rustup toolchain install`)
 
 [tier2-native]: https://doc.rust-lang.org/nightly/rustc/target-tier-policy.html#tier-2-with-host-tools
 

src/building/suggested.md

@@ -59,14 +59,6 @@ always overrides the inner ones.
 
 ## Configuring `rust-analyzer` for `rustc`
 
-### Checking the "library" tree
-
-Checking the "library" tree requires a stage1 compiler, which can be a heavy process on some computers.
-For this reason, bootstrap has a flag called `--skip-std-check-if-no-download-rustc` that skips checking the
-"library" tree if `rust.download-rustc` isn't available. If you want to avoid putting a heavy load on your computer
-with `rust-analyzer`, you can add the `--skip-std-check-if-no-download-rustc` flag to your `./x check` command in
-the `rust-analyzer` configuration.
-
 ### Project-local rust-analyzer setup
 
 `rust-analyzer` can help you check and format your code whenever you save a
@@ -315,15 +307,51 @@ lets you use `cargo fmt`.
 [the section on vscode]: suggested.md#configuring-rust-analyzer-for-rustc
 [the section on rustup]: how-to-build-and-run.md?highlight=rustup#creating-a-rustup-toolchain
 
-## Faster Builds with CI-rustc  
+## Faster builds with `--keep-stage`.
 
-If you are not working on the compiler, you often don't need to build the compiler tree.
-For example, you can skip building the compiler and only build the `library` tree or the
-tools under `src/tools`. To achieve that, you have to enable this by setting the `download-rustc`
-option in your configuration. This tells bootstrap to use the latest nightly compiler for `stage > 0`
-steps, meaning it will have two precompiled compilers: stage0 compiler and `download-rustc` compiler
-for `stage > 0` steps. This way, it will never need to build the in-tree compiler. As a result, your
-build time will be significantly reduced by not building the in-tree compiler.
+Sometimes just checking whether the compiler builds is not enough. A common
+example is that you need to add a `debug!` statement to inspect the value of
+some state or better understand the problem. In that case, you don't really need
+a full build. By bypassing bootstrap's cache invalidation, you can often get
+these builds to complete very fast (e.g., around 30 seconds). The only catch is
+this requires a bit of fudging and may produce compilers that don't work (but
+that is easily detected and fixed).
+
+The sequence of commands you want is as follows:
+
+- Initial build: `./x build library`
+  - As [documented previously], this will build a functional stage1 compiler as
+    part of running all stage0 commands (which include building a `std`
+    compatible with the stage1 compiler) as well as the first few steps of the
+    "stage 1 actions" up to "stage1 (sysroot stage1) builds std".
+- Subsequent builds: `./x build library --keep-stage 1`
+  - Note that we added the `--keep-stage 1` flag here
+
+[documented previously]: ./how-to-build-and-run.md#building-the-compiler
+
+As mentioned, the effect of `--keep-stage 1` is that we just _assume_ that the
+old standard library can be re-used. If you are editing the compiler, this is
+almost always true: you haven't changed the standard library, after all. But
+sometimes, it's not true: for example, if you are editing the "metadata" part of
+the compiler, which controls how the compiler encodes types and other states
+into the `rlib` files, or if you are editing things that wind up in the metadata
+(such as the definition of the MIR).
+
+**The TL;DR is that you might get weird behavior from a compile when using
+`--keep-stage 1`** -- for example, strange [ICEs](../appendix/glossary.html#ice)
+or other panics. In that case, you should simply remove the `--keep-stage 1`
+from the command and rebuild. That ought to fix the problem.
+
+You can also use `--keep-stage 1` when running tests. Something like this:
+
+- Initial test run: `./x test tests/ui`
+- Subsequent test run: `./x test tests/ui --keep-stage 1`
+
+### Iterating the standard library with `--keep-stage`
+
+If you are making changes to the standard library, you can use `./x build
+--keep-stage 0 library` to iteratively rebuild the standard library without
+rebuilding the compiler.
 
 ## Using incremental compilation
 

src/contributing.md

@@ -364,7 +364,7 @@ To find documentation-related issues, use the [A-docs label].
 
 You can find documentation style guidelines in [RFC 1574].
 
-To build the standard library documentation, use `x doc --stage 1 library --open`.
+To build the standard library documentation, use `x doc --stage 0 library --open`.
 To build the documentation for a book (e.g. the unstable book), use `x doc src/doc/unstable-book.`
 Results should appear in `build/host/doc`, as well as automatically open in your default browser.
 See [Building Documentation](./building/compiler-documenting.md#building-documentation) for more

src/diagnostics.md

@@ -553,7 +553,7 @@ compiler](#linting-early-in-the-compiler).
 
 
 [AST nodes]: the-parser.md
-[AST lowering]: ./hir/lowering.md
+[AST lowering]: ast-lowering.md
 [HIR nodes]: hir.md
 [MIR nodes]: mir/index.md
 [macro expansion]: macro-expansion.md

src/getting-started.md

@@ -158,6 +158,9 @@ feel comfortable jumping straight into the large `rust-lang/rust` codebase.
 The following tasks are doable without much background knowledge but are
 incredibly helpful:
 
+- [Cleanup crew][iceb]: find minimal reproductions of ICEs, bisect
+  regressions, etc. This is a way of helping that saves a ton of time for
+  others to fix an error later.
 - [Writing documentation][wd]: if you are feeling a bit more intrepid, you could try
   to read a part of the code and write doc comments for it. This will help you
   to learn some part of the compiler while also producing a useful artifact!
@@ -176,6 +179,7 @@ incredibly helpful:
 [users]: https://users.rust-lang.org/
 [so]: http://stackoverflow.com/questions/tagged/rust
 [community-library]: https://github.com/rust-lang/rfcs/labels/A-community-library
+[iceb]: ./notification-groups/cleanup-crew.md
 [wd]: ./contributing.md#writing-documentation
 [wg]: https://rust-lang.github.io/compiler-team/working-groups/
 [triage]: ./contributing.md#issue-triage

src/git.md

@@ -142,8 +142,7 @@ The most common cause is that you rebased after a change and ran `git add .` wit
 `x` to update the submodules.  Alternatively, you might have run `cargo fmt` instead of `x fmt`
 and modified files in a submodule, then committed the changes.
 
-To fix it, do the following things (if you changed a submodule other than cargo,
-replace `src/tools/cargo` with the path to that submodule):
+To fix it, do the following things:
 
 1. See which commit has the accidental changes: `git log --stat -n1 src/tools/cargo`
 2. Revert the changes to that commit: `git checkout <my-commit>~ src/tools/cargo`. Type `~`

src/hir.md

@@ -5,7 +5,7 @@
 The HIR – "High-Level Intermediate Representation" – is the primary IR used
 in most of rustc. It is a compiler-friendly representation of the abstract
 syntax tree (AST) that is generated after parsing, macro expansion, and name
-resolution (see [Lowering](./hir/lowering.md) for how the HIR is created).
+resolution (see [Lowering](./ast-lowering.html) for how the HIR is created).
 Many parts of HIR resemble Rust surface syntax quite closely, with
 the exception that some of Rust's expression forms have been desugared away.
 For example, `for` loops are converted into a `loop` and do not appear in

src/hir/ambig-unambig-ty-and-consts.md

@@ -1,63 +0,0 @@
-# Ambig/Unambig Types and Consts
-
-Types and Consts args in the HIR can be in two kinds of positions ambiguous (ambig) or unambiguous (unambig). Ambig positions are where
-it would be valid to parse either a type or a const, unambig positions are where only one kind would be valid to
-parse.
-
-```rust
-fn func<T, const N: usize>(arg: T) {
-    //                           ^ Unambig type position
-    let a: _ = arg; 
-    //     ^ Unambig type position
-
-    func::<T, N>(arg);
-    //     ^  ^
-    //     ^^^^ Ambig position 
-
-    let _: [u8; 10];
-    //      ^^  ^^ Unambig const position
-    //      ^^ Unambig type position
-}
-
-```
-
-Most types/consts in ambig positions are able to be disambiguated as either a type or const during parsing. Single segment paths are always represented as types in the AST but may get resolved to a const parameter during name resolution, then lowered to a const argument during ast-lowering. The only generic arguments which remain ambiguous after lowering are inferred generic arguments (`_`) in path segments. For example, in `Foo<_>` it is not clear whether the `_` argument is an inferred type argument, or an inferred const argument.
-
-In unambig positions, inferred arguments are represented with [`hir::TyKind::Infer`][ty_infer] or [`hir::ConstArgKind::Infer`][const_infer] depending on whether it is a type or const position respectively.
-In ambig positions, inferred arguments are represented with `hir::GenericArg::Infer`.
-
-A naive implementation of this would result in there being potentially 5 places where you might think an inferred type/const could be found in the HIR from looking at the structure of the HIR:
-1. In unambig type position as a `hir::TyKind::Infer`
-2. In unambig const arg position as a `hir::ConstArgKind::Infer`
-3. In an ambig position as a [`GenericArg::Type(TyKind::Infer)`][generic_arg_ty]
-4. In an ambig position as a [`GenericArg::Const(ConstArgKind::Infer)`][generic_arg_const]
-5. In an ambig position as a [`GenericArg::Infer`][generic_arg_infer]
-
-Note that places 3 and 4 would never actually be possible to encounter as we always lower to `GenericArg::Infer` in generic arg position. 
-
-This has a few failure modes:
-- People may write visitors which check for `GenericArg::Infer` but forget to check for `hir::TyKind/ConstArgKind::Infer`, only handling infers in ambig positions by accident.
-- People may write visitors which check for `hir::TyKind/ConstArgKind::Infer` but forget to check for `GenericArg::Infer`, only handling infers in unambig positions by accident.
-- People may write visitors which check for `GenerArg::Type/Const(TyKind/ConstArgKind::Infer)` and `GenerigArg::Infer`, not realising that we never represent inferred types/consts in ambig positions as a `GenericArg::Type/Const`.
-- People may write visitors which check for *only* `TyKind::Infer` and not `ConstArgKind::Infer` forgetting that there are also inferred const arguments (and vice versa).
-
-To make writing HIR visitors less error prone when caring about inferred types/consts we have a relatively complex system:
-
-1. We have different types in the compiler for when a type or const is in an unambig or ambig position, `hir::Ty<AmbigArg>` and `hir::Ty<()>`. [`AmbigArg`][ambig_arg] is an uninhabited type which we use in the `Infer` variant of `TyKind` and `ConstArgKind` to selectively "disable" it if we are in an ambig position.
-
-2. The [`visit_ty`][visit_ty] and [`visit_const_arg`][visit_const_arg] methods on HIR visitors only accept the ambig position versions of types/consts. Unambig types/consts are implicitly converted to ambig types/consts during the visiting process, with the `Infer` variant handled by a dedicated [`visit_infer`][visit_infer] method.
-
-This has a number of benefits:
-- It's clear that `GenericArg::Type/Const` cannot represent inferred type/const arguments
-- Implementors of `visit_ty` and `visit_const_arg` will never encounter inferred types/consts making it impossible to write a visitor that seems to work right but handles edge cases wrong 
-- The `visit_infer` method handles *all* cases of inferred type/consts in the HIR making it easy for visitors to handle inferred type/consts in one dedicated place and not forget cases
-
-[ty_infer]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/enum.TyKind.html#variant.Infer
-[const_infer]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/enum.ConstArgKind.html#variant.Infer
-[generic_arg_ty]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/enum.GenericArg.html#variant.Type
-[generic_arg_const]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/enum.GenericArg.html#variant.Const
-[generic_arg_infer]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/enum.GenericArg.html#variant.Infer
-[ambig_arg]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/enum.AmbigArg.html
-[visit_ty]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/intravisit/trait.Visitor.html#method.visit_ty
-[visit_const_arg]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/intravisit/trait.Visitor.html#method.visit_const_arg
-[visit_infer]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/intravisit/trait.Visitor.html#method.visit_infer

src/llvm-coverage-instrumentation.md

@@ -1,4 +1,4 @@
-# LLVM source-based code coverage
+# LLVM Source-Based Code Coverage
 
 <!-- toc -->
 

src/notification-groups/about.md

@@ -21,7 +21,9 @@ search for existing issues that haven't been claimed yet.
 Here's the list of the notification groups:
 - [Apple](./apple.md)
 - [ARM](./arm.md)
+- [Cleanup Crew](./cleanup-crew.md)
 - [Emscripten](./emscripten.md)
+- [LLVM Icebreakers](./llvm.md)
 - [RISC-V](./risc-v.md)
 - [WASI](./wasi.md)
 - [WebAssembly](./wasm.md)
@@ -62,7 +64,9 @@ Example PRs:
 
 * [Example of adding yourself to the Apple group.](https://github.com/rust-lang/team/pull/1434)
 * [Example of adding yourself to the ARM group.](https://github.com/rust-lang/team/pull/358)
+* [Example of adding yourself to the Cleanup Crew.](https://github.com/rust-lang/team/pull/221)
 * [Example of adding yourself to the Emscripten group.](https://github.com/rust-lang/team/pull/1579)
+* [Example of adding yourself to the LLVM group.](https://github.com/rust-lang/team/pull/140)
 * [Example of adding yourself to the RISC-V group.](https://github.com/rust-lang/team/pull/394)
 * [Example of adding yourself to the WASI group.](https://github.com/rust-lang/team/pull/1580)
 * [Example of adding yourself to the WebAssembly group.](https://github.com/rust-lang/team/pull/1581)
@@ -77,7 +81,9 @@ group. For example:
 ```text
 @rustbot ping apple
 @rustbot ping arm
+@rustbot ping cleanup-crew
 @rustbot ping emscripten
+@rustbot ping icebreakers-llvm
 @rustbot ping risc-v
 @rustbot ping wasi
 @rustbot ping wasm
@@ -86,12 +92,12 @@ group. For example:
 
 To make some commands shorter and easier to remember, there are aliases,
 defined in the [`triagebot.toml`] file. For example, all of these commands
-are equivalent and will ping the Apple group:
+are equivalent and will ping the Cleanup Crew:
 
 ```text
-@rustbot ping apple
-@rustbot ping macos
-@rustbot ping ios
+@rustbot ping cleanup
+@rustbot ping bisect
+@rustbot ping reduce
 ```
 
 Keep in mind that these aliases are meant to make humans' life easier.

src/notification-groups/cleanup-crew.md

@@ -0,0 +1,90 @@
+# Cleanup Crew
+
+**Github Label:** [ICEBreaker-Cleanup-Crew] <br>
+**Ping command:** `@rustbot ping cleanup-crew`
+
+[ICEBreaker-Cleanup-Crew]: https://github.com/rust-lang/rust/labels/ICEBreaker-Cleanup-Crew
+
+The "Cleanup Crew" are focused on improving bug reports. Specifically,
+the goal is to try to ensure that every bug report has all the
+information that will be needed for someone to fix it:
+
+* a minimal, standalone example that shows the problem
+* links to duplicates or related bugs
+* if the bug is a regression (something that used to work, but no longer does),
+  then a bisection to the PR or nightly that caused the regression
+
+This kind of cleanup is invaluable in getting bugs fixed. Better
+still, it can be done by anybody who knows Rust, without any
+particularly deep knowledge of the compiler.
+
+Let's look a bit at the workflow for doing "cleanup crew" actions.
+
+## Finding a minimal, standalone example
+
+Here the ultimate goal is to produce an example that reproduces the same
+problem but without relying on any external crates. Such a test ought to contain
+as little code as possible, as well. This will make it much easier to isolate the problem.
+
+However, even if the "ultimate minimal test" cannot be achieved, it's
+still useful to post incremental minimizations. For example, if you
+can eliminate some of the external dependencies, that is helpful, and
+so forth. 
+
+It's particularly useful to reduce to an example that works
+in the [Rust playground](https://play.rust-lang.org/), rather than
+requiring people to checkout a cargo build.
+
+There are many resources for how to produce minimized test cases. Here
+are a few:
+
+* The [rust-reduce](https://github.com/jethrogb/rust-reduce) tool can try to reduce
+  code automatically.
+  * The [C-reduce](https://github.com/csmith-project/creduce) tool also works
+    on Rust code, though it requires that you start from a single
+    file. (A post explaining how to do it can be found [here](https://insaneinside.net/2017/09/12/whole-crate-bug-reduction-with-creduce.html).)
+* pnkfelix's [Rust Bug Minimization Patterns] blog post
+  * This post focuses on "heavy bore" techniques, where you are
+    starting with a large, complex cargo project that you wish to
+    narrow down to something standalone.
+
+[Rust Bug Minimization Patterns]: http://blog.pnkfx.org/blog/2019/11/18/rust-bug-minimization-patterns/
+
+## Links to duplicate or related bugs
+
+If you are on the "Cleanup Crew", you will sometimes see multiple bug
+reports that seem very similar. You can link one to the other just by
+mentioning the other bug number in a Github comment. Sometimes it is
+useful to close duplicate bugs. But if you do so, you should always
+copy any test case from the bug you are closing to the other bug that
+remains open, as sometimes duplicate-looking bugs will expose
+different facets of the same problem.
+
+## Bisecting regressions
+
+For regressions (something that used to work, but no longer does), it
+is super useful if we can figure out precisely when the code stopped
+working.  The gold standard is to be able to identify the precise
+**PR** that broke the code, so we can ping the author, but even
+narrowing it down to a nightly build is helpful, especially as that
+then gives us a range of PRs. (One other challenge is that we
+sometimes land "rollup" PRs, which combine multiple PRs into one.)
+
+### cargo-bisect-rustc
+
+To help in figuring out the cause of a regression we have a tool
+called [cargo-bisect-rustc]. It will automatically download and test
+various builds of rustc. For recent regressions, it is even able to
+use the builds from our CI to track down the regression to a specific
+PR; for older regressions, it will simply identify a nightly.
+
+To learn to use [cargo-bisect-rustc], check out [this blog post][learn], which
+gives a quick introduction to how it works. Additionally, there is a [Guide]
+which goes into more detail on how to use it. You can also ask questions at
+the Zulip stream [`#t-compiler/cargo-bisect-rustc`][zcbr], or help in
+improving the tool.
+
+[cargo-bisect-rustc]: https://github.com/rust-lang/cargo-bisect-rustc/
+[learn]: https://blog.rust-lang.org/inside-rust/2019/12/18/bisecting-rust-compiler.html
+[zcbr]: https://rust-lang.zulipchat.com/#narrow/stream/217417-t-compiler.2Fcargo-bisect-rustc
+[Guide]: https://rust-lang.github.io/cargo-bisect-rustc/

src/notification-groups/llvm.md

@@ -0,0 +1,38 @@
+# LLVM Icebreakers Notification group
+
+**Github Label:** [A-LLVM] <br>
+**Ping command:** `@rustbot ping icebreakers-llvm`
+
+[A-LLVM]: https://github.com/rust-lang/rust/labels/A-LLVM
+
+*Note*: this notification group is *not* the same as the LLVM working group
+(WG-llvm).
+
+The "LLVM Icebreakers Notification Group" are focused on bugs that center around
+LLVM. These bugs often arise because of LLVM optimizations gone awry, or as the
+result of an LLVM upgrade. The goal here is:
+
+- to determine whether the bug is a result of us generating invalid LLVM IR,
+  or LLVM misoptimizing;
+- if the former, to fix our IR;
+- if the latter, to try and file a bug on LLVM (or identify an existing bug).
+
+The group may also be asked to weigh in on other sorts of LLVM-focused
+questions.
+
+## Helpful tips and options
+
+The ["Debugging LLVM"][d] section of the
+rustc-dev-guide gives a step-by-step process for how to help debug bugs
+caused by LLVM. In particular, it discusses how to emit LLVM IR, run
+the LLVM IR optimization pipelines, and so forth. You may also find
+it useful to look at the various codegen options listed under `-C help`
+and the internal options under `-Z help` -- there are a number that
+pertain to LLVM (just search for LLVM).
+
+[d]: ../backend/debugging.md
+
+## If you do narrow to an LLVM bug
+
+The ["Debugging LLVM"][d] section also describes what to do once
+you've identified the bug.

src/offload/installation.md

@@ -8,7 +8,7 @@ First you need to clone and configure the Rust repository:
 ```bash
 git clone --depth=1 git@github.com:rust-lang/rust.git
 cd rust
-./configure --enable-llvm-link-shared --release-channel=nightly --enable-llvm-assertions --enable-offload --enable-enzyme --enable-clang --enable-lld --enable-option-checking --enable-ninja --disable-docs
+./configure --enable-llvm-link-shared --release-channel=nightly --enable-llvm-assertions --enable-offload --enable-clang --enable-lld --enable-option-checking --enable-ninja --disable-docs
 ```
 
 Afterwards you can build rustc using:
@@ -18,7 +18,7 @@ Afterwards you can build rustc using:
 
 Afterwards rustc toolchain link will allow you to use it through cargo:
 ```
-rustup toolchain link offload build/host/stage1
+rustup toolchain link enzyme build/host/stage1
 rustup toolchain install nightly # enables -Z unstable-options
 ```
 
@@ -35,37 +35,3 @@ ninja
 ninja install
 ```
 This gives you a working LLVM build.
-
-
-## Testing
-run
-```
-./x.py test --stage 1 tests/codegen/gpu_offload
-```
-
-## Usage
-It is important to use a clang compiler build on the same llvm as rustc. Just calling clang without the full path will likely use your system clang, which probably will be incompatible.
-```
-/absolute/path/to/rust/build/x86_64-unknown-linux-gnu/stage1/bin/rustc --edition=2024 --crate-type cdylib src/main.rs --emit=llvm-ir  -O -C lto=fat -Cpanic=abort -Zoffload=Enable
-/absolute/path/to/rust/build/x86_64-unknown-linux-gnu/llvm/bin/clang++ -fopenmp --offload-arch=native -g  -O3 main.ll -o main -save-temps
-LIBOMPTARGET_INFO=-1  ./main
-```
-The first step will generate a `main.ll` file, which has enough instructions to cause the offload runtime to move data to and from a gpu.
-The second step will use clang as the compilation driver to compile our IR file down to a working binary. Only a very small Rust subset will work out of the box here, unless
-you use features like build-std, which are not covered by this guide. Look at the codegen test to get a feeling for how to write a working example.
-In the last step you can run your binary, if all went well you will see a data transfer being reported:
-```
-omptarget device 0 info: Entering OpenMP data region with being_mapper at unknown:0:0 with 1 arguments:
-omptarget device 0 info: tofrom(unknown)[1024]
-omptarget device 0 info: Creating new map entry with HstPtrBase=0x00007fffffff9540, HstPtrBegin=0x00007fffffff9540, TgtAllocBegin=0x0000155547200000, TgtPtrBegin=0x0000155547200000, Size=1024, DynRefCount=1, HoldRefCount=0, Name=unknown
-omptarget device 0 info: Copying data from host to device, HstPtr=0x00007fffffff9540, TgtPtr=0x0000155547200000, Size=1024, Name=unknown
-omptarget device 0 info: OpenMP Host-Device pointer mappings after block at unknown:0:0:
-omptarget device 0 info: Host Ptr           Target Ptr         Size (B) DynRefCount HoldRefCount Declaration
-omptarget device 0 info: 0x00007fffffff9540 0x0000155547200000 1024     1           0            unknown at unknown:0:0
-// some other output
-omptarget device 0 info: Exiting OpenMP data region with end_mapper at unknown:0:0 with 1 arguments:
-omptarget device 0 info: tofrom(unknown)[1024]
-omptarget device 0 info: Mapping exists with HstPtrBegin=0x00007fffffff9540, TgtPtrBegin=0x0000155547200000, Size=1024, DynRefCount=0 (decremented, delayed deletion), HoldRefCount=0
-omptarget device 0 info: Copying data from device to host, TgtPtr=0x0000155547200000, HstPtr=0x00007fffffff9540, Size=1024, Name=unknown
-omptarget device 0 info: Removing map entry with HstPtrBegin=0x00007fffffff9540, TgtPtrBegin=0x0000155547200000, Size=1024, Name=unknown
-```

src/offload/internals.md

@@ -1,9 +0,0 @@
-# std::offload
-
-This module is under active development. Once upstream, it should allow Rust developers to run Rust code on GPUs.
-We aim to develop a `rusty` GPU programming interface, which is safe, convenient and sufficiently fast by default.
-This includes automatic data movement to and from the GPU, in a efficient way. We will (later)
-also offer more advanced, possibly unsafe, interfaces which allow a higher degree of control.
-
-The implementation is based on LLVM's "offload" project, which is already used by OpenMP to run Fortran or C++ code on GPUs.
-While the project is under development, users will need to call other compilers like clang to finish the compilation process.

src/overview.md

@@ -410,7 +410,7 @@ For more details on bootstrapping, see
   - Guide: [The HIR](hir.md)
   - Guide: [Identifiers in the HIR](hir.md#identifiers-in-the-hir)
   - Guide: [The `HIR` Map](hir.md#the-hir-map)
-  - Guide: [Lowering `AST` to `HIR`](./hir/lowering.md)
+  - Guide: [Lowering `AST` to `HIR`](ast-lowering.md)
   - How to view `HIR` representation for your code `cargo rustc -- -Z unpretty=hir-tree`
   - Rustc `HIR` definition: [`rustc_hir`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/index.html)
   - Main entry point: **TODO**

src/part-5-intro.md

@@ -1,4 +1,4 @@
-# From MIR to binaries
+# From MIR to Binaries
 
 All of the preceding chapters of this guide have one thing in common:
 we never generated any executable machine code at all!

src/pat-exhaustive-checking.md

@@ -1,4 +1,4 @@
-# Pattern and exhaustiveness checking
+# Pattern and Exhaustiveness Checking
 
 In Rust, pattern matching and bindings have a few very helpful properties. The
 compiler will check that bindings are irrefutable when made and that match arms

src/profile-guided-optimization.md

@@ -1,4 +1,4 @@
-# Profile-guided optimization
+# Profile Guided Optimization
 
 <!-- toc -->
 
@@ -6,7 +6,7 @@
 This chapter describes what PGO is and how the support for it is
 implemented in `rustc`.
 
-## What is profiled-guided optimization?
+## What Is Profiled-Guided Optimization?
 
 The basic concept of PGO is to collect data about the typical execution of
 a program (e.g. which branches it is likely to take) and then use this data
@@ -52,7 +52,7 @@ instrumentation, via the experimental option
 [`-C instrument-coverage`](./llvm-coverage-instrumentation.md), but using these
 coverage results for PGO has not been attempted at this time.
 
-### Overall workflow
+### Overall Workflow
 
 Generating a PGO-optimized program involves the following four steps:
 
@@ -62,12 +62,12 @@ Generating a PGO-optimized program involves the following four steps:
 4. Compile the program again, this time making use of the profiling data
    (e.g. `rustc -C profile-use=merged.profdata main.rs`)
 
-### Compile-time aspects
+### Compile-Time Aspects
 
 Depending on which step in the above workflow we are in, two different things
 can happen at compile time:
 
-#### Create binaries with instrumentation
+#### Create Binaries with Instrumentation
 
 As mentioned above, the profiling instrumentation is added by LLVM.
 `rustc` instructs LLVM to do so [by setting the appropriate][pgo-gen-passmanager]
@@ -88,7 +88,7 @@ runtime are not removed [by marking the with the right export level][pgo-gen-sym
 [pgo-gen-symbols]:https://github.com/rust-lang/rust/blob/1.34.1/src/librustc_codegen_ssa/back/symbol_export.rs#L212-L225
 
 
-#### Compile binaries where optimizations make use of profiling data
+#### Compile Binaries Where Optimizations Make Use Of Profiling Data
 
 In the final step of the workflow described above, the program is compiled
 again, with the compiler using the gathered profiling data in order to drive
@@ -106,7 +106,7 @@ LLVM does the rest (e.g. setting branch weights, marking functions with
 `cold` or `inlinehint`, etc).
 
 
-### Runtime aspects
+### Runtime Aspects
 
 Instrumentation-based approaches always also have a runtime component, i.e.
 once we have an instrumented program, that program needs to be run in order
@@ -134,7 +134,7 @@ instrumentation artifacts show up in LLVM IR.
 [rmake-tests]: https://github.com/rust-lang/rust/tree/master/tests/run-make
 [codegen-test]: https://github.com/rust-lang/rust/blob/master/tests/codegen/pgo-instrumentation.rs
 
-## Additional information
+## Additional Information
 
 Clang's documentation contains a good overview on [PGO in LLVM][llvm-pgo].
 

src/profiling/with_perf.md

@@ -7,8 +7,8 @@ This is a guide for how to profile rustc with [perf](https://perf.wiki.kernel.or
 - Get a clean checkout of rust-lang/master, or whatever it is you want
   to profile.
 - Set the following settings in your `bootstrap.toml`:
-  - `rust.debuginfo-level = 1` - enables line debuginfo
-  - `rust.jemalloc = false` - lets you do memory use profiling with valgrind
+  - `debuginfo-level = 1` - enables line debuginfo
+  - `jemalloc = false` - lets you do memory use profiling with valgrind
   - leave everything else the defaults
 - Run `./x build` to get a full build
 - Make a rustup toolchain pointing to that result

src/queries/incremental-compilation-in-detail.md

@@ -1,4 +1,4 @@
-# Incremental compilation in detail
+# Incremental Compilation in detail
 
 <!-- toc -->
 
@@ -66,7 +66,7 @@ because it reads the up-to-date version of `Hir(bar)`. Also, we re-run
 `type_check_item(bar)` because result of `type_of(bar)` might have changed.
 
 
-## The problem with the basic algorithm: false positives
+## The Problem With The Basic Algorithm: False Positives
 
 If you read the previous paragraph carefully you'll notice that it says that
 `type_of(bar)` *might* have changed because one of its inputs has changed.
@@ -93,7 +93,7 @@ of examples like this and small changes to the input often potentially affect
 very large parts of the output binaries. As a consequence, we had to make the
 change detection system smarter and more accurate.
 
-## Improving accuracy: the red-green algorithm
+## Improving Accuracy: The red-green Algorithm
 
 The "false positives" problem can be solved by interleaving change detection
 and query re-evaluation. Instead of walking the graph all the way to the
@@ -191,7 +191,7 @@ then itself involve recursively invoking more queries, which can mean we come ba
 to the `try_mark_green()` algorithm for the dependencies recursively.
 
 
-## The real world: how persistence makes everything complicated
+## The Real World: How Persistence Makes Everything Complicated
 
 The sections above described the underlying algorithm for incremental
 compilation but because the compiler process exits after being finished and
@@ -258,7 +258,7 @@ the `LocalId`s within it are still the same.
 
 
 
-### Checking query results for changes: `HashStable` and `Fingerprint`s
+### Checking Query Results For Changes: HashStable And Fingerprints
 
 In order to do red-green-marking we often need to check if the result of a
 query has changed compared to the result it had during the previous
@@ -306,7 +306,7 @@ This approach works rather well but it's not without flaws:
   their stable equivalents while doing the hashing.
 
 
-### A tale of two `DepGraph`s: the old and the new
+### A Tale Of Two DepGraphs: The Old And The New
 
 The initial description of dependency tracking glosses over a few details
 that quickly become a head scratcher when actually trying to implement things.
@@ -344,7 +344,7 @@ new graph is serialized out to disk, alongside the query result cache, and can
 act as the previous dep-graph in a subsequent compilation session.
 
 
-### Didn't you forget something?: cache promotion
+### Didn't You Forget Something?: Cache Promotion
 
 The system described so far has a somewhat subtle property: If all inputs of a
 dep-node are green then the dep-node itself can be marked as green without
@@ -374,7 +374,7 @@ the result cache doesn't unnecessarily shrink again.
 
 
 
-# Incremental compilation and the compiler backend
+# Incremental Compilation and the Compiler Backend
 
 The compiler backend, the part involving LLVM, is using the query system but
 it is not implemented in terms of queries itself. As a consequence it does not
@@ -406,7 +406,7 @@ would save.
 
 
 
-## Query modifiers
+## Query Modifiers
 
 The query system allows for applying [modifiers][mod] to queries. These
 modifiers affect certain aspects of how the system treats the query with
@@ -472,7 +472,7 @@ respect to incremental compilation:
 [mod]: ../query.html#adding-a-new-kind-of-query
 
 
-## The projection query pattern
+## The Projection Query Pattern
 
 It's interesting to note that `eval_always` and `no_hash` can be used together
 in the so-called "projection query" pattern. It is often the case that there is
@@ -516,7 +516,7 @@ because we have the projections to take care of keeping things green as much
 as possible.
 
 
-# Shortcomings of the current system
+# Shortcomings of the Current System
 
 There are many things that still can be improved.
 

src/query.md

@@ -2,7 +2,7 @@
 
 <!-- toc -->
 
-As described in [Overview of the compiler], the Rust compiler
+As described in [the high-level overview of the compiler][hl], the Rust compiler
 is still (as of <!-- date-check --> July 2021) transitioning from a
 traditional "pass-based" setup to a "demand-driven" system. The compiler query
 system is the key to rustc's demand-driven organization.
@@ -13,7 +13,7 @@ there is a query called `type_of` that, given the [`DefId`] of
 some item, will compute the type of that item and return it to you.
 
 [`DefId`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_span/def_id/struct.DefId.html
-[Overview of the compiler]: overview.md#queries
+[hl]: ./compiler-src.md
 
 Query execution is *memoized*. The first time you invoke a
 query, it will go do the computation, but the next time, the result is
@@ -37,15 +37,12 @@ will in turn demand information about that crate, starting from the
   actual parsing.
 
 Although this vision is not fully realized, large sections of the
-compiler (for example, generating [MIR]) currently work exactly like this.
+compiler (for example, generating [MIR](./mir/index.md)) currently work exactly like this.
 
-[^incr-comp-detail]: The [Incremental compilation in detail] chapter gives a more
+[^incr-comp-detail]: The ["Incremental Compilation in Detail](queries/incremental-compilation-in-detail.md) chapter gives a more
 in-depth description of what queries are and how they work.
 If you intend to write a query of your own, this is a good read.
 
-[Incremental compilation in detail]: queries/incremental-compilation-in-detail.md
-[MIR]: mir/index.md
-
 ## Invoking queries
 
 Invoking a query is simple. The [`TyCtxt`] ("type context") struct offers a method
@@ -70,15 +67,9 @@ are cheaply cloneable; insert an `Rc` if necessary).
 ### Providers
 
 If, however, the query is *not* in the cache, then the compiler will
-call the corresponding **provider** function. A provider is a function
-implemented in a specific module and **manually registered** into the
-[`Providers`][providers_struct] struct during compiler initialization.
-The macro system generates the [`Providers`][providers_struct] struct,
-which acts as a function table for all query implementations, where each
-field is a function pointer to the actual provider.
-
-**Note:** The `Providers` struct is generated by macros and acts as a function table for all query implementations.
-It is **not** a Rust trait, but a plain struct with function pointer fields.
+try to find a suitable **provider**. A provider is a function that has
+been defined and linked into the compiler somewhere that contains the
+code to compute the result of the query.
 
 **Providers are defined per-crate.** The compiler maintains,
 internally, a table of providers for every crate, at least
@@ -106,18 +97,7 @@ fn provider<'tcx>(
 Providers take two arguments: the `tcx` and the query key.
 They return the result of the query.
 
-N.B. Most of the `rustc_*` crates only provide **local
-providers**. Almost all **extern providers** wind up going through the
-[`rustc_metadata` crate][rustc_metadata], which loads the information
-from the crate metadata. But in some cases there are crates that
-provide queries for *both* local and external crates, in which case
-they define both a `provide` and a `provide_extern` function, through
-[`wasm_import_module_map`][wasm_import_module_map], that `rustc_driver` can invoke.
-
-[rustc_metadata]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_metadata/index.html
-[wasm_import_module_map]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/back/symbol_export/fn.wasm_import_module_map.html
-
-### How providers are set up
+###  How providers are setup
 
 When the tcx is created, it is given the providers by its creator using
 the [`Providers`][providers_struct] struct. This struct is generated by
@@ -128,16 +108,19 @@ the macros here, but it is basically a big list of function pointers:
 ```rust,ignore
 struct Providers {
     type_of: for<'tcx> fn(TyCtxt<'tcx>, DefId) -> Ty<'tcx>,
-    // ... one field for each query
+    ...
 }
 ```
 
-#### How are providers registered?
+At present, we have one copy of the struct for local crates, and one
+for external crates, though the plan is that we may eventually have
+one per crate.
 
-The `Providers` struct is filled in during compiler initialization, mainly by the `rustc_driver` crate.  
-But the actual provider functions are implemented in various `rustc_*` crates (like `rustc_middle`, `rustc_hir_analysis`, etc).
-
-To register providers, each crate exposes a [`provide`][provide_fn] function that looks like this:
+These `Providers` structs are ultimately created and populated by
+`rustc_driver`, but it does this by distributing the work
+throughout the other `rustc_*` crates. This is done by invoking
+various [`provide`][provide_fn] functions. These functions tend to look
+something like this:
 
 [provide_fn]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/fn.provide.html
 
@@ -145,34 +128,41 @@ To register providers, each crate exposes a [`provide`][provide_fn] function tha
 pub fn provide(providers: &mut Providers) {
     *providers = Providers {
         type_of,
-        // ... add more providers here
         ..*providers
     };
 }
 ```
 
-- This function takes a mutable reference to the `Providers` struct and sets the fields to point to the correct provider functions.
-- You can also assign fields individually, e.g. `providers.type_of = type_of;`.
+That is, they take an `&mut Providers` and mutate it in place. Usually
+we use the formulation above just because it looks nice, but you could
+as well do `providers.type_of = type_of`, which would be equivalent.
+(Here, `type_of` would be a top-level function, defined as we saw
+before.) So, if we want to add a provider for some other query,
+let's call it `fubar`, into the crate above, we might modify the `provide()`
+function like so:
 
-#### Adding a new provider
+```rust,ignore
+pub fn provide(providers: &mut Providers) {
+    *providers = Providers {
+        type_of,
+        fubar,
+        ..*providers
+    };
+}
 
-Suppose you want to add a new query called `fubar`. You would:
+fn fubar<'tcx>(tcx: TyCtxt<'tcx>, key: DefId) -> Fubar<'tcx> { ... }
+```
 
-1. Implement the provider function:
-    ```rust,ignore
-    fn fubar<'tcx>(tcx: TyCtxt<'tcx>, key: DefId) -> Fubar<'tcx> { ... }
-    ```
-2. Register it in the `provide` function:
-    ```rust,ignore
-    pub fn provide(providers: &mut Providers) {
-        *providers = Providers {
-            fubar,
-            ..*providers
-        };
-    }
-    ```
+N.B. Most of the `rustc_*` crates only provide **local
+providers**. Almost all **extern providers** wind up going through the
+[`rustc_metadata` crate][rustc_metadata], which loads the information
+from the crate metadata. But in some cases there are crates that
+provide queries for *both* local and external crates, in which case
+they define both a `provide` and a `provide_extern` function, through
+[`wasm_import_module_map`][wasm_import_module_map], that `rustc_driver` can invoke.
 
----
+[rustc_metadata]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_metadata/index.html
+[wasm_import_module_map]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/back/symbol_export/fn.wasm_import_module_map.html
 
 ## Adding a new query
 

src/rustdoc-internals.md

@@ -270,6 +270,35 @@ in `test.rs` is the function `make_test`, which is where hand-written
 Some extra reading about `make_test` can be found
 [here](https://quietmisdreavus.net/code/2018/02/23/how-the-doctests-get-made/).
 
+## Dotting i's And Crossing t's
+
+So that's `rustdoc`'s code in a nutshell, but there's more things in the
+compiler that deal with it. Since we have the full `compiletest` suite at hand,
+there's a set of tests in `tests/rustdoc` that make sure the final `HTML` is
+what we expect in various situations. These tests also use a supplementary
+script, `src/etc/htmldocck.py`, that allows it to look through the final `HTML`
+using `XPath` notation to get a precise look at the output. The full
+description of all the commands available to `rustdoc` tests (e.g. [`@has`] and
+[`@matches`]) is in [`htmldocck.py`].
+
+To use multiple crates in a `rustdoc` test, add `//@ aux-build:filename.rs`
+to the top of the test file. `filename.rs` should be placed in an `auxiliary`
+directory relative to the test file with the comment. If you need to build
+docs for the auxiliary file, use `//@ build-aux-docs`.
+
+In addition, there are separate tests for the search index and `rustdoc`'s
+ability to query it. The files in `tests/rustdoc-js` each contain a
+different search query and the expected results, broken out by search tab.
+These files are processed by a script in `src/tools/rustdoc-js` and the `Node.js`
+runtime. These tests don't have as thorough of a writeup, but a broad example
+that features results in all tabs can be found in `basic.js`. The basic idea is
+that you match a given `QUERY` with a set of `EXPECTED` results, complete with
+the full item path of each item.
+
+[`@has`]: https://github.com/rust-lang/rust/blob/master/src/etc/htmldocck.py#L39
+[`@matches`]: https://github.com/rust-lang/rust/blob/master/src/etc/htmldocck.py#L44
+[`htmldocck.py`]: https://github.com/rust-lang/rust/blob/master/src/etc/htmldocck.py
+
 ## Testing Locally
 
 Some features of the generated `HTML` documentation might require local

src/rustdoc-internals/rustdoc-gui-test-suite.md

@@ -1,14 +0,0 @@
-# The `rustdoc-gui` test suite
-
-> **FIXME**: This section is a stub. Please help us flesh it out!
-
-This page is about the test suite named `rustdoc-gui` used to test the "GUI" of `rustdoc` (i.e., the HTML/JS/CSS as rendered in a browser).
-For other rustdoc-specific test suites, see [Rustdoc test suites].
-
-These use a NodeJS-based tool called [`browser-UI-test`] that uses [puppeteer] to run tests in a headless browser and check rendering and interactivity. For information on how to write this form of test, see [`tests/rustdoc-gui/README.md`][rustdoc-gui-readme] as well as [the description of the `.goml` format][goml-script]
-
-[Rustdoc test suites]: ../tests/compiletest.md#rustdoc-test-suites
-[`browser-UI-test`]: https://github.com/GuillaumeGomez/browser-UI-test/
-[puppeteer]: https://pptr.dev/
-[rustdoc-gui-readme]: https://github.com/rust-lang/rust/blob/master/tests/rustdoc-gui/README.md
-[goml-script]: https://github.com/GuillaumeGomez/browser-UI-test/blob/master/goml-script.md

src/rustdoc-internals/rustdoc-json-test-suite.md

@@ -1,3 +0,0 @@
-# The `rustdoc-json` test suite
-
-> **FIXME**: This section is a stub. It will be populated by [PR #2422](https://github.com/rust-lang/rustc-dev-guide/pull/2422/).

src/rustdoc-internals/rustdoc-test-suite.md

@@ -1,191 +1,112 @@
 # The `rustdoc` test suite
 
-This page is about the test suite named `rustdoc` used to test the HTML output of `rustdoc`.
-For other rustdoc-specific test suites, see [Rustdoc test suites].
+This page is specifically about the test suite named `rustdoc`.
+For other test suites used for testing rustdoc, see [Rustdoc tests](../rustdoc.md#tests).
 
-Each test file in this test suite is simply a Rust source file `file.rs` sprinkled with
-so-called *directives* located inside normal Rust code comments.
-These come in two flavors: *Compiletest* and *HtmlDocCk*.
+The `rustdoc` test suite is specifically used to test the HTML output of rustdoc.
 
-To learn more about the former, read [Compiletest directives].
-For the latter, continue reading.
+This is achieved by means of `htmldocck.py`, a custom checker script that leverages [XPath].
 
-Internally, [`compiletest`] invokes the supplementary checker script [`htmldocck.py`].
+[XPath]: https://en.wikipedia.org/wiki/XPath
 
-[Rustdoc test suites]: ../tests/compiletest.md#rustdoc-test-suites
-[`compiletest`]: ../tests/compiletest.md
-[`htmldocck.py`]: https://github.com/rust-lang/rust/blob/master/src/etc/htmldocck.py
+## Directives
+Directives to htmldocck are similar to those given to `compiletest` in that they take the form of `//@` comments.
 
-## HtmlDocCk Directives
+In addition to the directives listed here,
+`rustdoc` tests also support most
+[compiletest directives](../tests/directives.html).
 
-Directives to HtmlDocCk are assertions that place constraints on the generated HTML.
-They look similar to those given to `compiletest` in that they take the form of `//@` comments
-but ultimately, they are completey distinct and processed by different programs.
+All `PATH`s in directives are relative to the rustdoc output directory (`build/TARGET/test/rustdoc/TESTNAME`),
+so it is conventional to use a `#![crate_name = "foo"]` attribute to avoid
+having to write a long crate name multiple times.
+To avoid repetition, `-` can be used in any `PATH` argument to re-use the previous `PATH` argument.
 
-[XPath] is used to query parts of the HTML document tree.
-
-**Introductory example**:
-
-```rust,ignore (illustrative)
-//@ has file/type.Alias.html
-//@ has - '//*[@class="rust item-decl"]//code' 'type Alias = Option<i32>;'
-pub type Alias = Option<i32>;
-```
-
-Here, we check that documentation generated for crate `file` contains a page for the
-public type alias `Alias` where the code block that is found at the top contains the
-expected rendering of the item. The `//*[@class="rust item-decl"]//code` is an XPath
-expression.
-
-Conventionally, you place these directives directly above the thing they are meant to test.
-Technically speaking however, they don't need to be as HtmlDocCk only looks for the directives.
-
-All directives take a `PATH` argument.
-To avoid repetition, `-` can be passed to it to re-use the previous `PATH` argument.
-Since the path contains the name of the crate, it is conventional to add a
-`#![crate_name = "foo"]` attribute to the crate root to shorten the resulting path.
-
-All arguments take the form of shell-style (single or double) quoted strings,
+All arguments take the form of quoted strings
+(both single and double quotes are supported),
 with the exception of `COUNT` and the special `-` form of `PATH`.
 
-All directives (except `files`) can be *negated* by putting a `!` in front of their name.
-Before you add negated directives, please read about [their caveats](#caveats).
+Directives are assertions that place constraints on the generated HTML.
+
+All directives (except `files`) can be negated by putting a `!` in front of their name.
 
 Similar to shell commands,
 directives can extend across multiple lines if their last char is `\`.
 In this case, the start of the next line should be `//`, with no `@`.
 
-Use the special string `{{channel}}` in XPaths, `PATTERN` arguments and [snapshot files](#snapshot)
-if you'd like to refer to the URL `https://doc.rust-lang.org/CHANNEL` where `CHANNEL` refers to the
-current release channel (e.g, `stable` or `nightly`).
-
-Listed below are all possible directives:
-
-[XPath]: https://en.wikipedia.org/wiki/XPath
+For example, `//@ !has 'foo/struct.Bar.html'` checks that crate `foo` does not have a page for a struct named `Bar` in the crate root.
 
 ### `has`
 
-> Usage 1: `//@ has PATH`
+Usage 1: `//@ has PATH`
+Usage 2: `//@ has PATH XPATH PATTERN`
 
-Check that the file given by `PATH` exists.
+In the first form, `has` checks that a given file exists.
 
-> Usage 2: `//@ has PATH XPATH PATTERN`
-
-Checks that the text of each element / attribute / text selected by `XPATH` in the
-whitespace-normalized[^1] file given by `PATH` matches the
-(also whitespace-normalized) string `PATTERN`.
-
-**Tip**: If you'd like to avoid whitespace normalization and/or if you'd like to match with a regex,
-use `matches` instead.
-
-### `hasraw`
-
-> Usage: `//@ hasraw PATH PATTERN`
-
-Checks that the contents of the whitespace-normalized[^1] file given by `PATH`
-matches the (also whitespace-normalized) string `PATTERN`.
-
-**Tip**: If you'd like to avoid whitespace normalization and / or if you'd like to match with a
-regex, use `matchesraw` instead.
+In the second form, `has` is an alias for `matches`,
+except `PATTERN` is a whitespace-normalized[^1] string instead of a regex.
 
 ### `matches`
 
-> Usage: `//@ matches PATH XPATH PATTERN`
+Usage: `//@ matches PATH XPATH PATTERN`
 
-Checks that the text of each element / attribute / text selected by `XPATH` in the
-file given by `PATH` matches the Python-flavored[^2] regex `PATTERN`.
+Checks that the text of each element selected by `XPATH` in `PATH` matches the python-flavored regex `PATTERN`.
 
 ### `matchesraw`
 
-> Usage: `//@ matchesraw PATH PATTERN`
+Usage: `//@ matchesraw PATH PATTERN`
 
-Checks that the contents of the file given by `PATH` matches the
-Python-flavored[^2] regex `PATTERN`.
+Checks that the contents of the file `PATH` matches the regex `PATTERN`.
+
+### `hasraw`
+
+Usage: `//@ hasraw PATH PATTERN`
+
+Same as `matchesraw`, except `PATTERN` is a whitespace-normalized[^1] string instead of a regex.
 
 ### `count`
 
-> Usage: `//@ count PATH XPATH COUNT`
+Usage: `//@ count PATH XPATH COUNT`
 
-Checks that there are exactly `COUNT` matches for `XPATH` within the file given by `PATH`.
+Checks that there are exactly `COUNT` matches for `XPATH` within the file `PATH`.
 
 ### `snapshot`
 
-> Usage: `//@ snapshot NAME PATH XPATH`
+Usage: `//@ snapshot NAME PATH XPATH`
 
-Checks that the element / text selected by `XPATH` in the file given by `PATH` matches the
-pre-recorded subtree or text (the "snapshot") in file `FILE_STEM.NAME.html` where `FILE_STEM`
-is the file stem of the test file.
+Creates a snapshot test named NAME.
+A snapshot test captures a subtree of the DOM, at the location
+determined by the XPath, and compares it to a pre-recorded value
+in a file. The file's name is the test's name with the `.rs` extension
+replaced with `.NAME.html`, where NAME is the snapshot's name.
 
-Pass the `--bless` option to `compiletest` to accept the current subtree/text as expected.
-This will overwrite the aforementioned file (or create it if it doesn't exist). It will
-automatically normalize the channel-dependent URL `https://doc.rust-lang.org/CHANNEL` to
-the special string `{{channel}}`.
+htmldocck supports the `--bless` option to accept the current subtree
+as expected, saving it to the file determined by the snapshot's name.
+compiletest's `--bless` flag is forwarded to htmldocck.
 
 ### `has-dir`
 
-> Usage: `//@ has-dir PATH`
+Usage: `//@ has-dir PATH`
 
-Checks for the existence of the directory given by `PATH`.
+Checks for the existence of directory `PATH`.
 
 ### `files`
 
-> Usage: `//@ files PATH ENTRIES`
+Usage: `//@ files PATH ENTRIES`
 
-Checks that the directory given by `PATH` contains exactly `ENTRIES`.
-`ENTRIES` is a Python-like list of strings inside a quoted string.
+Checks that the directory `PATH` contains exactly `ENTRIES`.
+`ENTRIES` is a python list of strings inside a quoted string,
+as if it were to be parsed by `eval`.
+(note that the list is actually parsed by `shlex.split`,
+so it cannot contain arbitrary python expressions).
 
-**Example**: `//@ files "foo/bar" '["index.html", "sidebar-items.js"]'`
+Example: `//@ files "foo/bar" '["index.html", "sidebar-items.js"]'`
 
-[^1]: Whitespace normalization means that all spans of consecutive whitespace are replaced with a single space.
-[^2]: They are Unicode aware (flag `UNICODE` is set), match case-sensitively and in single-line mode.
-
-## Compiletest Directives (Brief)
-
-As mentioned in the introduction, you also have access to [compiletest directives].
-Most importantly, they allow you to register auxiliary crates and
-to pass flags to the `rustdoc` binary under test.
-It's *strongly recommended* to read that chapter if you don't know anything about them yet.
-
-Here are some details that are relevant to this test suite specifically:
-
-* While you can use both `//@ compile-flags` and `//@ doc-flags` to pass flags to `rustdoc`,
-  prefer to user the latter to show intent. The former is meant for `rustc`.
-* Add `//@ build-aux-docs` to the test file that has auxiliary crates to not only compile the
-  auxiliaries with `rustc` but to also document them with `rustdoc`.
-
-## Caveats
-
-Testing for the absence of an element or a piece of text is quite fragile and not very future proof.
-
-It's not unusual that the *shape* of the generated HTML document tree changes from time to time.
-This includes for example renamings of CSS classes.
-
-Whenever that happens, *positive* checks will either continue to match the intended element /
-attribute / text (if their XPath expression is general / loose enough) and
-thus continue to test the correct thing or they won't in which case they would fail thereby
-forcing the author of the change to look at them.
-
-Compare that to *negative* checks (e.g., `//@ !has PATH XPATH PATTERN`) which won't fail if their
-XPath expression "no longer" matches. The author who changed "the shape" thus won't get notified and
-as a result someone else can unintentionally reintroduce `PATTERN` into the generated docs without
-the original negative check failing.
-
-**Note**: Please avoid the use of *negated* checks!
-
-**Tip**: If you can't avoid it, please **always** pair it with an analogous positive check in the
-immediate vicinity, so people changing "the shape" have a chance to notice and to update the
-negated check!
+[^1]: Whitespace normalization means that all spans of consecutive whitespace are replaced with a single space.  The files themselves are also whitespace-normalized.
 
 ## Limitations
-
-HtmlDocCk uses the XPath implementation from the Python standard library.
+`htmldocck.py` uses the xpath implementation from the standard library.
 This leads to several limitations:
-
 * All `XPATH` arguments must start with `//` due to a flaw in the implementation.
 * Many XPath features (functions, axies, etc.) are not supported.
 * Only well-formed HTML can be parsed (hopefully rustdoc doesn't output mismatched tags).
 
-Furthmore, compiletest [revisions] are not supported.
-
-[revisions]: ../tests/compiletest.md#revisions
-[compiletest directives]: ../tests/directives.md

src/rustdoc.md

@@ -67,29 +67,43 @@ does is call the `main()` that's in this crate's `lib.rs`, though.)
 
 ## Code structure
 
-All paths in this section are relative to `src/librustdoc/` in the rust-lang/rust repository.
-
+* All paths in this section are relative to `src/librustdoc` in the rust-lang/rust repository.
 * Most of the HTML printing code is in `html/format.rs` and `html/render/mod.rs`.
-  It's in a bunch of functions returning `impl std::fmt::Display`.
-* The data types that get rendered by the functions mentioned above are defined in `clean/types.rs`.
-  The functions responsible for creating them from the `HIR` and the `rustc_middle::ty` IR
-  live in `clean/mod.rs`.
+  It's in a bunch of `fmt::Display` implementations and supplementary
+  functions.
+* The types that got `Display` impls above are defined in `clean/mod.rs`, right
+  next to the custom `Clean` trait used to process them out of the rustc HIR.
 * The bits specific to using rustdoc as a test harness are in
   `doctest.rs`.
 * The Markdown renderer is loaded up in `html/markdown.rs`, including functions
   for extracting doctests from a given block of Markdown.
 * Frontend CSS and JavaScript are stored in `html/static/`.
-  * Re. JavaScript, type annotations are written using [TypeScript-flavored JSDoc]
-comments and an external `.d.ts` file.
-    This way, the code itself remains plain, valid JavaScript.
-    We only use `tsc` as a linter.
-
-[TypeScript-flavored JSDoc]: https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html
 
 ## Tests
 
-`rustdoc`'s integration tests are split across several test suites.
-See [Rustdoc tests suites](tests/compiletest.md#rustdoc-test-suites) for more details.
+* Tests on search engine and index are located in `tests/rustdoc-js` and `tests/rustdoc-js-std`.
+  The format is specified
+  [in the search guide](rustdoc-internals/search.md#testing-the-search-engine).
+* Tests on the "UI" of rustdoc (the terminal output it produces when run) are in
+  `tests/rustdoc-ui`
+* Tests on the "GUI" of rustdoc (the HTML, JS, and CSS as rendered in a browser)
+  are in `tests/rustdoc-gui`. These use a [NodeJS tool called
+  browser-UI-test](https://github.com/GuillaumeGomez/browser-UI-test/) that uses
+  puppeteer to run tests in a headless browser and check rendering and
+  interactivity.  For information on how to write this form of test,
+  see [`tests/rustdoc-gui/README.md`][rustdoc-gui-readme]
+  as well as [the description of the `.goml` format][goml-script]
+* Tests on the structure of rustdoc HTML output are located in `tests/rustdoc`,
+  where they're handled by the test runner of bootstrap and
+  the supplementary script `src/etc/htmldocck.py`.
+  [These tests have several extra directives available to them](./rustdoc-internals/rustdoc-test-suite.md).
+* Additionally, JavaScript type annotations are written using [TypeScript-flavored JSDoc]
+  comments and an external d.ts file. The code itself is plain, valid JavaScript; we only
+  use tsc as a linter.
+
+[TypeScript-flavored JSDoc]: https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html
+[rustdoc-gui-readme]: https://github.com/rust-lang/rust/blob/master/tests/rustdoc-gui/README.md
+[goml-script]: https://github.com/GuillaumeGomez/browser-UI-test/blob/master/goml-script.md
 
 ## Constraints
 

src/sanitizers.md

@@ -1,4 +1,4 @@
-# Sanitizers support
+# Sanitizers Support
 
 The rustc compiler contains support for following sanitizers:
 

src/tests/ci.md

@@ -66,9 +66,9 @@ kinds of builds (sets of jobs).
 ### Pull Request builds
 
 After each push to a pull request, a set of `pr` jobs are executed. Currently,
-these execute the `x86_64-gnu-llvm-X`, `x86_64-gnu-tools`, `mingw-check-1`, `mingw-check-2`
-and `mingw-check-tidy` jobs, all running on Linux. These execute a relatively short
-(~40 minutes) and lightweight test suite that should catch common issues. More
+these execute the `x86_64-gnu-llvm-X`, `x86_64-gnu-tools`, `mingw-check` and
+`mingw-check-tidy` jobs, all running on Linux. These execute a relatively short
+(~30 minutes) and lightweight test suite that should catch common issues. More
 specifically, they run a set of lints, they try to perform a cross-compile check
 build to Windows mingw (without producing any artifacts) and they test the
 compiler using a *system* version of LLVM. Unfortunately, it would take too many

src/tests/compiletest.md

@@ -56,9 +56,6 @@ incremental compilation. The various suites are defined in
 
 The following test suites are available, with links for more information:
 
-[`tests`]: https://github.com/rust-lang/rust/blob/master/tests
-[`src/tools/compiletest/src/common.rs`]: https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/common.rs
-
 ### Compiler-specific test suites
 
 | Test suite                                | Purpose                                                                                                             |
@@ -74,7 +71,6 @@ The following test suites are available, with links for more information:
 | [`mir-opt`](#mir-opt-tests)               | Check MIR generation and optimizations                                                                              |
 | [`coverage`](#coverage-tests)             | Check coverage instrumentation                                                                                      |
 | [`coverage-run-rustdoc`](#coverage-tests) | `coverage` tests that also run instrumented doctests                                                                |
-| [`crashes`](#crashes-tests)               | Check that the compiler ICEs/panics/crashes on certain inputs to catch accidental fixes                             |
 
 ### General purpose test suite
 
@@ -82,23 +78,19 @@ The following test suites are available, with links for more information:
 
 ### Rustdoc test suites
 
-| Test suite                           | Purpose                                                                  |
-|--------------------------------------|--------------------------------------------------------------------------|
-| [`rustdoc`][rustdoc-html-tests]      | Check HTML output of `rustdoc`                                           |
-| [`rustdoc-gui`][rustdoc-gui-tests]   | Check `rustdoc`'s GUI using a web browser                                |
-| [`rustdoc-js`][rustdoc-js-tests]     | Check `rustdoc`'s search engine and index                                |
-| [`rustdoc-js-std`][rustdoc-js-tests] | Check `rustdoc`'s search engine and index on the std library docs        |
-| [`rustdoc-json`][rustdoc-json-tests] | Check JSON output of `rustdoc`                                           |
-| `rustdoc-ui`                         | Check terminal output of `rustdoc` ([see also](ui.md))                   |
+See [Rustdoc tests](../rustdoc.md#tests) for more details.
 
-Some rustdoc-specific tests can also be found in `ui/rustdoc/`.
-These check rustdoc-related or -specific lints that (also) run as part of `rustc`, not (only) `rustdoc`.
-Run-make tests pertaining to rustdoc are typically named `run-make/rustdoc-*/`.
+| Test suite       | Purpose                                                                  |
+|------------------|--------------------------------------------------------------------------|
+| `rustdoc`        | Check `rustdoc` generated files contain the expected documentation       |
+| `rustdoc-gui`    | Check `rustdoc`'s GUI using a web browser                                |
+| `rustdoc-js`     | Check `rustdoc` search is working as expected                            |
+| `rustdoc-js-std` | Check rustdoc search is working as expected specifically on the std docs |
+| `rustdoc-json`   | Check JSON output of `rustdoc`                                           |
+| `rustdoc-ui`     | Check terminal output of `rustdoc`                                       |
 
-[rustdoc-html-tests]: ../rustdoc-internals/rustdoc-test-suite.md
-[rustdoc-gui-tests]: ../rustdoc-internals/rustdoc-gui-test-suite.md
-[rustdoc-js-tests]: ../rustdoc-internals/search.md#testing-the-search-engine
-[rustdoc-json-tests]: ../rustdoc-internals/rustdoc-json-test-suite.md
+[`tests`]: https://github.com/rust-lang/rust/blob/master/tests
+[`src/tools/compiletest/src/common.rs`]: https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/common.rs
 
 ### Pretty-printer tests
 
@@ -115,7 +107,7 @@ default behavior without any commands is to:
 2. Run `rustc -Zunpretty=normal` on the output of the previous step.
 3. The output of the previous two steps should be the same.
 4. Run `rustc -Zno-codegen` on the output to make sure that it can type check
-   (similar to `cargo check`).
+   (this is similar to running `cargo check`).
 
 If any of the commands above fail, then the test fails.
 

src/tests/directives.md

@@ -202,7 +202,6 @@ settings:
   `//@ needs-crate-type: cdylib, proc-macro` will cause the test to be ignored
   on `wasm32-unknown-unknown` target because the target does not support the
   `proc-macro` crate type.
-- `needs-target-std` — ignores if target platform does not have std support.
 
 The following directives will check LLVM support:
 
@@ -262,7 +261,7 @@ Consider writing the test as a proper incremental test instead.
 
 | Directive   | Explanation                                                  | Supported test suites                    | Possible values           |
 |-------------|--------------------------------------------------------------|------------------------------------------|---------------------------|
-| `doc-flags` | Flags passed to `rustdoc` when building the test or aux file | `rustdoc`, `rustdoc-js`, `rustdoc-json`  | Any valid `rustdoc` flags |
+| `doc-flags` | Flags passed to `rustdoc` when building the test or aux file | `rustdoc`, `rustdoc-js`, `rustdoc-json` | Any valid `rustdoc` flags |
 
 <!--
 **FIXME(rustdoc)**: what does `check-test-line-numbers-match` do?
@@ -270,17 +269,6 @@ Asked in
 <https://rust-lang.zulipchat.com/#narrow/stream/266220-t-rustdoc/topic/What.20is.20the.20.60check-test-line-numbers-match.60.20directive.3F>.
 -->
 
-#### Test-suite-specific directives
-
-The test suites [`rustdoc`][rustdoc-html-tests], [`rustdoc-js`/`rustdoc-js-std`][rustdoc-js-tests]
-and [`rustdoc-json`][rustdoc-json-tests] each feature an additional set of directives whose basic
-syntax resembles the one of compiletest directives but which are ultimately read and checked by
-separate tools. For more information, please read their respective chapters as linked above.
-
-[rustdoc-html-tests]: ../rustdoc-internals/rustdoc-test-suite.md
-[rustdoc-js-tests]: ../rustdoc-internals/search.html#testing-the-search-engine
-[rustdoc-json-tests]: ../rustdoc-internals/rustdoc-json-test-suite.md
-
 ### Pretty printing
 
 See [Pretty-printer](compiletest.md#pretty-printer-tests).

src/tests/ui.md

@@ -220,12 +220,8 @@ negligible (i.e. there is no semantic difference between `//~ ERROR` and
 `//~ERROR` although the former is more common in the codebase).
 
 `~? <diagnostic kind>` (example being `~? ERROR`)
-is used to match diagnostics _without_ line info at all,
-or where the line info is outside the main test file[^main test file].
-These annotations can be placed on any line in the test file.
-
-[^main test file]: This is a file that has the `~?` annotations,
-as distinct from aux files, or sources that we have no control over.
+is used to match diagnostics without line information.
+These can be placed on any line in the test file, but are conventionally placed at the end.
 
 ### Error annotation examples
 

src/ty.md

@@ -62,8 +62,8 @@ Here is a summary:
 | Describe the *syntax* of a type: what the user wrote (with some desugaring).  | Describe the *semantics* of a type: the meaning of what the user wrote. |
 | Each `rustc_hir::Ty` has its own spans corresponding to the appropriate place in the program. | Doesn’t correspond to a single place in the user’s program. |
 | `rustc_hir::Ty` has generics and lifetimes; however, some of those lifetimes are special markers like [`LifetimeKind::Implicit`][implicit]. | `ty::Ty` has the full type, including generics and lifetimes, even if the user left them out |
-| `fn foo(x: u32) -> u32 { }` - Two `rustc_hir::Ty` representing each usage of `u32`, each has its own `Span`s, and `rustc_hir::Ty` doesn’t tell us that both are the same type | `fn foo(x: u32) -> u32 { }` - One `ty::Ty` for all instances of `u32` throughout the program, and `ty::Ty` tells us that both usages of `u32` mean the same type. |
-| `fn foo(x: &u32) -> &u32 { }` - Two `rustc_hir::Ty` again. Lifetimes for the references show up in the `rustc_hir::Ty`s using a special marker, [`LifetimeKind::Implicit`][implicit]. | `fn foo(x: &u32) -> &u32 { }`- A single `ty::Ty`. The `ty::Ty` has the hidden lifetime param. |
+| `fn foo(x: u32) → u32 { }` - Two `rustc_hir::Ty` representing each usage of `u32`, each has its own `Span`s, and `rustc_hir::Ty` doesn’t tell us that both are the same type | `fn foo(x: u32) → u32 { }` - One `ty::Ty` for all instances of `u32` throughout the program, and `ty::Ty` tells us that both usages of `u32` mean the same type. |
+| `fn foo(x: &u32) -> &u32)` - Two `rustc_hir::Ty` again. Lifetimes for the references show up in the `rustc_hir::Ty`s using a special marker, [`LifetimeKind::Implicit`][implicit]. | `fn foo(x: &u32) -> &u32)`- A single `ty::Ty`. The `ty::Ty` has the hidden lifetime param. |
 
 [implicit]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/enum.LifetimeKind.html#variant.Implicit
 

triagebot.toml

@@ -72,23 +72,6 @@ days-threshold = 7
 # Documentation at: https://forge.rust-lang.org/triagebot/pr-assignment.html
 [assign]
 
-# NOTE: do not add `[assign.owners]` if we still wish to keep the opt-in
-# reviewer model, as `[assign.owners]` will cause triagebot auto-reviewer
-# assignment to kick in.
-
-# Custom PR welcome message for when no auto reviewer assignment is performed
-# and no explicit manual reviewer selection is made.
-# Documentation at: https://forge.rust-lang.org/triagebot/pr-assignment.html#custom-welcome-messages
-[assign.custom_welcome_messages]
-welcome-message = ""
-welcome-message-no-reviewer = """\
-Thanks for the PR. If you have write access, feel free to merge this PR if it \
-does not need reviews. You can request a review using `r? rustc-dev-guide` or \
-`r? <username>`.
-"""
-
-# Groups for `r? <group>`.
-# Documentation at: https://forge.rust-lang.org/triagebot/pr-assignment.html#usage
 # Keep members alphanumerically sorted.
 [assign.adhoc_groups]
 rustc-dev-guide = [