;
+```
+
+Here, we check that documentation generated for crate `file` contains a page for the
+public type alias `Alias` where the code block that is found at the top contains the
+expected rendering of the item. The `//*[@class="rust item-decl"]//code` is an XPath
+expression.
+
+Conventionally, you place these directives directly above the thing they are meant to test.
+Technically speaking however, they don't need to be as HtmlDocCk only looks for the directives.
+
+All directives take a `PATH` argument.
+To avoid repetition, `-` can be passed to it to re-use the previous `PATH` argument.
+Since the path contains the name of the crate, it is conventional to add a
+`#![crate_name = "foo"]` attribute to the crate root to shorten the resulting path.
+
+All arguments take the form of shell-style (single or double) quoted strings,
with the exception of `COUNT` and the special `-` form of `PATH`.
-Directives are assertions that place constraints on the generated HTML.
-
-All directives (except `files`) can be negated by putting a `!` in front of their name.
+All directives (except `files`) can be *negated* by putting a `!` in front of their name.
+Before you add negated directives, please read about [their caveats](#caveats).
Similar to shell commands,
directives can extend across multiple lines if their last char is `\`.
In this case, the start of the next line should be `//`, with no `@`.
-For example, `//@ !has 'foo/struct.Bar.html'` checks that crate `foo` does not have a page for a struct named `Bar` in the crate root.
+Use the special string `{{channel}}` in XPaths, `PATTERN` arguments and [snapshot files](#snapshot)
+if you'd like to refer to the URL `https://doc.rust-lang.org/CHANNEL` where `CHANNEL` refers to the
+current release channel (e.g, `stable` or `nightly`).
+
+Listed below are all possible directives:
+
+[XPath]: https://en.wikipedia.org/wiki/XPath
### `has`
-Usage 1: `//@ has PATH`
-Usage 2: `//@ has PATH XPATH PATTERN`
+> Usage 1: `//@ has PATH`
-In the first form, `has` checks that a given file exists.
+Check that the file given by `PATH` exists.
-In the second form, `has` is an alias for `matches`,
-except `PATTERN` is a whitespace-normalized[^1] string instead of a regex.
+> Usage 2: `//@ has PATH XPATH PATTERN`
-### `matches`
+Checks that the text of each element / attribute / text selected by `XPATH` in the
+whitespace-normalized[^1] file given by `PATH` matches the
+(also whitespace-normalized) string `PATTERN`.
-Usage: `//@ matches PATH XPATH PATTERN`
-
-Checks that the text of each element selected by `XPATH` in `PATH` matches the python-flavored regex `PATTERN`.
-
-### `matchesraw`
-
-Usage: `//@ matchesraw PATH PATTERN`
-
-Checks that the contents of the file `PATH` matches the regex `PATTERN`.
+**Tip**: If you'd like to avoid whitespace normalization and/or if you'd like to match with a regex,
+use `matches` instead.
### `hasraw`
-Usage: `//@ hasraw PATH PATTERN`
+> Usage: `//@ hasraw PATH PATTERN`
-Same as `matchesraw`, except `PATTERN` is a whitespace-normalized[^1] string instead of a regex.
+Checks that the contents of the whitespace-normalized[^1] file given by `PATH`
+matches the (also whitespace-normalized) string `PATTERN`.
+
+**Tip**: If you'd like to avoid whitespace normalization and / or if you'd like to match with a
+regex, use `matchesraw` instead.
+
+### `matches`
+
+> Usage: `//@ matches PATH XPATH PATTERN`
+
+Checks that the text of each element / attribute / text selected by `XPATH` in the
+file given by `PATH` matches the Python-flavored[^2] regex `PATTERN`.
+
+### `matchesraw`
+
+> Usage: `//@ matchesraw PATH PATTERN`
+
+Checks that the contents of the file given by `PATH` matches the
+Python-flavored[^2] regex `PATTERN`.
### `count`
-Usage: `//@ count PATH XPATH COUNT`
+> Usage: `//@ count PATH XPATH COUNT`
-Checks that there are exactly `COUNT` matches for `XPATH` within the file `PATH`.
+Checks that there are exactly `COUNT` matches for `XPATH` within the file given by `PATH`.
### `snapshot`
-Usage: `//@ snapshot NAME PATH XPATH`
+> Usage: `//@ snapshot NAME PATH XPATH`
-Creates a snapshot test named NAME.
-A snapshot test captures a subtree of the DOM, at the location
-determined by the XPath, and compares it to a pre-recorded value
-in a file. The file's name is the test's name with the `.rs` extension
-replaced with `.NAME.html`, where NAME is the snapshot's name.
+Checks that the element / text selected by `XPATH` in the file given by `PATH` matches the
+pre-recorded subtree or text (the "snapshot") in file `FILE_STEM.NAME.html` where `FILE_STEM`
+is the file stem of the test file.
-htmldocck supports the `--bless` option to accept the current subtree
-as expected, saving it to the file determined by the snapshot's name.
-compiletest's `--bless` flag is forwarded to htmldocck.
+Pass the `--bless` option to `compiletest` to accept the current subtree/text as expected.
+This will overwrite the aforementioned file (or create it if it doesn't exist). It will
+automatically normalize the channel-dependent URL `https://doc.rust-lang.org/CHANNEL` to
+the special string `{{channel}}`.
### `has-dir`
-Usage: `//@ has-dir PATH`
+> Usage: `//@ has-dir PATH`
-Checks for the existence of directory `PATH`.
+Checks for the existence of the directory given by `PATH`.
### `files`
-Usage: `//@ files PATH ENTRIES`
+> Usage: `//@ files PATH ENTRIES`
-Checks that the directory `PATH` contains exactly `ENTRIES`.
-`ENTRIES` is a python list of strings inside a quoted string,
-as if it were to be parsed by `eval`.
-(note that the list is actually parsed by `shlex.split`,
-so it cannot contain arbitrary python expressions).
+Checks that the directory given by `PATH` contains exactly `ENTRIES`.
+`ENTRIES` is a Python-like list of strings inside a quoted string.
-Example: `//@ files "foo/bar" '["index.html", "sidebar-items.js"]'`
+**Example**: `//@ files "foo/bar" '["index.html", "sidebar-items.js"]'`
-[^1]: Whitespace normalization means that all spans of consecutive whitespace are replaced with a single space. The files themselves are also whitespace-normalized.
+[^1]: Whitespace normalization means that all spans of consecutive whitespace are replaced with a single space.
+[^2]: They are Unicode aware (flag `UNICODE` is set), match case-sensitively and in single-line mode.
+
+## Compiletest Directives (Brief)
+
+As mentioned in the introduction, you also have access to [compiletest directives].
+Most importantly, they allow you to register auxiliary crates and
+to pass flags to the `rustdoc` binary under test.
+It's *strongly recommended* to read that chapter if you don't know anything about them yet.
+
+Here are some details that are relevant to this test suite specifically:
+
+* While you can use both `//@ compile-flags` and `//@ doc-flags` to pass flags to `rustdoc`,
+ prefer to user the latter to show intent. The former is meant for `rustc`.
+* Add `//@ build-aux-docs` to the test file that has auxiliary crates to not only compile the
+ auxiliaries with `rustc` but to also document them with `rustdoc`.
+
+## Caveats
+
+Testing for the absence of an element or a piece of text is quite fragile and not very future proof.
+
+It's not unusual that the *shape* of the generated HTML document tree changes from time to time.
+This includes for example renamings of CSS classes.
+
+Whenever that happens, *positive* checks will either continue to match the intended element /
+attribute / text (if their XPath expression is general / loose enough) and
+thus continue to test the correct thing or they won't in which case they would fail thereby
+forcing the author of the change to look at them.
+
+Compare that to *negative* checks (e.g., `//@ !has PATH XPATH PATTERN`) which won't fail if their
+XPath expression "no longer" matches. The author who changed "the shape" thus won't get notified and
+as a result someone else can unintentionally reintroduce `PATTERN` into the generated docs without
+the original negative check failing.
+
+**Note**: Please avoid the use of *negated* checks!
+
+**Tip**: If you can't avoid it, please **always** pair it with an analogous positive check in the
+immediate vicinity, so people changing "the shape" have a chance to notice and to update the
+negated check!
## Limitations
-`htmldocck.py` uses the xpath implementation from the standard library.
+
+HtmlDocCk uses the XPath implementation from the Python standard library.
This leads to several limitations:
+
* All `XPATH` arguments must start with `//` due to a flaw in the implementation.
* Many XPath features (functions, axies, etc.) are not supported.
* Only well-formed HTML can be parsed (hopefully rustdoc doesn't output mismatched tags).
+Furthmore, compiletest [revisions] are not supported.
+
+[revisions]: ../tests/compiletest.md#revisions
+[compiletest directives]: ../tests/directives.md
diff --git a/src/rustdoc.md b/src/rustdoc.md
index de70ba63..52ae48c3 100644
--- a/src/rustdoc.md
+++ b/src/rustdoc.md
@@ -67,43 +67,29 @@ does is call the `main()` that's in this crate's `lib.rs`, though.)
## Code structure
-* All paths in this section are relative to `src/librustdoc` in the rust-lang/rust repository.
+All paths in this section are relative to `src/librustdoc/` in the rust-lang/rust repository.
+
* Most of the HTML printing code is in `html/format.rs` and `html/render/mod.rs`.
- It's in a bunch of `fmt::Display` implementations and supplementary
- functions.
-* The types that got `Display` impls above are defined in `clean/mod.rs`, right
- next to the custom `Clean` trait used to process them out of the rustc HIR.
+ It's in a bunch of functions returning `impl std::fmt::Display`.
+* The data types that get rendered by the functions mentioned above are defined in `clean/types.rs`.
+ The functions responsible for creating them from the `HIR` and the `rustc_middle::ty` IR
+ live in `clean/mod.rs`.
* The bits specific to using rustdoc as a test harness are in
`doctest.rs`.
* The Markdown renderer is loaded up in `html/markdown.rs`, including functions
for extracting doctests from a given block of Markdown.
* Frontend CSS and JavaScript are stored in `html/static/`.
+ * Re. JavaScript, type annotations are written using [TypeScript-flavored JSDoc]
+comments and an external `.d.ts` file.
+ This way, the code itself remains plain, valid JavaScript.
+ We only use `tsc` as a linter.
+
+[TypeScript-flavored JSDoc]: https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html
## Tests
-* Tests on search engine and index are located in `tests/rustdoc-js` and `tests/rustdoc-js-std`.
- The format is specified
- [in the search guide](rustdoc-internals/search.md#testing-the-search-engine).
-* Tests on the "UI" of rustdoc (the terminal output it produces when run) are in
- `tests/rustdoc-ui`
-* Tests on the "GUI" of rustdoc (the HTML, JS, and CSS as rendered in a browser)
- are in `tests/rustdoc-gui`. These use a [NodeJS tool called
- browser-UI-test](https://github.com/GuillaumeGomez/browser-UI-test/) that uses
- puppeteer to run tests in a headless browser and check rendering and
- interactivity. For information on how to write this form of test,
- see [`tests/rustdoc-gui/README.md`][rustdoc-gui-readme]
- as well as [the description of the `.goml` format][goml-script]
-* Tests on the structure of rustdoc HTML output are located in `tests/rustdoc`,
- where they're handled by the test runner of bootstrap and
- the supplementary script `src/etc/htmldocck.py`.
- [These tests have several extra directives available to them](./rustdoc-internals/rustdoc-test-suite.md).
-* Additionally, JavaScript type annotations are written using [TypeScript-flavored JSDoc]
- comments and an external d.ts file. The code itself is plain, valid JavaScript; we only
- use tsc as a linter.
-
-[TypeScript-flavored JSDoc]: https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html
-[rustdoc-gui-readme]: https://github.com/rust-lang/rust/blob/master/tests/rustdoc-gui/README.md
-[goml-script]: https://github.com/GuillaumeGomez/browser-UI-test/blob/master/goml-script.md
+`rustdoc`'s integration tests are split across several test suites.
+See [Rustdoc tests suites](tests/compiletest.md#rustdoc-test-suites) for more details.
## Constraints
diff --git a/src/tests/ci.md b/src/tests/ci.md
index ef3f4c70..96e4edc1 100644
--- a/src/tests/ci.md
+++ b/src/tests/ci.md
@@ -186,9 +186,11 @@ Note that if you start the default try job using `@bors try`, it will skip build
Multiple try builds can execute concurrently across different PRs.
-bors identify try jobs by commit hash. This means that if you have two PRs
+
+Bors identifies try jobs by commit hash. This means that if you have two PRs
containing the same (latest) commits, running `@bors try` will result in the
*same* try job and it really confuses `bors`. Please refrain from doing so.
+
[rustc-perf]: https://github.com/rust-lang/rustc-perf
diff --git a/src/tests/compiletest.md b/src/tests/compiletest.md
index e1b23748..ded30234 100644
--- a/src/tests/compiletest.md
+++ b/src/tests/compiletest.md
@@ -56,6 +56,9 @@ incremental compilation. The various suites are defined in
The following test suites are available, with links for more information:
+[`tests`]: https://github.com/rust-lang/rust/blob/master/tests
+[`src/tools/compiletest/src/common.rs`]: https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/common.rs
+
### Compiler-specific test suites
| Test suite | Purpose |
@@ -71,6 +74,7 @@ The following test suites are available, with links for more information:
| [`mir-opt`](#mir-opt-tests) | Check MIR generation and optimizations |
| [`coverage`](#coverage-tests) | Check coverage instrumentation |
| [`coverage-run-rustdoc`](#coverage-tests) | `coverage` tests that also run instrumented doctests |
+| [`crashes`](#crashes-tests) | Check that the compiler ICEs/panics/crashes on certain inputs to catch accidental fixes |
### General purpose test suite
@@ -78,19 +82,23 @@ The following test suites are available, with links for more information:
### Rustdoc test suites
-See [Rustdoc tests](../rustdoc.md#tests) for more details.
+| Test suite | Purpose |
+|--------------------------------------|--------------------------------------------------------------------------|
+| [`rustdoc`][rustdoc-html-tests] | Check HTML output of `rustdoc` |
+| [`rustdoc-gui`][rustdoc-gui-tests] | Check `rustdoc`'s GUI using a web browser |
+| [`rustdoc-js`][rustdoc-js-tests] | Check `rustdoc`'s search engine and index |
+| [`rustdoc-js-std`][rustdoc-js-tests] | Check `rustdoc`'s search engine and index on the std library docs |
+| [`rustdoc-json`][rustdoc-json-tests] | Check JSON output of `rustdoc` |
+| `rustdoc-ui` | Check terminal output of `rustdoc` ([see also](ui.md)) |
-| Test suite | Purpose |
-|------------------|--------------------------------------------------------------------------|
-| `rustdoc` | Check `rustdoc` generated files contain the expected documentation |
-| `rustdoc-gui` | Check `rustdoc`'s GUI using a web browser |
-| `rustdoc-js` | Check `rustdoc` search is working as expected |
-| `rustdoc-js-std` | Check rustdoc search is working as expected specifically on the std docs |
-| `rustdoc-json` | Check JSON output of `rustdoc` |
-| `rustdoc-ui` | Check terminal output of `rustdoc` |
+Some rustdoc-specific tests can also be found in `ui/rustdoc/`.
+These check rustdoc-related or -specific lints that (also) run as part of `rustc`, not (only) `rustdoc`.
+Run-make tests pertaining to rustdoc are typically named `run-make/rustdoc-*/`.
-[`tests`]: https://github.com/rust-lang/rust/blob/master/tests
-[`src/tools/compiletest/src/common.rs`]: https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/common.rs
+[rustdoc-html-tests]: ../rustdoc-internals/rustdoc-test-suite.md
+[rustdoc-gui-tests]: ../rustdoc-internals/rustdoc-gui-test-suite.md
+[rustdoc-js-tests]: ../rustdoc-internals/search.md#testing-the-search-engine
+[rustdoc-json-tests]: ../rustdoc-internals/rustdoc-json-test-suite.md
### Pretty-printer tests
@@ -107,7 +115,7 @@ default behavior without any commands is to:
2. Run `rustc -Zunpretty=normal` on the output of the previous step.
3. The output of the previous two steps should be the same.
4. Run `rustc -Zno-codegen` on the output to make sure that it can type check
- (this is similar to running `cargo check`).
+ (similar to `cargo check`).
If any of the commands above fail, then the test fails.
@@ -438,7 +446,9 @@ To work around this when working on a particular test, temporarily create a
with these contents:
+
Be careful not to add this `Cargo.toml` or its `Cargo.lock` to your actual PR!
+
```toml
diff --git a/src/tests/directives.md b/src/tests/directives.md
index 8a862417..839076b8 100644
--- a/src/tests/directives.md
+++ b/src/tests/directives.md
@@ -202,6 +202,7 @@ settings:
`//@ needs-crate-type: cdylib, proc-macro` will cause the test to be ignored
on `wasm32-unknown-unknown` target because the target does not support the
`proc-macro` crate type.
+- `needs-target-std` — ignores if target platform does not have std support.
The following directives will check LLVM support:
@@ -248,18 +249,20 @@ ignoring debuggers.
| `no-prefer-dynamic` | Don't use `-C prefer-dynamic`, don't build as a dylib via a `--crate-type=dylib` preset flag | `ui`, `crashes` | N/A |
+
Tests (outside of `run-make`) that want to use incremental tests not in the
incremental test-suite must not pass `-C incremental` via `compile-flags`, and
must instead use the `//@ incremental` directive.
Consider writing the test as a proper incremental test instead.
+
### Rustdoc
| Directive | Explanation | Supported test suites | Possible values |
|-------------|--------------------------------------------------------------|------------------------------------------|---------------------------|
-| `doc-flags` | Flags passed to `rustdoc` when building the test or aux file | `rustdoc`, `rustdoc-js`, `rustdoc-json` | Any valid `rustdoc` flags |
+| `doc-flags` | Flags passed to `rustdoc` when building the test or aux file | `rustdoc`, `rustdoc-js`, `rustdoc-json` | Any valid `rustdoc` flags |
+#### Test-suite-specific directives
+
+The test suites [`rustdoc`][rustdoc-html-tests], [`rustdoc-js`/`rustdoc-js-std`][rustdoc-js-tests]
+and [`rustdoc-json`][rustdoc-json-tests] each feature an additional set of directives whose basic
+syntax resembles the one of compiletest directives but which are ultimately read and checked by
+separate tools. For more information, please read their respective chapters as linked above.
+
+[rustdoc-html-tests]: ../rustdoc-internals/rustdoc-test-suite.md
+[rustdoc-js-tests]: ../rustdoc-internals/search.html#testing-the-search-engine
+[rustdoc-json-tests]: ../rustdoc-internals/rustdoc-json-test-suite.md
+
### Pretty printing
See [Pretty-printer](compiletest.md#pretty-printer-tests).
diff --git a/src/tests/minicore.md b/src/tests/minicore.md
index 507b259e..def9aaf8 100644
--- a/src/tests/minicore.md
+++ b/src/tests/minicore.md
@@ -7,9 +7,11 @@ ui/codegen/assembly test suites. It provides `core` stubs for tests that need to
build for cross-compiled targets but do not need/want to run.
+
Please note that [`minicore`] is only intended for `core` items, and explicitly
**not** `std` or `alloc` items because `core` items are applicable to a wider
range of tests.
+
A test can use [`minicore`] by specifying the `//@ add-core-stubs` directive.
diff --git a/src/tests/running.md b/src/tests/running.md
index 73c38736..6526fe9c 100644
--- a/src/tests/running.md
+++ b/src/tests/running.md
@@ -8,6 +8,7 @@ development because it takes a really long time. For local development, see the
subsection after on how to run a subset of tests.
+
Running plain `./x test` will build the stage 1 compiler and then run the whole
test suite. This not only include `tests/`, but also `library/`, `compiler/`,
`src/tools/` package tests and more.
@@ -16,6 +17,7 @@ You usually only want to run a subset of the test suites (or even a smaller set
of tests than that) which you expect will exercise your changes. PR CI exercises
a subset of test collections, and merge queue CI will exercise all of the test
collection.
+
```text
@@ -116,8 +118,10 @@ By listing which test suites you want to run,
you avoid having to run tests for components you did not change at all.
+
Note that bors only runs the tests with the full stage 2 build; therefore, while
the tests **usually** work fine with stage 1, there are some limitations.
+
### Run all tests using a stage 2 compiler
diff --git a/src/tests/ui.md b/src/tests/ui.md
index 3402838d..25d3efdb 100644
--- a/src/tests/ui.md
+++ b/src/tests/ui.md
@@ -220,8 +220,12 @@ negligible (i.e. there is no semantic difference between `//~ ERROR` and
`//~ERROR` although the former is more common in the codebase).
`~? ` (example being `~? ERROR`)
-is used to match diagnostics without line information.
-These can be placed on any line in the test file, but are conventionally placed at the end.
+is used to match diagnostics _without_ line info at all,
+or where the line info is outside the main test file[^main test file].
+These annotations can be placed on any line in the test file.
+
+[^main test file]: This is a file that has the `~?` annotations,
+as distinct from aux files, or sources that we have no control over.
### Error annotation examples
diff --git a/triagebot.toml b/triagebot.toml
index 978802ed..b3f4c2d2 100644
--- a/triagebot.toml
+++ b/triagebot.toml
@@ -72,6 +72,23 @@ days-threshold = 7
# Documentation at: https://forge.rust-lang.org/triagebot/pr-assignment.html
[assign]
+# NOTE: do not add `[assign.owners]` if we still wish to keep the opt-in
+# reviewer model, as `[assign.owners]` will cause triagebot auto-reviewer
+# assignment to kick in.
+
+# Custom PR welcome message for when no auto reviewer assignment is performed
+# and no explicit manual reviewer selection is made.
+# Documentation at: https://forge.rust-lang.org/triagebot/pr-assignment.html#custom-welcome-messages
+[assign.custom_welcome_messages]
+welcome-message = ""
+welcome-message-no-reviewer = """\
+Thanks for the PR. If you have write access, feel free to merge this PR if it \
+does not need reviews. You can request a review using `r? rustc-dev-guide` or \
+`r? `.
+"""
+
+# Groups for `r? `.
+# Documentation at: https://forge.rust-lang.org/triagebot/pr-assignment.html#usage
# Keep members alphanumerically sorted.
[assign.adhoc_groups]
rustc-dev-guide = [