Update documentation for coverage tests
This commit is contained in:
Zalathar
Tshepang Mbambo
parent
b025ec19a2
commit
a19ce0794b
|
|
@ -273,20 +273,22 @@ since it will not be called), and adds a new `FunctionCoverage`, with
|
|||
|
||||
## Testing LLVM Coverage
|
||||
|
||||
[(See also the compiletest documentation for the `tests/coverage-map` and
|
||||
`tests/run-coverage` test suites.)](./tests/compiletest.md#coverage-tests)
|
||||
[(See also the compiletest documentation for the `tests/coverage`
|
||||
test suite.)](./tests/compiletest.md#coverage-tests)
|
||||
|
||||
Coverage instrumentation in the MIR is validated by a `mir-opt` test:
|
||||
[`tests/mir-opt/instrument_coverage.rs`].
|
||||
|
||||
Coverage instrumentation in LLVM IR is validated by the [`tests/coverage-map`]
|
||||
test suite. These tests compile a test program to LLVM IR assembly, and then
|
||||
Coverage instrumentation in LLVM IR is validated by the [`tests/coverage`]
|
||||
test suite in `coverage-map` mode.
|
||||
These tests compile a test program to LLVM IR assembly, and then
|
||||
use the [`src/tools/coverage-dump`] tool to extract and pretty-print the
|
||||
coverage mappings that would be embedded in the final binary.
|
||||
|
||||
End-to-end testing of coverage instrumentation and coverage reporting is
|
||||
performed by the [`tests/run-coverage`] and [`tests/run-coverage-rustdoc`]
|
||||
test suites. These tests compile and run a test program with coverage
|
||||
performed by the [`tests/coverage`] test suite in `coverage-run` mode,
|
||||
and by the [`tests/coverage-run-rustdoc`] test suite.
|
||||
These tests compile and run a test program with coverage
|
||||
instrumentation, then use LLVM tools to convert the coverage data into a
|
||||
human-readable coverage report.
|
||||
|
||||
|
|
@ -296,19 +298,19 @@ expected LLVM IR instructions and structured data for a coverage-enabled
|
|||
program, including various checks for Coverage Map-related metadata and the LLVM
|
||||
intrinsic calls to increment the runtime counters.
|
||||
|
||||
Expected results for the `coverage-map`, `run-coverage`, `run-coverage-rustdoc`,
|
||||
Expected results for the `coverage`, `coverage-run-rustdoc`,
|
||||
and `mir-opt` tests can be refreshed by running:
|
||||
|
||||
```shell
|
||||
./x test tests/*coverage* --bless
|
||||
./x test coverage --bless
|
||||
./x test coverage-run-rustdoc --bless
|
||||
./x test tests/mir-opt --bless
|
||||
```
|
||||
|
||||
[`tests/mir-opt/instrument_coverage.rs`]: https://github.com/rust-lang/rust/blob/master/tests/mir-opt/instrument_coverage.rs
|
||||
[`tests/coverage-map`]: https://github.com/rust-lang/rust/tree/master/tests/coverage-map
|
||||
[`tests/coverage`]: https://github.com/rust-lang/rust/tree/master/tests/coverage
|
||||
[`src/tools/coverage-dump`]: https://github.com/rust-lang/rust/tree/master/src/tools/coverage-dump
|
||||
[`tests/run-coverage`]: https://github.com/rust-lang/rust/tree/master/tests/run-coverage
|
||||
[`tests/run-coverage-rustdoc`]: https://github.com/rust-lang/rust/tree/master/tests/run-coverage-rustdoc
|
||||
[`tests/coverage-run-rustdoc`]: https://github.com/rust-lang/rust/tree/master/tests/coverage-run-rustdoc
|
||||
[`coverage-llvmir`]: https://github.com/rust-lang/rust/tree/master/tests/run-make/coverage-llvmir
|
||||
|
||||
## Implementation Details of the `InstrumentCoverage` MIR Pass
|
||||
|
|
|
|||
|
|
@ -67,11 +67,8 @@ The following test suites are available, with links for more information:
|
|||
- `run-make-fulldeps` — `run-make` tests which require a linkable build of `rustc`,
|
||||
or the rust demangler
|
||||
- [`run-pass-valgrind`](#valgrind-tests) — tests run with Valgrind
|
||||
- [`coverage-map`](#coverage-tests) - tests for coverage maps produced by
|
||||
coverage instrumentation
|
||||
- [`run-coverage`](#coverage-tests) - tests that run an instrumented program
|
||||
and check its coverage report
|
||||
- [`run-coverage-rustdoc`](#coverage-tests) - coverage tests that also run
|
||||
- [`coverage`](#coverage-tests) - tests for coverage instrumentation
|
||||
- [`coverage-run-rustdoc`](#coverage-tests) - coverage tests that also run
|
||||
instrumented doctests
|
||||
- [Rustdoc tests](../rustdoc.md#tests):
|
||||
- `rustdoc` — tests for rustdoc, making sure that the generated files
|
||||
|
|
@ -402,7 +399,28 @@ These may be removed in the future.
|
|||
|
||||
### Coverage tests
|
||||
|
||||
The tests in [`tests/coverage-map`] test the mappings between source code
|
||||
The tests in [`tests/coverage`] are shared by multiple test modes that test
|
||||
coverage instrumentation in different ways.
|
||||
Running the `coverage` test suite will automatically run each test in all of
|
||||
the different coverage modes.
|
||||
|
||||
Each mode also has an alias to run the coverage tests in just that mode:
|
||||
|
||||
```bash
|
||||
./x test coverage # runs all of tests/coverage in all coverage modes
|
||||
./x test tests/coverage # same as above
|
||||
|
||||
./x test tests/coverage/if.rs # runs the specified test in all coverage modes
|
||||
|
||||
./x test coverage-map # runs all of tests/coverage in "coverage-map" mode only
|
||||
./x test coverage-run # runs all of tests/coverage in "coverage-run" mode only
|
||||
|
||||
./x test coverage-map -- tests/coverage/if.rs # runs the specified test in "coverage-map" mode only
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
In `coverage-map` mode, these tests verify the mappings between source code
|
||||
regions and coverage counters that are emitted by LLVM.
|
||||
They compile the test with `--emit=llvm-ir`,
|
||||
then use a custom tool ([`src/tools/coverage-dump`])
|
||||
|
|
@ -416,18 +434,18 @@ coverage reports.
|
|||
|
||||
As a rule of thumb, any PR that doesn't change coverage-specific
|
||||
code should **feel free to re-bless** the `coverage-map` tests as necessary,
|
||||
without worrying about the actual changes, as long as the `run-coverage` tests
|
||||
without worrying about the actual changes, as long as the `coverage-run` tests
|
||||
still pass.
|
||||
|
||||
---
|
||||
|
||||
The tests in [`tests/run-coverage`] perform an end-to-end test of coverage reporting.
|
||||
In `coverage-run` mode, these tests perform an end-to-end test of coverage reporting.
|
||||
They compile a test program with coverage instrumentation, run that program to
|
||||
produce raw coverage data, and then use LLVM tools to process that data into a
|
||||
human-readable code coverage report.
|
||||
|
||||
Instrumented binaries need to be linked against the LLVM profiler runtime,
|
||||
so `run-coverage` tests are **automatically skipped**
|
||||
so `coverage-run` tests are **automatically skipped**
|
||||
unless the profiler runtime is enabled in `config.toml`:
|
||||
|
||||
```toml
|
||||
|
|
@ -439,14 +457,15 @@ profiler = true
|
|||
This also means that they typically don't run in PR CI jobs,
|
||||
though they do run as part of the full set of CI jobs used for merging.
|
||||
|
||||
The tests in [`tests/run-coverage-rustdoc`] also run instrumented doctests and
|
||||
include them in the coverage report. This avoids having to build rustdoc when
|
||||
only running the main `run-coverage` suite.
|
||||
---
|
||||
|
||||
[`tests/coverage-map`]: https://github.com/rust-lang/rust/tree/master/tests/coverage-map
|
||||
The tests in [`tests/coverage-run-rustdoc`] also run instrumented doctests and
|
||||
include them in the coverage report. This avoids having to build rustdoc when
|
||||
only running the main `coverage` suite.
|
||||
|
||||
[`tests/coverage`]: https://github.com/rust-lang/rust/tree/master/tests/coverage
|
||||
[`src/tools/coverage-dump`]: https://github.com/rust-lang/rust/tree/master/src/tools/coverage-dump
|
||||
[`tests/run-coverage`]: https://github.com/rust-lang/rust/tree/master/tests/run-coverage
|
||||
[`tests/run-coverage-rustdoc`]: https://github.com/rust-lang/rust/tree/master/tests/run-coverage-rustdoc
|
||||
[`tests/coverage-run-rustdoc`]: https://github.com/rust-lang/rust/tree/master/tests/coverage-run-rustdoc
|
||||
|
||||
|
||||
## Building auxiliary crates
|
||||
|
|
|
|||
Loading…
Reference in New Issue