Update documentation for coverage tests
This commit is contained in:
parent
b025ec19a2
commit
a19ce0794b
|
|
@ -273,20 +273,22 @@ since it will not be called), and adds a new `FunctionCoverage`, with
|
||||||
|
|
||||||
## Testing LLVM Coverage
|
## Testing LLVM Coverage
|
||||||
|
|
||||||
[(See also the compiletest documentation for the `tests/coverage-map` and
|
[(See also the compiletest documentation for the `tests/coverage`
|
||||||
`tests/run-coverage` test suites.)](./tests/compiletest.md#coverage-tests)
|
test suite.)](./tests/compiletest.md#coverage-tests)
|
||||||
|
|
||||||
Coverage instrumentation in the MIR is validated by a `mir-opt` test:
|
Coverage instrumentation in the MIR is validated by a `mir-opt` test:
|
||||||
[`tests/mir-opt/instrument_coverage.rs`].
|
[`tests/mir-opt/instrument_coverage.rs`].
|
||||||
|
|
||||||
Coverage instrumentation in LLVM IR is validated by the [`tests/coverage-map`]
|
Coverage instrumentation in LLVM IR is validated by the [`tests/coverage`]
|
||||||
test suite. These tests compile a test program to LLVM IR assembly, and then
|
test suite in `coverage-map` mode.
|
||||||
|
These tests compile a test program to LLVM IR assembly, and then
|
||||||
use the [`src/tools/coverage-dump`] tool to extract and pretty-print the
|
use the [`src/tools/coverage-dump`] tool to extract and pretty-print the
|
||||||
coverage mappings that would be embedded in the final binary.
|
coverage mappings that would be embedded in the final binary.
|
||||||
|
|
||||||
End-to-end testing of coverage instrumentation and coverage reporting is
|
End-to-end testing of coverage instrumentation and coverage reporting is
|
||||||
performed by the [`tests/run-coverage`] and [`tests/run-coverage-rustdoc`]
|
performed by the [`tests/coverage`] test suite in `coverage-run` mode,
|
||||||
test suites. These tests compile and run a test program with coverage
|
and by the [`tests/coverage-run-rustdoc`] test suite.
|
||||||
|
These tests compile and run a test program with coverage
|
||||||
instrumentation, then use LLVM tools to convert the coverage data into a
|
instrumentation, then use LLVM tools to convert the coverage data into a
|
||||||
human-readable coverage report.
|
human-readable coverage report.
|
||||||
|
|
||||||
|
|
@ -296,19 +298,19 @@ expected LLVM IR instructions and structured data for a coverage-enabled
|
||||||
program, including various checks for Coverage Map-related metadata and the LLVM
|
program, including various checks for Coverage Map-related metadata and the LLVM
|
||||||
intrinsic calls to increment the runtime counters.
|
intrinsic calls to increment the runtime counters.
|
||||||
|
|
||||||
Expected results for the `coverage-map`, `run-coverage`, `run-coverage-rustdoc`,
|
Expected results for the `coverage`, `coverage-run-rustdoc`,
|
||||||
and `mir-opt` tests can be refreshed by running:
|
and `mir-opt` tests can be refreshed by running:
|
||||||
|
|
||||||
```shell
|
```shell
|
||||||
./x test tests/*coverage* --bless
|
./x test coverage --bless
|
||||||
|
./x test coverage-run-rustdoc --bless
|
||||||
./x test tests/mir-opt --bless
|
./x test tests/mir-opt --bless
|
||||||
```
|
```
|
||||||
|
|
||||||
[`tests/mir-opt/instrument_coverage.rs`]: https://github.com/rust-lang/rust/blob/master/tests/mir-opt/instrument_coverage.rs
|
[`tests/mir-opt/instrument_coverage.rs`]: https://github.com/rust-lang/rust/blob/master/tests/mir-opt/instrument_coverage.rs
|
||||||
[`tests/coverage-map`]: https://github.com/rust-lang/rust/tree/master/tests/coverage-map
|
[`tests/coverage`]: https://github.com/rust-lang/rust/tree/master/tests/coverage
|
||||||
[`src/tools/coverage-dump`]: https://github.com/rust-lang/rust/tree/master/src/tools/coverage-dump
|
[`src/tools/coverage-dump`]: https://github.com/rust-lang/rust/tree/master/src/tools/coverage-dump
|
||||||
[`tests/run-coverage`]: https://github.com/rust-lang/rust/tree/master/tests/run-coverage
|
[`tests/coverage-run-rustdoc`]: https://github.com/rust-lang/rust/tree/master/tests/coverage-run-rustdoc
|
||||||
[`tests/run-coverage-rustdoc`]: https://github.com/rust-lang/rust/tree/master/tests/run-coverage-rustdoc
|
|
||||||
[`coverage-llvmir`]: https://github.com/rust-lang/rust/tree/master/tests/run-make/coverage-llvmir
|
[`coverage-llvmir`]: https://github.com/rust-lang/rust/tree/master/tests/run-make/coverage-llvmir
|
||||||
|
|
||||||
## Implementation Details of the `InstrumentCoverage` MIR Pass
|
## Implementation Details of the `InstrumentCoverage` MIR Pass
|
||||||
|
|
|
||||||
|
|
@ -67,11 +67,8 @@ The following test suites are available, with links for more information:
|
||||||
- `run-make-fulldeps` — `run-make` tests which require a linkable build of `rustc`,
|
- `run-make-fulldeps` — `run-make` tests which require a linkable build of `rustc`,
|
||||||
or the rust demangler
|
or the rust demangler
|
||||||
- [`run-pass-valgrind`](#valgrind-tests) — tests run with Valgrind
|
- [`run-pass-valgrind`](#valgrind-tests) — tests run with Valgrind
|
||||||
- [`coverage-map`](#coverage-tests) - tests for coverage maps produced by
|
- [`coverage`](#coverage-tests) - tests for coverage instrumentation
|
||||||
coverage instrumentation
|
- [`coverage-run-rustdoc`](#coverage-tests) - coverage tests that also run
|
||||||
- [`run-coverage`](#coverage-tests) - tests that run an instrumented program
|
|
||||||
and check its coverage report
|
|
||||||
- [`run-coverage-rustdoc`](#coverage-tests) - coverage tests that also run
|
|
||||||
instrumented doctests
|
instrumented doctests
|
||||||
- [Rustdoc tests](../rustdoc.md#tests):
|
- [Rustdoc tests](../rustdoc.md#tests):
|
||||||
- `rustdoc` — tests for rustdoc, making sure that the generated files
|
- `rustdoc` — tests for rustdoc, making sure that the generated files
|
||||||
|
|
@ -402,7 +399,28 @@ These may be removed in the future.
|
||||||
|
|
||||||
### Coverage tests
|
### Coverage tests
|
||||||
|
|
||||||
The tests in [`tests/coverage-map`] test the mappings between source code
|
The tests in [`tests/coverage`] are shared by multiple test modes that test
|
||||||
|
coverage instrumentation in different ways.
|
||||||
|
Running the `coverage` test suite will automatically run each test in all of
|
||||||
|
the different coverage modes.
|
||||||
|
|
||||||
|
Each mode also has an alias to run the coverage tests in just that mode:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./x test coverage # runs all of tests/coverage in all coverage modes
|
||||||
|
./x test tests/coverage # same as above
|
||||||
|
|
||||||
|
./x test tests/coverage/if.rs # runs the specified test in all coverage modes
|
||||||
|
|
||||||
|
./x test coverage-map # runs all of tests/coverage in "coverage-map" mode only
|
||||||
|
./x test coverage-run # runs all of tests/coverage in "coverage-run" mode only
|
||||||
|
|
||||||
|
./x test coverage-map -- tests/coverage/if.rs # runs the specified test in "coverage-map" mode only
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
In `coverage-map` mode, these tests verify the mappings between source code
|
||||||
regions and coverage counters that are emitted by LLVM.
|
regions and coverage counters that are emitted by LLVM.
|
||||||
They compile the test with `--emit=llvm-ir`,
|
They compile the test with `--emit=llvm-ir`,
|
||||||
then use a custom tool ([`src/tools/coverage-dump`])
|
then use a custom tool ([`src/tools/coverage-dump`])
|
||||||
|
|
@ -416,18 +434,18 @@ coverage reports.
|
||||||
|
|
||||||
As a rule of thumb, any PR that doesn't change coverage-specific
|
As a rule of thumb, any PR that doesn't change coverage-specific
|
||||||
code should **feel free to re-bless** the `coverage-map` tests as necessary,
|
code should **feel free to re-bless** the `coverage-map` tests as necessary,
|
||||||
without worrying about the actual changes, as long as the `run-coverage` tests
|
without worrying about the actual changes, as long as the `coverage-run` tests
|
||||||
still pass.
|
still pass.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
The tests in [`tests/run-coverage`] perform an end-to-end test of coverage reporting.
|
In `coverage-run` mode, these tests perform an end-to-end test of coverage reporting.
|
||||||
They compile a test program with coverage instrumentation, run that program to
|
They compile a test program with coverage instrumentation, run that program to
|
||||||
produce raw coverage data, and then use LLVM tools to process that data into a
|
produce raw coverage data, and then use LLVM tools to process that data into a
|
||||||
human-readable code coverage report.
|
human-readable code coverage report.
|
||||||
|
|
||||||
Instrumented binaries need to be linked against the LLVM profiler runtime,
|
Instrumented binaries need to be linked against the LLVM profiler runtime,
|
||||||
so `run-coverage` tests are **automatically skipped**
|
so `coverage-run` tests are **automatically skipped**
|
||||||
unless the profiler runtime is enabled in `config.toml`:
|
unless the profiler runtime is enabled in `config.toml`:
|
||||||
|
|
||||||
```toml
|
```toml
|
||||||
|
|
@ -439,14 +457,15 @@ profiler = true
|
||||||
This also means that they typically don't run in PR CI jobs,
|
This also means that they typically don't run in PR CI jobs,
|
||||||
though they do run as part of the full set of CI jobs used for merging.
|
though they do run as part of the full set of CI jobs used for merging.
|
||||||
|
|
||||||
The tests in [`tests/run-coverage-rustdoc`] also run instrumented doctests and
|
---
|
||||||
include them in the coverage report. This avoids having to build rustdoc when
|
|
||||||
only running the main `run-coverage` suite.
|
|
||||||
|
|
||||||
[`tests/coverage-map`]: https://github.com/rust-lang/rust/tree/master/tests/coverage-map
|
The tests in [`tests/coverage-run-rustdoc`] also run instrumented doctests and
|
||||||
|
include them in the coverage report. This avoids having to build rustdoc when
|
||||||
|
only running the main `coverage` suite.
|
||||||
|
|
||||||
|
[`tests/coverage`]: https://github.com/rust-lang/rust/tree/master/tests/coverage
|
||||||
[`src/tools/coverage-dump`]: https://github.com/rust-lang/rust/tree/master/src/tools/coverage-dump
|
[`src/tools/coverage-dump`]: https://github.com/rust-lang/rust/tree/master/src/tools/coverage-dump
|
||||||
[`tests/run-coverage`]: https://github.com/rust-lang/rust/tree/master/tests/run-coverage
|
[`tests/coverage-run-rustdoc`]: https://github.com/rust-lang/rust/tree/master/tests/coverage-run-rustdoc
|
||||||
[`tests/run-coverage-rustdoc`]: https://github.com/rust-lang/rust/tree/master/tests/run-coverage-rustdoc
|
|
||||||
|
|
||||||
|
|
||||||
## Building auxiliary crates
|
## Building auxiliary crates
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue