mirror
/
rustc-dev-guide
mirror of https://github.com/rust-lang/rustc-dev-guide.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Fix long lines in compiletest

This commit is contained in:
Mark Mansi 2018-03-23 21:01:59 -05:00 committed by Who? Me?!
parent a4c3361302
commit fc16c05abe
1 changed files with 109 additions and 56 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

165
src/compiletest.md
View File

@ -1,42 +1,66 @@
# `compiletest` # `compiletest`
## Introduction ## Introduction
`compiletest` is the main test harness of the Rust test suite. It allows `compiletest` is the main test harness of the Rust test suite. It allows
test authors to organize large numbers of tests (the Rust compiler has many test authors to organize large numbers of tests (the Rust compiler has many
thousands), efficient test execution (parallel execution is supported), and thousands), efficient test execution (parallel execution is supported), and
allows the test author to configure behavior and expected results of both allows the test author to configure behavior and expected results of both
individual and groups of tests. individual and groups of tests.
`compiletest` tests may check test code for success, for failure or in some cases, even failure to compile. Tests are `compiletest` tests may check test code for success, for failure or in some
typically organized as a Rust source file with annotations in comments before and/or within the test code, which serve to cases, even failure to compile. Tests are typically organized as a Rust source
direct `compiletest` on if or how to run the test, what behavior to expect, and more. If you are unfamiliar with the compiler file with annotations in comments before and/or within the test code, which
testing framework, see [`this chapter`](./tests/intro.html) for additional background. serve to direct `compiletest` on if or how to run the test, what behavior to
expect, and more. If you are unfamiliar with the compiler testing framework,
see [`this chapter`](./tests/intro.html) for additional background.
The tests themselves are typically (but not always) organized into "suites"--for example, `run-pass`, a folder The tests themselves are typically (but not always) organized into
representing tests that should succeed, `run-fail`, a folder holding tests that should compile successfully, but return "suites"--for example, `run-pass`, a folder representing tests that should
a failure (non-zero status), `compile-fail`, a folder holding tests that should fail to compile, and many more. The various succeed, `run-fail`, a folder holding tests that should compile successfully,
suites are defined in [src/tools/compiletest/src/common.rs](https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/common.rs) in the `pub struct Config` declaration. And a very good but return a failure (non-zero status), `compile-fail`, a folder holding tests
introduction to the different suites of compiler tests along with details about them can be found in [`Adding new tests`](./tests/adding.html). that should fail to compile, and many more. The various suites are defined in
[src/tools/compiletest/src/common.rs][common] in the `pub struct Config`
declaration. And a very good introduction to the different suites of compiler
tests along with details about them can be found in [`Adding new
tests`](./tests/adding.html).
## Adding a new test file ## Adding a new test file
Briefly, simply create your new test in the appropriate location under [src/test](https://github.com/rust-lang/rust/tree/master/src/test). No registration of test files is necessary as
`compiletest` will scan the [src/test](https://github.com/rust-lang/rust/tree/master/src/test) subfolder recursively, and will execute any Rust source files it finds as tests. Briefly, simply create your new test in the appropriate location under
See [`Adding new tests`](./tests/adding.html) for a complete guide on how to adding new tests. [src/test][test]. No registration of test files is necessary as `compiletest`
will scan the [src/test][test] subfolder recursively, and will execute any Rust
source files it finds as tests. See [`Adding new tests`](./tests/adding.html)
for a complete guide on how to adding new tests.
## Header Commands ## Header Commands
Source file annotations which appear in comments near the top of the source file *before* any test code are known as header
commands. These commands can instruct `compiletest` to ignore this test, set expectations on whether it is expected to Source file annotations which appear in comments near the top of the source
succeed at compiling, or what the test's return code is expected to be. Header commands (and their inline counterparts, file *before* any test code are known as header commands. These commands can
Error Info commands) are described more fully [here](./tests/adding.html#header-commands-configuring-rustc). instruct `compiletest` to ignore this test, set expectations on whether it is
expected to succeed at compiling, or what the test's return code is expected to
be. Header commands (and their inline counterparts, Error Info commands) are
described more fully
[here](./tests/adding.html#header-commands-configuring-rustc).
### Adding a new header command ### Adding a new header command
Header commands are defined in the `TestProps` struct in [src/tools/compiletest/src/header.rs](https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/header.rs). At a high level, there are dozens of test properties defined here, all set to default values in the `TestProp` struct's `impl` block. Any test can override this
default value by specifying the property in question as header command as a comment (`//`) in the test source file, before any source code. Header commands are defined in the `TestProps` struct in
[src/tools/compiletest/src/header.rs][header]. At a high level, there are
dozens of test properties defined here, all set to default values in the
`TestProp` struct's `impl` block. Any test can override this default value by
specifying the property in question as header command as a comment (`//`) in
the test source file, before any source code.
#### Using a header command #### Using a header command
Here is an example, specifying the `must-compile-successfully` header command, which takes no arguments, followed by the
`failure-status` header command, which takes a single argument (which, in this case is a value of 1). `failure-status` is Here is an example, specifying the `must-compile-successfully` header command,
instructing `compiletest` to expect a failure status of 1 (rather than the current Rust default of 101 at the time of this which takes no arguments, followed by the `failure-status` header command,
writing). The header command and the argument list (if present) are typically separated by a colon: which takes a single argument (which, in this case is a value of 1).
`failure-status` is instructing `compiletest` to expect a failure status of 1
(rather than the current Rust default of 101 at the time of this writing). The
header command and the argument list (if present) are typically separated by a
colon:
``` ```
// Copyright 2018 The Rust Project Developers. See the COPYRIGHT // Copyright 2018 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at // file at the top-level directory of this distribution and at
@ -61,36 +85,51 @@ fn main() -> Result<(), Box<Error>> {
``` ```
#### Adding a new header command property #### Adding a new header command property
One would add a new header command if there is a need to define some test property or behavior on an individual, test-by-test
basis. A header command property serves as the header command's backing store (holds the command's current value) at One would add a new header command if there is a need to define some test
runtime. property or behavior on an individual, test-by-test basis. A header command
property serves as the header command's backing store (holds the command's
current value) at runtime.
To add a new header command property: To add a new header command property:
1. Look for the `pub struct TestProps` declaration in [src/tools/compiletest/src/header.rs](https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/header.rs) and add 1. Look for the `pub struct TestProps` declaration in
the new public property to the end of the declaration. [src/tools/compiletest/src/header.rs][header] and add the new public
2. Look for the `impl TestProps` implementation block immediately following the struct declaration and initialize the new property to the end of the declaration.
property to its default value. 2. Look for the `impl TestProps` implementation block immediately following
the struct declaration and initialize the new property to its default
value.
#### Adding a new header command parser #### Adding a new header command parser
When `compiletest` encounters a test file, it parses the file a line at a time by calling every parser defined in the
`Config` struct's implementation block, also in [src/tools/compiletest/src/header.rs](https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/header.rs) (note the `Config` struct's declaration
block is found in [src/tools/compiletest/src/common.rs](https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/common.rs). `TestProps`'s `load_from()` method will try passing the current
line of text to each parser, which, in turn typically checks to see if the line begins with a particular commented (`//`)
header command such as `// must-compile-successfully` or `// failure-status`. Whitespace after the comment marker is
optional.
Parsers will override a given header command property's default value merely by being specified in the test file as a header When `compiletest` encounters a test file, it parses the file a line at a time
command or by having a parameter value specified in the test file, depending on the header command. by calling every parser defined in the `Config` struct's implementation block,
also in [src/tools/compiletest/src/header.rs][header] (note the `Config`
struct's declaration block is found in
[src/tools/compiletest/src/common.rs][common]. `TestProps`'s `load_from()`
method will try passing the current line of text to each parser, which, in turn
typically checks to see if the line begins with a particular commented (`//`)
header command such as `// must-compile-successfully` or `// failure-status`.
Whitespace after the comment marker is optional.
Parsers defined in `impl Config` are typically named `parse_<header_command>` (note kebab-case `<header-command>` transformed Parsers will override a given header command property's default value merely by
to snake-case `<header_command>`). `impl Config` also defines several 'low-level' parsers which make it simple to parse being specified in the test file as a header command or by having a parameter
common patterns like simple presence or not (`parse_name_directive()`), header-command:parameter(s) value specified in the test file, depending on the header command.
(`parse_name_value_directive()`), optional parsing only if a particular `cfg` attribute is defined (`has_cfg_prefix()`) and
many more. The low-level parsers are found near the end of the `impl Config` block; be sure to look through them and their Parsers defined in `impl Config` are typically named `parse_<header_command>`
associated parsers immediately above to see how they are used to avoid writing additional parsing code unneccessarily. (note kebab-case `<header-command>` transformed to snake-case
`<header_command>`). `impl Config` also defines several 'low-level' parsers
which make it simple to parse common patterns like simple presence or not
(`parse_name_directive()`), header-command:parameter(s)
(`parse_name_value_directive()`), optional parsing only if a particular `cfg`
attribute is defined (`has_cfg_prefix()`) and many more. The low-level parsers
are found near the end of the `impl Config` block; be sure to look through them
and their associated parsers immediately above to see how they are used to
avoid writing additional parsing code unneccessarily.
As a concrete example, here is the implementation for the
`parse_failure_status()` parser, in
[src/tools/compiletest/src/header.rs][header]:
As a concrete example, here is the implementation for the `parse_failure_status()` parser, in
[src/tools/compiletest/src/header.rs](https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/header.rs):
```diff ```diff
@@ -232,6 +232,7 @@ pub struct TestProps { @@ -232,6 +232,7 @@ pub struct TestProps {
// customized normalization rules // customized normalization rules
@ -132,14 +171,21 @@ As a concrete example, here is the implementation for the `parse_failure_status(
``` ```
## Implementing the behavior change ## Implementing the behavior change
When a test invokes a particular header command, it is expected that some behavior will change as a result. What behavior,
obviously, will depend on the purpose of the header command. In the case of `failure-status`, the behavior that changes
is that `compiletest` expects the failure code defined by the header command invoked in the test, rather than the default
value.
Although specific to `failure-status` (as every header command will have a different implementation in order to invoke When a test invokes a particular header command, it is expected that some
behavior change) perhaps it is helpful to see the behavior change implementation of one case, simply as an example. To implement `failure-status`, the `check_correct_failure_status()` function found in the `TestCx` implementation block, behavior will change as a result. What behavior, obviously, will depend on the
located in [src/tools/compiletest/src/runtest.rs](https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/runtest.rs), was modified as per below: purpose of the header command. In the case of `failure-status`, the behavior
that changes is that `compiletest` expects the failure code defined by the
header command invoked in the test, rather than the default value.
Although specific to `failure-status` (as every header command will have a
different implementation in order to invoke behavior change) perhaps it is
helpful to see the behavior change implementation of one case, simply as an
example. To implement `failure-status`, the `check_correct_failure_status()`
function found in the `TestCx` implementation block, located in
[src/tools/compiletest/src/runtest.rs](https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/runtest.rs),
was modified as per below:
```diff ```diff
@@ -295,11 +295,14 @@ impl<'test> TestCx<'test> { @@ -295,11 +295,14 @@ impl<'test> TestCx<'test> {
} }
@ -176,7 +222,14 @@ located in [src/tools/compiletest/src/runtest.rs](https://github.com/rust-lang/r
} }
} }
``` ```
Note the use of `self.props.failure_status` to access the header command property. In tests which do not specify the failure Note the use of `self.props.failure_status` to access the header command
status header command, `self.props.failure_status` will evaluate to the default value of 101 at the time of this writing. property. In tests which do not specify the failure status header command,
But for a test which specifies a header command of, for example, `// failure-status: 1`, `self.props.failure_status` will `self.props.failure_status` will evaluate to the default value of 101 at the
evaluate to 1, as `parse_failure_status()` will have overridden the `TestProps` default value, for that test specifically. time of this writing. But for a test which specifies a header command of, for
example, `// failure-status: 1`, `self.props.failure_status` will evaluate to
1, as `parse_failure_status()` will have overridden the `TestProps` default
value, for that test specifically.
[test]: https://github.com/rust-lang/rust/tree/master/src/test
[header]: https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/header.rs
[common]: https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/common.rs