From c13c5a019faab37ee03b259688bf89e73a2ce4d9 Mon Sep 17 00:00:00 2001 From: John Renner Date: Tue, 24 Jul 2018 08:58:25 -0700 Subject: [PATCH] Add testing chapter --- src/SUMMARY.md | 1 + src/testing.md | 145 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 146 insertions(+) create mode 100644 src/testing.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 206bf5d3..27a6e38f 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -17,6 +17,7 @@ - [Incremental compilation](./incremental-compilation.md) - [Debugging and Testing](./incrcomp-debugging.md) - [The parser](./the-parser.md) +- [Testing](./testing.md) - [Macro expansion](./macro-expansion.md) - [Name resolution](./name-resolution.md) - [The HIR (High-level IR)](./hir.md) diff --git a/src/testing.md b/src/testing.md new file mode 100644 index 00000000..abe6849a --- /dev/null +++ b/src/testing.md @@ -0,0 +1,145 @@ +### The `#[test]` attribute +Today, rust programmers rely on a built in attribute called `#[test]`. +All you have to do is mark a function as a test and include some asserts like so: + +```rust,ignore +#[test] +fn my_test() { + assert!(2+2 == 4); +} +``` + +When this program is compiled using `rustc --test` or `cargo test`, it will +produce an executable that can run this, and any other test function. This +method of testing allows tests to live alongside code in an organic way. You +can even put tests inside private modules: + +```rust,ignore +mod my_priv_mod { + fn my_priv_func() -> bool {} + + #[test] + fn test_priv_func() { + assert!(my_priv_func()); + } +} +``` +Private items can thus be easily tested without worrying about how to expose +the them to any sort of external testing apparatus. This is key to the +ergonomics of testing in Rust. Semantically, however, it's rather odd. +How does any sort of `main` function invoke these tests if they're not visible? +What exactly is `rustc --test` doing? + +`#[test]` is implemented as a syntactic transformation inside the compiler's +[`libsyntax` crate][libsyntax]. Essentially, it's a fancy macro, that +rewrites the crate in 3 steps: + +#### Step 1: Re-Exporting + +As mentioned earlier, tests can exist inside private modules, so we need a way of +exposing them to the main function, without breaking any existing code. To that end, +`libsyntax` will create local modules called `__test_reexports` that recursively reexport tests. +This expansion translates the above example into: + +```rust,ignore +mod my_priv_mod { + fn my_priv_func() -> bool {} + + pub fn test_priv_func() { + assert!(my_priv_func()); + } + + pub mod __test_reexports { + pub use super::test_priv_func; + } +} +``` + +Now, our test can be accessed as +`my_priv_mod::__test_reexports::test_priv_func`. For deeper module +structures, `__test_reexports` will reexport modules that contain tests, so a +test at `a::b::my_test` becomes +`a::__test_reexports::b::__test_reexports::my_test`. While this process seems +pretty safe, what happens if there is an existing `__test_reexports` module? +The answer: nothing. + +To explain, we need to understand [how the AST represents +identifiers][Ident]. The name of every function, variable, module, etc. is +not stored as a string, but rather as an opaque [Symbol][Symbol] which is +essentially an ID number for each identifier. The compiler keeps a separate +hashtable that allows us to recover the human-readable name of a Symbol when +necessary (such as when printing a syntax error). When the compiler generates +the `__test_reexports` module, it generates a new Symbol for the identifier, +so while the compiler-generated `__test_reexports` may share a name with your +hand-written one, it will not share a Symbol. This technique prevents name +collision during code generation and is the foundation of Rust's macro +hygiene. + +#### Step 2: Harness Generation +Now that our tests are accessible from the root of our crate, we need to do something with them. +`libsyntax` generates a module like so: + +```rust,ignore +pub mod __test { + extern crate test; + const TESTS: &'static [self::test::TestDescAndFn] = &[/*...*/]; + + #[main] + pub fn main() { + self::test::test_static_main(TESTS); + } +} +``` + +While this transformation is simple, it gives us a lot of insight into how tests are actually run. +The tests are aggregated into an array and passed to a test runner called `test_static_main`. +We'll come back to exactly what `TestDescAndFn` is, but for now, the key takeaway is that there is a crate +called [`test`][test] that is part of Rust core, that implements all of the runtime for testing. `test`'s interface is unstable, +so the only stable way to interact with it is through the `#[test]` macro. + +#### Step 3: Test Object Generation +If you've written tests in Rust before, you may be familiar with some of the optional attributes available on test functions. +For example, a test can be annotated with `#[should_panic]` if we expect the test to cause a panic. It looks something like this: + +```rust,ignore +#[test] +#[should_panic] +fn foo() { + panic!("intentional"); +} +``` + +This means our tests are more than just simple functions, they have configuration information as well. `test` encodes this configuration +data into a struct called [`TestDesc`][TestDesc]. For each test function in a crate, `libsyntax` will parse its attributes and generate a `TestDesc` instance. +It then combines the `TestDesc` and test function into the predictably named `TestDescAndFn` struct, that `test_static_main` operates on. +For a given test, the generated `TestDescAndFn` instance looks like so: + +```rust,ignore +self::test::TestDescAndFn{ + desc: self::test::TestDesc{ + name: self::test::StaticTestName("foo"), + ignore: false, + should_panic: self::test::ShouldPanic::Yes, + allow_fail: false, + }, + testfn: self::test::StaticTestFn(|| + self::test::assert_test_result(::crate::__test_reexports::foo())), +} +``` + +Once we've constructed an array of these test objects, they're passed to the +test runner via the harness generated in step 2. + +### Inspecting the generated code +On nightly rust, there's an unstable flag called `unpretty` that you can use to print out the module source after macro expansion: + +```bash +$ rustc my_mod.rs -Z unpretty=hir +``` + +[test]: https://doc.rust-lang.org/test/index.html +[TestDesc]: https://doc.rust-lang.org/test/struct.TestDesc.html +[Symbol]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/struct.Ident.html +[Ident]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/struct.Ident.html +[eRFC]: https://github.com/rust-lang/rfcs/blob/master/text/2318-custom-test-frameworks.md +[libsyntax]: https://github.com/rust-lang/rust/tree/master/src/libsyntax \ No newline at end of file