mirror
/
rustc-dev-guide
mirror of https://github.com/rust-lang/rustc-dev-guide.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Some edits to address review comments

This commit is contained in:
Mark Mansi 2019-03-23 13:47:03 -05:00 committed by Who? Me?!
parent 01882f9f56
commit 0999bcd79d
1 changed files with 113 additions and 59 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

172
src/closure.md
View File

@ -1,8 +1,24 @@
# Closure Expansion in rustc # Closure Expansion in rustc
Let's start with a few examples This section describes how rustc handles closures. Closures in Rust are
effectively "desugared" into structs that contain the values they use (or
references to the values they use) from their creator's stack frame. rustc has
the job of figuring out which values a closure uses and how, so it can decide
whether to capture a given variable by shared reference, mutable reference, or
by move. rustc also has to figure out which the closure traits ([`Fn`][fn],
[`FnMut`][fn_mut], or [`FnOnce`][fn_once]) a closure is capable of
implementing.
[fn]: https://doc.rust-lang.org/std/ops/trait.Fn.html
[fn_mut]:https://doc.rust-lang.org/std/ops/trait.FnMut.html
[fn_once]: https://doc.rust-lang.org/std/ops/trait.FnOnce.html
Let's start with a few examples:
### Example 1 ### Example 1
To start, let's take a look at how the closure in the following example is desugared:
```rust ```rust
fn closure(f: impl Fn()) { fn closure(f: impl Fn()) {
f(); f();
@ -14,22 +30,37 @@ fn main() {
println!("Value of x after return {}", x); println!("Value of x after return {}", x);
} }
``` ```
Let's say the above is the content of a file called immut.rs. If we compile immut.rs using the command
Let's say the above is the content of a file called `immut.rs`. If we compile
`immut.rs` using the following command. The [`-Zdump-mir=all`][dump-mir] flag will cause
`rustc` to generate and dump the [MIR][mir] to a directory called `mir_dump`.
```console
> rustc +stage1 immut.rs -Zdump-mir=all
``` ```
rustc +stage1 immut.rs -Zdump-mir=all
``` [mir]: ./mir/index.md
we will see a newly generated directory in our current working directory called mir_dump, which will [dump-mir]: ./mir/passes.md
contain several files. If we look at file `rustc.main.-------.mir_map.0.mir`, we will find, among
After we run this command, we will see a newly generated directory in our
current working directory called `mir_dump`, which will contain several files.
If we look at file `rustc.main.-------.mir_map.0.mir`, we will find, among
other things, it also contains this line: other things, it also contains this line:
```rust,ignore ```rust,ignore
_4 = &_1; // bb0[6]: scope 1 at immut.rs:7:13: 7:36 _4 = &_1;
_3 = [closure@immut.rs:7:13: 7:36] { x: move _4 }; // bb0[7]: scope 1 at immut.rs:7:13: 7:36 _3 = [closure@immut.rs:7:13: 7:36] { x: move _4 };
``` ```
Here in first line `_4 = &_1;`, the mir_dump tells us that x was borrowed as an immutable reference.
This is what we would hope as our closure just reads x. Note that in the MIR examples in this chapter, `_1` is `x`.
Here in first line `_4 = &_1;`, the `mir_dump` tells us that `x` was borrowed
as an immutable reference. This is what we would hope as our closure just
reads `x`.
### Example 2 ### Example 2
Here is another example:
```rust ```rust
fn closure(mut f: impl FnMut()) { fn closure(mut f: impl FnMut()) {
f(); f();
@ -46,13 +77,16 @@ fn main() {
``` ```
```rust,ignore ```rust,ignore
_4 = &mut _1; // bb0[6]: scope 1 at mut.rs:7:13: 10:6 _4 = &mut _1;
_3 = [closure@mut.rs:7:13: 10:6] { x: move _4 }; // bb0[7]: scope 1 at mut.rs:7:13: 10:6 _3 = [closure@mut.rs:7:13: 10:6] { x: move _4 };
``` ```
This time along, in the line `_4 = &mut _1;`, we see that the borrow is changed to mutable borrow. This time along, in the line `_4 = &mut _1;`, we see that the borrow is changed to mutable borrow.
fair enough as the closure increments x by 10. Fair enough! The closure increments `x` by 10.
### Example 3 ### Example 3
One more example:
```rust ```rust
fn closure(f: impl FnOnce()) { fn closure(f: impl FnOnce()) {
f(); f();
@ -70,57 +104,66 @@ fn main() {
```rust,ignore ```rust,ignore
_6 = [closure@move.rs:7:13: 9:6] { x: move _1 }; // bb16[3]: scope 1 at move.rs:7:13: 9:6 _6 = [closure@move.rs:7:13: 9:6] { x: move _1 }; // bb16[3]: scope 1 at move.rs:7:13: 9:6
``` ```
Here, x is directly moved into the closure and the access to it will not be permitted after the Here, `x` is directly moved into the closure and the access to it will not be permitted after the
closure. closure.
## Inferences in the compiler
Now let's dive into rustc code and see how all these inferences are done by the compiler. Now let's dive into rustc code and see how all these inferences are done by the compiler.
Let's start with defining a term that we will be using quite a bit in the rest of the discussion - Let's start with defining a term that we will be using quite a bit in the rest of the discussion -
*upvar*. An **upvar** is a variable that is local to the function, where the closure is defined. So, *upvar*. An **upvar** is a variable that is local to the function where the closure is defined. So,
in the above examples, **x** will be an upvar to the closure. They are also sometimes referred to as in the above examples, **x** will be an upvar to the closure. They are also sometimes referred to as
the *free variables* meaning they are not bound to the context of the closure. the *free variables* meaning they are not bound to the context of the closure.
`src/librustc/ty/query/mod.rs` defines a query called *freevars* for this purpose. [`src/librustc/ty/query/mod.rs`][freevars] defines a query called *freevars* for this purpose.
So, we know that other than lazy invocation, one other thing that the distinguishes a closure from a [freevars]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/query/queries/struct.freevars.html
normal function is that it can use the upvars. Because, it borrows these upvars from its surrounding
context, therfore the compiler has to determine the upvar's borrow type. The compiler starts with Other than lazy invocation, one other thing that the distinguishes a closure from a
normal function is that it can use the upvars. It borrows these upvars from its surrounding
context; therfore the compiler has to determine the upvar's borrow type. The compiler starts with
assigning an immutable borrow type and lowers the restriction (that is, changes it from assigning an immutable borrow type and lowers the restriction (that is, changes it from
**immutable** to **mutable** to **move**) as needed, based on the usage. In the Example 1 above, the **immutable** to **mutable** to **move**) as needed, based on the usage. In the Example 1 above, the
closure only uses the variable for printing but does not modify it in any way and therefore, in the closure only uses the variable for printing but does not modify it in any way and therefore, in the
mir_dump, we find the borrow type for the upvar x to be immutable. In example 2, however the `mir_dump`, we find the borrow type for the upvar `x` to be immutable. In example 2, however, the
closure modifies x and increments it by some value. Because of this mutation, the compiler, which closure modifies `x` and increments it by some value. Because of this mutation, the compiler, which
started off assigning x as an immutable reference type, has to adjust it as mutable reference. started off assigning `x` as an immutable reference type, has to adjust it as a mutable reference.
Likewise in the third example, the closure drops the vector and therefore this requires the variable Likewise in the third example, the closure drops the vector and therefore this requires the variable
x to be moved into the closure. Depending on the borrow kind, the closure has to implement the `x` to be moved into the closure. Depending on the borrow kind, the closure has to implement the
appropriate trait. Fn trait for immutable borrow, FnMut for mutable borrow and FnOnce for move appropriate trait: `Fn` trait for immutable borrow, `FnMut` for mutable borrow,
semantics. and `FnOnce` for move semantics.
Most of the code related to the closure is in the src/librustc_typeck/check/upvar.rs file and the Most of the code related to the closure is in the
data structures are declared in the file src/librustc/ty/mod.rs. [`src/librustc_typeck/check/upvar.rs`][upvar] file and the data structures are
declared in the file [`src/librustc/ty/mod.rs`][ty].
[upvar]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_typeck/check/upvar/index.html
[ty]:https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/index.html
Before we go any further, let's discuss how we can examine the flow of coontrol through the rustc Before we go any further, let's discuss how we can examine the flow of coontrol through the rustc
codebase. For the closure part specifically, I would set the RUST_LOG as under and collect the codebase. For closures specifically, set the `RUST_LOG` env variable as below and collect the
output in a file output in a file:
``` ```console
RUST_LOG=rustc_typeck::check::upvar rustc +stage1 -Zdump-mir=all <.rs file to compile> 2> <file > RUST_LOG=rustc_typeck::check::upvar rustc +stage1 -Zdump-mir=all \
where the output will be dumped> <.rs file to compile> 2> <file where the output will be dumped>
``` ```
This uses the stage1 compiler. This uses the stage1 compiler and enables `debug!` logging for the
`rustc_typeck::check::upvar` module.
The other option is to step through the code using lldb or gdb. The other option is to step through the code using lldb or gdb.
``` 1. `rust-lldb build/x86_64-apple-darwin/stage1/bin/rustc test.rs`
1. rust-lldb build/x86_64-apple-darwin/stage1/bin/rustc test.rs 2. In lldb:
2. b upvar.rs:134 // Setting the breakpoint on a certain line in the upvar.rs file 1. `b upvar.rs:134` // Setting the breakpoint on a certain line in the upvar.rs file`
3. r // Run the program until it hits the breakpoint 2. `r` // Run the program until it hits the breakpoint
```
Let's start with the file: `upvar.rs`. This file has something called the euv::ExprUseVisitor which Let's start with [`upvar.rs`][upvar]. This file has something called
walks the source of the closure and it gets called back for each upvar that is borrowed, mutated or the [`euv::ExprUseVisitor`][euv] which walks the source of the closure and
moved. invokes a callbackfor each upvar that is borrowed, mutated, or moved.
[euv]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/expr_use_visitor/struct.ExprUseVisitor.html
```rust ```rust
fn main() { fn main() {
@ -132,28 +175,39 @@ fn main() {
} }
``` ```
In the above example, our visitor will be called twice, for the lines marked 1 and 2, once as a In the above example, our visitor will be called twice, for the lines marked 1 and 2, once for a
shared borrow and another one as a mutable borrow. It will also tell as what was borrowed. The shared borrow and another one for a mutable borrow. It will also tell us what was borrowed.
callbacks get invoked at the delegate. The delegate is of type `struct InferBorrowKind` which has a
few fields but the one we are interested in is the `adjust_upvar_captures` which is of type
`FxHashMap<UpvarId, UpvarCapture<'tcx>>` which tells us for each upvar, which mode of borrow did we
require. The modes of borrow can be ByValue (moved) or ByRef (borrowed) and for ByRef borrows, it
can be one among shared, shallow, unique or mut as defined in the `src/librustc/mir/mod.rs`
The method callbacks are the method implementations of the euv::Delegate trait for InferBorrowKind. The callbacks are defined by implementing the [`Delegate`][delegate] trait. The
**consume** callback is for *move* of a variable, **borrow** callback if there is a *borrow* of some [`InferBorrowKind`][ibk] type implements `Delegate` and keeps a map that
kind, shared or mutable and **mutate** when we see an *assignment* of something. We will see that records for each upvar which mode of borrow was required. The modes of borrow
all these callbacks have a common argument *cmt* which stands for category, Mutability and Type and can be `ByValue` (moved) or `ByRef` (borrowed). For `ByRef` borrows, it can be
is defined in *src/librustc/middle/mem_categorization.rs*. Borrowing from the code comments *cmt *is `shared`, `shallow`, `unique` or `mut` as defined in the
a complete categorization of a value indicating where it originated and how it is located, as well [`src/librustc/mir/mod.rs`][mir_mod].
as the mutability of the memory in which the value is stored.** Based on the callback (consume,
borrow etc.), we will call the relevant *adjust_upvar_borrow_kind_for_<something>* and pass the cmt
along. Once the borrow type is adjusted, we store it in the table, which basically says for this
closure, these set of borrows were made.
``` [mir_mod]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/mir/index.html
`Delegate` defines a few different methods (the different callbacks):
**consume**: for *move* of a variable, **borrow** for a *borrow* of some kind
(shared or mutable), and **mutate** when we see an *assignment* of something.
All of these callbacks have a common argument *cmt* which stands for Category,
Mutability and Type and is defined in
[`src/librustc/middle/mem_categorization.rs`][cmt]. Borrowing from the code
comments, "`cmt` is a complete categorization of a value indicating where it
originated and how it is located, as well as the mutability of the memory in
which the value is stored". Based on the callback (consume, borrow etc.), we
will call the relevant *adjust_upvar_borrow_kind_for_<something>* and pass the
`cmt` along. Once the borrow type is adjusted, we store it in the table, which
basically says what borrows were made for each closure.
```rust,ignore
self.tables self.tables
.borrow_mut() .borrow_mut()
.upvar_capture_map .upvar_capture_map
.extend(delegate.adjust_upvar_captures); .extend(delegate.adjust_upvar_captures);
``` ```
[delegate]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/expr_use_visitor/trait.Delegate.html
[ibk]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_typeck/check/upvar/struct.InferBorrowKind.html
[cmt]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/mem_categorization/index.html