diff --git a/src/closure.md b/src/closure.md index 3bf96598..a31893b0 100644 --- a/src/closure.md +++ b/src/closure.md @@ -1,8 +1,24 @@ # Closure Expansion in rustc -Let's start with a few examples +This section describes how rustc handles closures. Closures in Rust are +effectively "desugared" into structs that contain the values they use (or +references to the values they use) from their creator's stack frame. rustc has +the job of figuring out which values a closure uses and how, so it can decide +whether to capture a given variable by shared reference, mutable reference, or +by move. rustc also has to figure out which the closure traits ([`Fn`][fn], +[`FnMut`][fn_mut], or [`FnOnce`][fn_once]) a closure is capable of +implementing. + +[fn]: https://doc.rust-lang.org/std/ops/trait.Fn.html +[fn_mut]:https://doc.rust-lang.org/std/ops/trait.FnMut.html +[fn_once]: https://doc.rust-lang.org/std/ops/trait.FnOnce.html + +Let's start with a few examples: ### Example 1 + +To start, let's take a look at how the closure in the following example is desugared: + ```rust fn closure(f: impl Fn()) { f(); @@ -14,22 +30,37 @@ fn main() { println!("Value of x after return {}", x); } ``` -Let's say the above is the content of a file called immut.rs. If we compile immut.rs using the command + +Let's say the above is the content of a file called `immut.rs`. If we compile +`immut.rs` using the following command. The [`-Zdump-mir=all`][dump-mir] flag will cause +`rustc` to generate and dump the [MIR][mir] to a directory called `mir_dump`. +```console +> rustc +stage1 immut.rs -Zdump-mir=all ``` -rustc +stage1 immut.rs -Zdump-mir=all -``` -we will see a newly generated directory in our current working directory called mir_dump, which will -contain several files. If we look at file `rustc.main.-------.mir_map.0.mir`, we will find, among + +[mir]: ./mir/index.md +[dump-mir]: ./mir/passes.md + +After we run this command, we will see a newly generated directory in our +current working directory called `mir_dump`, which will contain several files. +If we look at file `rustc.main.-------.mir_map.0.mir`, we will find, among other things, it also contains this line: ```rust,ignore -_4 = &_1; // bb0[6]: scope 1 at immut.rs:7:13: 7:36 -_3 = [closure@immut.rs:7:13: 7:36] { x: move _4 }; // bb0[7]: scope 1 at immut.rs:7:13: 7:36 +_4 = &_1; +_3 = [closure@immut.rs:7:13: 7:36] { x: move _4 }; ``` -Here in first line `_4 = &_1;`, the mir_dump tells us that x was borrowed as an immutable reference. -This is what we would hope as our closure just reads x. + +Note that in the MIR examples in this chapter, `_1` is `x`. + +Here in first line `_4 = &_1;`, the `mir_dump` tells us that `x` was borrowed +as an immutable reference. This is what we would hope as our closure just +reads `x`. ### Example 2 + +Here is another example: + ```rust fn closure(mut f: impl FnMut()) { f(); @@ -46,13 +77,16 @@ fn main() { ``` ```rust,ignore -_4 = &mut _1; // bb0[6]: scope 1 at mut.rs:7:13: 10:6 -_3 = [closure@mut.rs:7:13: 10:6] { x: move _4 }; // bb0[7]: scope 1 at mut.rs:7:13: 10:6 +_4 = &mut _1; +_3 = [closure@mut.rs:7:13: 10:6] { x: move _4 }; ``` This time along, in the line `_4 = &mut _1;`, we see that the borrow is changed to mutable borrow. -fair enough as the closure increments x by 10. +Fair enough! The closure increments `x` by 10. ### Example 3 + +One more example: + ```rust fn closure(f: impl FnOnce()) { f(); @@ -70,57 +104,66 @@ fn main() { ```rust,ignore _6 = [closure@move.rs:7:13: 9:6] { x: move _1 }; // bb16[3]: scope 1 at move.rs:7:13: 9:6 ``` -Here, x is directly moved into the closure and the access to it will not be permitted after the +Here, `x` is directly moved into the closure and the access to it will not be permitted after the closure. +## Inferences in the compiler Now let's dive into rustc code and see how all these inferences are done by the compiler. Let's start with defining a term that we will be using quite a bit in the rest of the discussion - -*upvar*. An **upvar** is a variable that is local to the function, where the closure is defined. So, +*upvar*. An **upvar** is a variable that is local to the function where the closure is defined. So, in the above examples, **x** will be an upvar to the closure. They are also sometimes referred to as the *free variables* meaning they are not bound to the context of the closure. -`src/librustc/ty/query/mod.rs` defines a query called *freevars* for this purpose. +[`src/librustc/ty/query/mod.rs`][freevars] defines a query called *freevars* for this purpose. -So, we know that other than lazy invocation, one other thing that the distinguishes a closure from a -normal function is that it can use the upvars. Because, it borrows these upvars from its surrounding -context, therfore the compiler has to determine the upvar's borrow type. The compiler starts with +[freevars]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/query/queries/struct.freevars.html + +Other than lazy invocation, one other thing that the distinguishes a closure from a +normal function is that it can use the upvars. It borrows these upvars from its surrounding +context; therfore the compiler has to determine the upvar's borrow type. The compiler starts with assigning an immutable borrow type and lowers the restriction (that is, changes it from **immutable** to **mutable** to **move**) as needed, based on the usage. In the Example 1 above, the closure only uses the variable for printing but does not modify it in any way and therefore, in the -mir_dump, we find the borrow type for the upvar x to be immutable. In example 2, however the -closure modifies x and increments it by some value. Because of this mutation, the compiler, which -started off assigning x as an immutable reference type, has to adjust it as mutable reference. +`mir_dump`, we find the borrow type for the upvar `x` to be immutable. In example 2, however, the +closure modifies `x` and increments it by some value. Because of this mutation, the compiler, which +started off assigning `x` as an immutable reference type, has to adjust it as a mutable reference. Likewise in the third example, the closure drops the vector and therefore this requires the variable -x to be moved into the closure. Depending on the borrow kind, the closure has to implement the -appropriate trait. Fn trait for immutable borrow, FnMut for mutable borrow and FnOnce for move -semantics. +`x` to be moved into the closure. Depending on the borrow kind, the closure has to implement the +appropriate trait: `Fn` trait for immutable borrow, `FnMut` for mutable borrow, +and `FnOnce` for move semantics. -Most of the code related to the closure is in the src/librustc_typeck/check/upvar.rs file and the -data structures are declared in the file src/librustc/ty/mod.rs. +Most of the code related to the closure is in the +[`src/librustc_typeck/check/upvar.rs`][upvar] file and the data structures are +declared in the file [`src/librustc/ty/mod.rs`][ty]. + +[upvar]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_typeck/check/upvar/index.html +[ty]:https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/index.html Before we go any further, let's discuss how we can examine the flow of coontrol through the rustc -codebase. For the closure part specifically, I would set the RUST_LOG as under and collect the -output in a file +codebase. For closures specifically, set the `RUST_LOG` env variable as below and collect the +output in a file: -``` -RUST_LOG=rustc_typeck::check::upvar rustc +stage1 -Zdump-mir=all <.rs file to compile> 2> +```console +> RUST_LOG=rustc_typeck::check::upvar rustc +stage1 -Zdump-mir=all \ + <.rs file to compile> 2> ``` -This uses the stage1 compiler. +This uses the stage1 compiler and enables `debug!` logging for the +`rustc_typeck::check::upvar` module. The other option is to step through the code using lldb or gdb. -``` -1. rust-lldb build/x86_64-apple-darwin/stage1/bin/rustc test.rs -2. b upvar.rs:134 // Setting the breakpoint on a certain line in the upvar.rs file -3. r // Run the program until it hits the breakpoint -``` +1. `rust-lldb build/x86_64-apple-darwin/stage1/bin/rustc test.rs` +2. In lldb: + 1. `b upvar.rs:134` // Setting the breakpoint on a certain line in the upvar.rs file` + 2. `r` // Run the program until it hits the breakpoint -Let's start with the file: `upvar.rs`. This file has something called the euv::ExprUseVisitor which -walks the source of the closure and it gets called back for each upvar that is borrowed, mutated or -moved. +Let's start with [`upvar.rs`][upvar]. This file has something called +the [`euv::ExprUseVisitor`][euv] which walks the source of the closure and +invokes a callbackfor each upvar that is borrowed, mutated, or moved. + +[euv]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/expr_use_visitor/struct.ExprUseVisitor.html ```rust fn main() { @@ -132,28 +175,39 @@ fn main() { } ``` -In the above example, our visitor will be called twice, for the lines marked 1 and 2, once as a -shared borrow and another one as a mutable borrow. It will also tell as what was borrowed. The -callbacks get invoked at the delegate. The delegate is of type `struct InferBorrowKind` which has a -few fields but the one we are interested in is the `adjust_upvar_captures` which is of type -`FxHashMap>` which tells us for each upvar, which mode of borrow did we -require. The modes of borrow can be ByValue (moved) or ByRef (borrowed) and for ByRef borrows, it -can be one among shared, shallow, unique or mut as defined in the `src/librustc/mir/mod.rs` +In the above example, our visitor will be called twice, for the lines marked 1 and 2, once for a +shared borrow and another one for a mutable borrow. It will also tell us what was borrowed. -The method callbacks are the method implementations of the euv::Delegate trait for InferBorrowKind. -**consume** callback is for *move* of a variable, **borrow** callback if there is a *borrow* of some -kind, shared or mutable and **mutate** when we see an *assignment* of something. We will see that -all these callbacks have a common argument *cmt* which stands for category, Mutability and Type and -is defined in *src/librustc/middle/mem_categorization.rs*. Borrowing from the code comments *cmt *is -a complete categorization of a value indicating where it originated and how it is located, as well -as the mutability of the memory in which the value is stored.** Based on the callback (consume, -borrow etc.), we will call the relevant *adjust_upvar_borrow_kind_for_* and pass the cmt -along. Once the borrow type is adjusted, we store it in the table, which basically says for this -closure, these set of borrows were made. +The callbacks are defined by implementing the [`Delegate`][delegate] trait. The +[`InferBorrowKind`][ibk] type implements `Delegate` and keeps a map that +records for each upvar which mode of borrow was required. The modes of borrow +can be `ByValue` (moved) or `ByRef` (borrowed). For `ByRef` borrows, it can be +`shared`, `shallow`, `unique` or `mut` as defined in the +[`src/librustc/mir/mod.rs`][mir_mod]. -``` +[mir_mod]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/mir/index.html + +`Delegate` defines a few different methods (the different callbacks): +**consume**: for *move* of a variable, **borrow** for a *borrow* of some kind +(shared or mutable), and **mutate** when we see an *assignment* of something. + +All of these callbacks have a common argument *cmt* which stands for Category, +Mutability and Type and is defined in +[`src/librustc/middle/mem_categorization.rs`][cmt]. Borrowing from the code +comments, "`cmt` is a complete categorization of a value indicating where it +originated and how it is located, as well as the mutability of the memory in +which the value is stored". Based on the callback (consume, borrow etc.), we +will call the relevant *adjust_upvar_borrow_kind_for_* and pass the +`cmt` along. Once the borrow type is adjusted, we store it in the table, which +basically says what borrows were made for each closure. + +```rust,ignore self.tables .borrow_mut() .upvar_capture_map .extend(delegate.adjust_upvar_captures); ``` + +[delegate]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/expr_use_visitor/trait.Delegate.html +[ibk]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_typeck/check/upvar/struct.InferBorrowKind.html +[cmt]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/middle/mem_categorization/index.html