mirror
/
rustc-dev-guide
mirror of https://github.com/rust-lang/rustc-dev-guide.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

move the last few things from the forge

This commit is contained in:
Mark Mansi 2019-12-06 11:06:38 -06:00 committed by Who? Me?!
parent 7c56708aab
commit cd85fbd68b
7 changed files with 314183 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
src/SUMMARY.md
View File

@ -17,6 +17,7 @@
- [Adding new tests](./tests/adding.md)
- [Using `compiletest` + commands to control test execution](./compiletest.md)
- [Walkthrough: a typical contribution](./walkthrough.md)
- [Bug Fix Procedure](./bug-fix-procedure.md)
- [Implementing new features](./implementing_new_features.md)
- [Stability attributes](./stability.md)
- [Stabilizing Features](./stabilization_guide.md)
@ -27,6 +28,7 @@
- [crates.io Dependencies](./crates-io.md)
- [Emitting Errors and other Diagnostics](diagnostics.md)
- [`LintStore`](./diagnostics/lintstore.md)
- [Diagnostic Codes](./diagnostics/diagnostic-codes.md)
- [ICE-breaker teams](ice-breaker/about.md)
- [LLVM ICE-breakers](ice-breaker/llvm.md)
- [Part 2: How rustc works](./part-2-intro.md)
@ -38,6 +40,7 @@
- [Incremental compilation](./queries/incremental-compilation.md)
- [Incremental compilation In Detail](./queries/incremental-compilation-in-detail.md)
- [Debugging and Testing](./incrcomp-debugging.md)
- [Profiling Queries](./queries/profiling.md)
- [Salsa](./salsa.md)
- [Lexing and Parsing](./the-parser.md)
- [`#[test]` Implementation](./test-implementation.md)

330
src/bug-fix-procedure.md Normal file
View File

@ -0,0 +1,330 @@
# Rustc Bug Fix Procedure
This page defines the best practices procedure for making bug fixes or soundness
corrections in the compiler that can cause existing code to stop compiling. This
text is based on
[RFC 1589](https://github.com/rust-lang/rfcs/blob/master/text/1589-rustc-bug-fix-procedure.md).
# Motivation
[motivation]: #motivation
From time to time, we encounter the need to make a bug fix, soundness
correction, or other change in the compiler which will cause existing code to
stop compiling. When this happens, it is important that we handle the change in
a way that gives users of Rust a smooth transition. What we want to avoid is
that existing programs suddenly stop compiling with opaque error messages: we
would prefer to have a gradual period of warnings, with clear guidance as to
what the problem is, how to fix it, and why the change was made. This RFC
describes the procedure that we have been developing for handling breaking
changes that aims to achieve that kind of smooth transition.
One of the key points of this policy is that (a) warnings should be issued
initially rather than hard errors if at all possible and (b) every change that
causes existing code to stop compiling will have an associated tracking issue.
This issue provides a point to collect feedback on the results of that change.
Sometimes changes have unexpectedly large consequences or there may be a way to
avoid the change that was not considered. In those cases, we may decide to
change course and roll back the change, or find another solution (if warnings
are being used, this is particularly easy to do).
### What qualifies as a bug fix?
Note that this RFC does not try to define when a breaking change is permitted.
That is already covered under [RFC 1122][]. This document assumes that the
change being made is in accordance with those policies. Here is a summary of the
conditions from RFC 1122:
- **Soundness changes:** Fixes to holes uncovered in the type system.
- **Compiler bugs:** Places where the compiler is not implementing the specified
semantics found in an RFC or lang-team decision.
- **Underspecified language semantics:** Clarifications to grey areas where the
compiler behaves inconsistently and no formal behavior had been previously
decided.
Please see [the RFC][rfc 1122] for full details!
# Detailed design
[design]: #detailed-design
The procedure for making a breaking change is as follows (each of these steps is
described in more detail below):
0. Do a **crater run** to assess the impact of the change.
1. Make a **special tracking issue** dedicated to the change.
1. Do not report an error right away. Instead, **issue forwards-compatibility
lint warnings**.
- Sometimes this is not straightforward. See the text below for suggestions
on different techniques we have employed in the past.
- For cases where warnings are infeasible:
- Report errors, but make every effort to give a targeted error message
that directs users to the tracking issue
- Submit PRs to all known affected crates that fix the issue
- or, at minimum, alert the owners of those crates to the problem and
direct them to the tracking issue
1. Once the change has been in the wild for at least one cycle, we can
**stabilize the change**, converting those warnings into errors.
Finally, for changes to libsyntax that will affect plugins, the general policy
is to batch these changes. That is discussed below in more detail.
### Tracking issue
Every breaking change should be accompanied by a **dedicated tracking issue**
for that change. The main text of this issue should describe the change being
made, with a focus on what users must do to fix their code. The issue should be
approachable and practical; it may make sense to direct users to an RFC or some
other issue for the full details. The issue also serves as a place where users
can comment with questions or other concerns.
A template for these breaking-change tracking issues can be found below. An
example of how such an issue should look can be [found
here][breaking-change-issue].
The issue should be tagged with (at least) `B-unstable` and `T-compiler`.
### Tracking issue template
This is a template to use for tracking issues:
```
This is the **summary issue** for the `YOUR_LINT_NAME_HERE`
future-compatibility warning and other related errors. The goal of
this page is describe why this change was made and how you can fix
code that is affected by it. It also provides a place to ask questions
or register a complaint if you feel the change should not be made. For
more information on the policy around future-compatibility warnings,
see our [breaking change policy guidelines][guidelines].
[guidelines]: LINK_TO_THIS_RFC
#### What is the warning for?
*Describe the conditions that trigger the warning and how they can be
fixed. Also explain why the change was made.**
#### When will this warning become a hard error?
At the beginning of each 6-week release cycle, the Rust compiler team
will review the set of outstanding future compatibility warnings and
nominate some of them for **Final Comment Period**. Toward the end of
the cycle, we will review any comments and make a final determination
whether to convert the warning into a hard error or remove it
entirely.
```
### Issuing future compatibility warnings
The best way to handle a breaking change is to begin by issuing
future-compatibility warnings. These are a special category of lint warning.
Adding a new future-compatibility warning can be done as follows.
```rust
// 1. Define the lint in `src/librustc/lint/builtin.rs`:
declare_lint! {
pub YOUR_ERROR_HERE,
Warn,
"illegal use of foo bar baz"
}
// 2. Add to the list of HardwiredLints in the same file:
impl LintPass for HardwiredLints {
fn get_lints(&self) -> LintArray {
lint_array!(
..,
YOUR_ERROR_HERE
)
}
}
// 3. Register the lint in `src/librustc_lint/lib.rs`:
store.register_future_incompatible(sess, vec![
...,
FutureIncompatibleInfo {
id: LintId::of(YOUR_ERROR_HERE),
reference: "issue #1234", // your tracking issue here!
},
]);
// 4. Report the lint:
tcx.lint_node(
lint::builtin::YOUR_ERROR_HERE,
path_id,
binding.span,
format!("some helper message here"));
```
#### Helpful techniques
It can often be challenging to filter out new warnings from older, pre-existing
errors. One technique that has been used in the past is to run the older code
unchanged and collect the errors it would have reported. You can then issue
warnings for any errors you would give which do not appear in that original set.
Another option is to abort compilation after the original code completes if
errors are reported: then you know that your new code will only execute when
there were no errors before.
#### Crater and crates.io
We should always do a crater run to assess impact. It is polite and considerate
to at least notify the authors of affected crates the breaking change. If we can
submit PRs to fix the problem, so much the better.
#### Is it ever acceptable to go directly to issuing errors?
Changes that are believed to have negligible impact can go directly to issuing
an error. One rule of thumb would be to check against `crates.io`: if fewer than
10 **total** affected projects are found (**not** root errors), we can move
straight to an error. In such cases, we should still make the "breaking change"
page as before, and we should ensure that the error directs users to this page.
In other words, everything should be the same except that users are getting an
error, and not a warning. Moreover, we should submit PRs to the affected
projects (ideally before the PR implementing the change lands in rustc).
If the impact is not believed to be negligible (e.g., more than 10 crates are
affected), then warnings are required (unless the compiler team agrees to grant
a special exemption in some particular case). If implementing warnings is not
feasible, then we should make an aggressive strategy of migrating crates before
we land the change so as to lower the number of affected crates. Here are some
techniques for approaching this scenario:
1. Issue warnings for subparts of the problem, and reserve the new errors for
the smallest set of cases you can.
2. Try to give a very precise error message that suggests how to fix the problem
and directs users to the tracking issue.
3. It may also make sense to layer the fix:
- First, add warnings where possible and let those land before proceeding to
issue errors.
- Work with authors of affected crates to ensure that corrected versions are
available _before_ the fix lands, so that downstream users can use them.
### Stabilization
After a change is made, we will **stabilize** the change using the same process
that we use for unstable features:
- After a new release is made, we will go through the outstanding tracking
issues corresponding to breaking changes and nominate some of them for **final
comment period** (FCP).
- The FCP for such issues lasts for one cycle. In the final week or two of the
cycle, we will review comments and make a final determination:
- Convert to error: the change should be made into a hard error.
- Revert: we should remove the warning and continue to allow the older code to
compile.
- Defer: can't decide yet, wait longer, or try other strategies.
Ideally, breaking changes should have landed on the **stable branch** of the
compiler before they are finalized.
<a name="guide">
### Removing a lint
Once we have decided to make a "future warning" into a hard error, we need a PR
that removes the custom lint. As an example, here are the steps required to
remove the `overlapping_inherent_impls` compatibility lint. First, convert the
name of the lint to uppercase (`OVERLAPPING_INHERENT_IMPLS`) ripgrep through the
source for that string. We will basically by converting each place where this
lint name is mentioned (in the compiler, we use the upper-case name, and a macro
automatically generates the lower-case string; so searching for
`overlapping_inherent_impls` would not find much).
#### Remove the lint.
The first reference you will likely find is the lint definition [in
`librustc/lint/builtin.rs` that resembles this][defsource]:
[defsource]: https://github.com/rust-lang/rust/blob/085d71c3efe453863739c1fb68fd9bd1beff214f/src/librustc/lint/builtin.rs#L171-L175
```rust
declare_lint! {
pub OVERLAPPING_INHERENT_IMPLS,
Deny, // this may also say Warning
"two overlapping inherent impls define an item with the same name were erroneously allowed"
}
```
This `declare_lint!` macro creates the relevant data structures. Remove it. You
will also find that there is a mention of `OVERLAPPING_INHERENT_IMPLS` later in
the file as [part of a `lint_array!`][lintarraysource]; remove it too,
[lintarraysource]: https://github.com/rust-lang/rust/blob/085d71c3efe453863739c1fb68fd9bd1beff214f/src/librustc/lint/builtin.rs#L252-L290
Next, you see see [a reference to `OVERLAPPING_INHERENT_IMPLS` in
`librustc_lint/lib.rs`][futuresource]. This defining the lint as a "future
compatibility lint":
```rust
FutureIncompatibleInfo {
id: LintId::of(OVERLAPPING_INHERENT_IMPLS),
reference: "issue #36889 <https://github.com/rust-lang/rust/issues/36889>",
},
```
Remove this too.
#### Add the lint to the list of removed lists.
In `src/librustc_lint/lib.rs` there is a list of "renamed and removed lints".
You can add this lint to the list:
```rust
store.register_removed("overlapping_inherent_impls", "converted into hard error, see #36889");
```
where `#36889` is the tracking issue for your lint.
#### Update the places that issue the lint
Finally, the last class of references you will see are the places that actually
**trigger** the lint itself (i.e., what causes the warnings to appear). These
you do not want to delete. Instead, you want to convert them into errors. In
this case, the [`add_lint` call][addlintsource] looks like this:
```rust
self.tcx.sess.add_lint(lint::builtin::OVERLAPPING_INHERENT_IMPLS,
node_id,
self.tcx.span_of_impl(item1).unwrap(),
msg);
```
We want to convert this into an error. In some cases, there may be an existing
error for this scenario. In others, we will need to allocate a fresh diagnostic
code.
[Instructions for allocating a fresh diagnostic code can be found here.](rustc-diagnostic-code.html)
You may want to mention in the extended description that the compiler behavior
changed on this point, and include a reference to the tracking issue for the
change.
Let's say that we've adopted `E0592` as our code. Then we can change the
`add_lint()` call above to something like:
```rust
struct_span_err!(self.tcx.sess, self.tcx.span_of_impl(item1).unwrap(), msg)
.emit();
```
#### Update tests
Finally, run the test suite. These should be some tests that used to reference
the `overlapping_inherent_impls` lint, those will need to be updated. In
general, if the test used to have `#[deny(overlapping_inherent_impls)]`, that
can just be removed.
```
./x.py test
```
#### All done!
Open a PR. =)
[addlintsource]: https://github.com/rust-lang/rust/blob/085d71c3efe453863739c1fb68fd9bd1beff214f/src/librustc_typeck/coherence/inherent.rs#L300-L303
[futuresource]: https://github.com/rust-lang/rust/blob/085d71c3efe453863739c1fb68fd9bd1beff214f/src/librustc_lint/lib.rs#L202-L205
<!-- -Links--------------------------------------------------------------------- -->
[rfc 1122]: https://github.com/rust-lang/rfcs/blob/master/text/1122-language-semver.md
[breaking-change-issue]: https://gist.github.com/nikomatsakis/631ec8b4af9a18b5d062d9d9b7d3d967

75
src/diagnostics/diagnostic-codes.md Normal file
View File

@ -0,0 +1,75 @@
# Diagnostic Codes
We generally try assign each error message a unique code like `E0123`. These
codes are defined in the compiler in the `diagnostics.rs` files found in each
crate, which basically consist of macros. The codes come in two varieties: those
that have an extended write-up, and those that do not. Whenever possible, if you
are making a new code, you should write an extended write-up.
### Allocating a fresh code
If you want to create a new error, you first need to find the next available
code. This is a bit tricky since the codes are defined in various crates. To do
it, run this obscure command:
```
./x.py test --stage 0 src/tools/tidy
```
This will invoke the tidy script, which generally checks that your code obeys
our coding conventions. One of those jobs is to check that diagnostic codes are
indeed unique. Once it is finished with that, tidy will print out the lowest
unused code:
```
...
tidy check (x86_64-apple-darwin)
* 470 error codes
* highest error code: E0591
...
```
Here we see the highest error code in use is `E0591`, so we _probably_ want
`E0592`. To be sure, run `rg E0592` and check, you should see no references.
Next, open `src/{crate}/diagnostics.rs` within the crate where you wish to issue
the error (e.g., `src/librustc_typeck/diagnostics.rs`). Ideally, you will add
the code (in its proper numerical order) into the `register_long_diagnostics!`
macro, sort of like this:
```rust
register_long_diagnostics! {
...
E0592: r##"
Your extended error text goes here!
"##,
}
```
But you can also add it without an extended description:
```rust
register_diagnostics! {
...
E0592, // put a description here
}
```
To actually issue the error, you can use the `struct_span_err!` macro:
```rust
struct_span_err!(self.tcx.sess, // some path to the session here
span, // whatever span in the source you want
E0592, // your new error code
&format!("text of the error"))
.emit() // actually issue the error
```
If you want to add notes or other snippets, you can invoke methods before you
call `.emit()`:
```rust
struct_span_err!(...)
.span_label(another_span, "something to label in the source")
.span_note(another_span, "some separate note, probably avoid these")
.emit_()
```

104
src/queries/example-0.counts.txt Normal file
View File

@ -0,0 +1,104 @@
translation,1,0.891
symbol_name,2658,0.733
def_symbol_name,2556,0.268
item_attrs,5566,0.162
type_of,6922,0.117
generics_of,8020,0.084
serialize dep graph,1,0.079
relevant_trait_impls_for,50,0.063
def_span,24875,0.061
expansion,1,0.059
const checking,1,0.055
adt_def,1141,0.048
trait_impls_of,32,0.045
is_copy_raw,47,0.045
is_foreign_item,2638,0.042
fn_sig,2172,0.033
adt_dtorck_constraint,2,0.023
impl_trait_ref,2434,0.023
typeck_tables_of,29,0.022
item-bodies checking,1,0.017
typeck_item_bodies,1,0.017
is_default_impl,2320,0.017
borrow checking,1,0.014
borrowck,4,0.014
mir_validated,4,0.013
adt_destructor,10,0.012
layout_raw,258,0.010
load_dep_graph,1,0.007
item-types checking,1,0.005
mir_const,2,0.005
name resolution,1,0.004
is_object_safe,35,0.003
is_sized_raw,89,0.003
parsing,1,0.003
is_freeze_raw,11,0.001
privacy checking,1,0.001
privacy_access_levels,5,0.001
resolving dependency formats,1,0.001
adt_sized_constraint,9,0.001
wf checking,1,0.001
liveness checking,1,0.001
compute_incremental_hashes_map,1,0.001
match checking,1,0.001
type collecting,1,0.001
param_env,31,0.000
effect checking,1,0.000
trait_def,140,0.000
lowering ast -> hir,1,0.000
predicates_of,70,0.000
extern_crate,319,0.000
lifetime resolution,1,0.000
is_const_fn,6,0.000
intrinsic checking,1,0.000
translation item collection,1,0.000
impl_polarity,15,0.000
creating allocators,1,0.000
language item collection,1,0.000
crate injection,1,0.000
early lint checks,1,0.000
indexing hir,1,0.000
maybe creating a macro crate,1,0.000
coherence checking,1,0.000
optimized_mir,6,0.000
is_panic_runtime,33,0.000
associated_item_def_ids,7,0.000
needs_drop_raw,10,0.000
lint checking,1,0.000
complete gated feature checking,1,0.000
stability index,1,0.000
region_maps,11,0.000
super_predicates_of,8,0.000
coherent_trait,2,0.000
AST validation,1,0.000
loop checking,1,0.000
static item recursion checking,1,0.000
variances_of,11,0.000
associated_item,5,0.000
plugin loading,1,0.000
looking for plugin registrar,1,0.000
stability checking,1,0.000
describe_def,15,0.000
variance testing,1,0.000
codegen unit partitioning,1,0.000
looking for entry point,1,0.000
checking for inline asm in case the target doesn't support it,1,0.000
inherent_impls,1,0.000
crate_inherent_impls,1,0.000
trait_of_item,7,0.000
crate_inherent_impls_overlap_check,1,0.000
attribute checking,1,0.000
internalize symbols,1,0.000
impl wf inference,1,0.000
death checking,1,0.000
reachability checking,1,0.000
reachable_set,1,0.000
is_exported_symbol,3,0.000
is_mir_available,2,0.000
unused lib feature checking,1,0.000
maybe building test harness,1,0.000
recursion limit,1,0.000
write allocator module,1,0.000
assert dep graph,1,0.000
plugin registration,1,0.000
write metadata,1,0.000

313330
src/queries/example-0.html Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
src/queries/example-0.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 104 KiB

341
src/queries/profiling.md Normal file
View File

@ -0,0 +1,341 @@
# Profiling Queries
In an effort to support _incremental compilation_, the latest design of the Rust
compiler consists of a _query-based_ model.
The details of this model are (currently) outside the scope of this document,
however, we explain [some background of this model](#background), in an effort
to explain how we profile its performance. We intend this profiling effort to
address [issue 42678](https://github.com/rust-lang/rust/issues/42678).
## Quick Start
### 0. Enable debug assertions
```
./configure --enable-debug-assertions
```
### 1. Compile `rustc`
Compile the compiler, up to at least stage 1:
```
python x.py --stage 1
```
### 2. Run `rustc`, with flags
Run the compiler on a source file, supplying two additional debugging flags with
`-Z`:
```
rustc -Z profile-queries -Z incremental=cache foo.rs
```
Regarding the two additional parameters:
- `-Z profile-queries` tells the compiler to run a separate thread that profiles
the queries made by the main compiler thread(s).
- `-Z incremental=cache` tells the compiler to "cache" various files that
describe the compilation dependencies, in the subdirectory `cache`.
This command will generate the following files:
- `profile_queries.html` consists of an HTML-based representation of the
[trace of queries](#trace-of-queries).
- `profile_queries.counts.txt` consists of a histogram, where each histogram
"bucket" is a query provider.
### 3. Run `rustc`, with `-Z time-passes`:
- This additional flag will add all timed passes to the output files mentioned
above, in step 2. As described below, these passes appear visually distinct
from the queries in the HTML output (they currently appear as green boxes, via
CSS).
### 4. Inspect the output
- 4(a). Open the HTML file (`profile_queries.html`) with a browser. See
[this section](#interpret-the-html-output) for an explanation of this file.
- 4(b). Open the data file (`profile_queries.counts.txt`) with a text editor, or
spreadsheet. See [this section](#interpret-the-data-output) for an explanation
of this file.
## Interpret the HTML Output
### Example 0
The following image gives some example output, from tracing the queries of
`hello_world.rs` (a single `main` function, that prints `"hello world"` via the
macro `println!`). This image only shows a short prefix of the total output; the
_actual_ output is much longer.
[![Example HTML output](./example-0.png)][profile-example-html]
[View full HTML output][profile-example-html]. Note; it could take up
to a second to properly render depending on your browser.
[profile-example-html]: https://github.com/rust-lang/rustc-guide/tree/master/src/queries/example-0.html
### Example 0 explanation
The trace of the queries has a formal structure; see
[Trace of Queries](#trace-of-queries) for details.
We style this formal structure as follows:
- **Timed passes:** Green boxes, when present (via `-Z time-passes`), represent
_timed passes_ in the compiler. In future versions, these passes may be
replaced by queries, explained below.
- **Labels:** Some green and red boxes are labeled with text. Where they are
present, the labels give the following information:
- The [query's _provider_](#queries), sans its _key_ and its _result_, which
are often too long to include in these labels.
- The _duration_ of the provider, as a fraction of the total time (for the
entire trace). This fraction includes the query's entire extent (that is,
the sum total of all of its sub-queries).
- **Query hits:** Blue dots represent query hits. They consist of leaves in the
trace's tree. (CSS class: `hit`).
- **Query misses:** Red boxes represent query misses. They consist of internal
nodes in the trace's tree. (CSS class: `miss`).
- **Nesting structure:** Many red boxes contain _nested boxes and dots_. This
nesting structure reflects that some providers _depend on_ results from other
providers, which consist of their nested children.
- Some red boxes are _labeled_ with text, and have highlighted borders (light
red, and bolded). (See [heuristics](#heuristics) for details).
## Heuristics
Heuristics-based CSS Classes:
- `important` -- Trace nodes are `important` if they have an extent of 6 (or
more), _or_ they have a duration fraction of one percent (or more). These
numbers are simple heuristics (currently hard-coded, but easy to modify).
Important nodes are styled with textual labels, and highlighted borders (light
red, and bolded).
- `frac-50`, `-40`, ... -- Trace nodes whose total duration (self and children)
take a large fraction of the total duration, at or above 50%, 40%, and so on.
We style nodes these with larger font and padding.
## Interpret the Data Output
The file `profile_queries.counts.txt` contains a table of information about the
queries, organized around their providers.
For each provider (or timed pass, when `-Z time-passes` is present), we produce:
- A total **count** --- the total number of times this provider was queried
- A total **duration** --- the total number of seconds spent running this
provider, _including_ all providers it may depend on. To get a sense of this
dependency structure, and inspect a more fine-grained view of these durations,
see [this section](#interpret-the-html-output).
These rows are **sorted by total duration**, in descending order.
### Counts: Example 0
The following example `profile_queries.counts.txt` file results from running on
a hello world program (a single main function that uses `println` to print
`"hellow world").
As explained above, the columns consist of `provider/pass`, `count`, `duration`:
```
translation,1,0.891
symbol_name,2658,0.733
def_symbol_name,2556,0.268
item_attrs,5566,0.162
type_of,6922,0.117
generics_of,8020,0.084
serialize dep graph,1,0.079
relevant_trait_impls_for,50,0.063
def_span,24875,0.061
expansion,1,0.059
const checking,1,0.055
adt_def,1141,0.048
trait_impls_of,32,0.045
is_copy_raw,47,0.045
is_foreign_item,2638,0.042
fn_sig,2172,0.033
adt_dtorck_constraint,2,0.023
impl_trait_ref,2434,0.023
typeck_tables_of,29,0.022
item-bodies checking,1,0.017
typeck_item_bodies,1,0.017
is_default_impl,2320,0.017
borrow checking,1,0.014
borrowck,4,0.014
mir_validated,4,0.013
adt_destructor,10,0.012
layout_raw,258,0.010
load_dep_graph,1,0.007
item-types checking,1,0.005
mir_const,2,0.005
name resolution,1,0.004
is_object_safe,35,0.003
is_sized_raw,89,0.003
parsing,1,0.003
is_freeze_raw,11,0.001
privacy checking,1,0.001
privacy_access_levels,5,0.001
resolving dependency formats,1,0.001
adt_sized_constraint,9,0.001
wf checking,1,0.001
liveness checking,1,0.001
compute_incremental_hashes_map,1,0.001
match checking,1,0.001
type collecting,1,0.001
param_env,31,0.000
effect checking,1,0.000
trait_def,140,0.000
lowering ast -> hir,1,0.000
predicates_of,70,0.000
extern_crate,319,0.000
lifetime resolution,1,0.000
is_const_fn,6,0.000
intrinsic checking,1,0.000
translation item collection,1,0.000
impl_polarity,15,0.000
creating allocators,1,0.000
language item collection,1,0.000
crate injection,1,0.000
early lint checks,1,0.000
indexing hir,1,0.000
maybe creating a macro crate,1,0.000
coherence checking,1,0.000
optimized_mir,6,0.000
is_panic_runtime,33,0.000
associated_item_def_ids,7,0.000
needs_drop_raw,10,0.000
lint checking,1,0.000
complete gated feature checking,1,0.000
stability index,1,0.000
region_maps,11,0.000
super_predicates_of,8,0.000
coherent_trait,2,0.000
AST validation,1,0.000
loop checking,1,0.000
static item recursion checking,1,0.000
variances_of,11,0.000
associated_item,5,0.000
plugin loading,1,0.000
looking for plugin registrar,1,0.000
stability checking,1,0.000
describe_def,15,0.000
variance testing,1,0.000
codegen unit partitioning,1,0.000
looking for entry point,1,0.000
checking for inline asm in case the target doesn't support it,1,0.000
inherent_impls,1,0.000
crate_inherent_impls,1,0.000
trait_of_item,7,0.000
crate_inherent_impls_overlap_check,1,0.000
attribute checking,1,0.000
internalize symbols,1,0.000
impl wf inference,1,0.000
death checking,1,0.000
reachability checking,1,0.000
reachable_set,1,0.000
is_exported_symbol,3,0.000
is_mir_available,2,0.000
unused lib feature checking,1,0.000
maybe building test harness,1,0.000
recursion limit,1,0.000
write allocator module,1,0.000
assert dep graph,1,0.000
plugin registration,1,0.000
write metadata,1,0.000
```
# Background
We give some background about the query model of the Rust compiler.
## Def IDs
In the query model, many queries have a key that consists of a Def ID. The Rust
compiler uses Def IDs to distinguish definitions in the input Rust program.
From the compiler source code (`src/librustc/hir/def_id.rs`):
```
/// A DefId identifies a particular *definition*, by combining a crate
/// index and a def index.
#[derive(Clone, Eq, Ord, PartialOrd, PartialEq, RustcEncodable, RustcDecodable, Hash, Copy)]
pub struct DefId {
pub krate: CrateNum,
pub index: DefIndex,
}
```
## Queries
A query relates a _key_ to a _result_, either by invoking a _provider_ that
computes this result, or by reusing a cached result that was provided earlier.
We explain each term in more detail:
- Query **Provider**: Each kind of query has a pre-defined _provider_, which
refers to the compiler behavior that provides an answer to the query. These
providers may nest; see [trace of queries](#trace-of-queries) for more
information about this nesting structure.
_Example providers:_
- `typeck_tables_of` -- Typecheck a Def ID; produce "tables" of type
information.
- `borrowck` -- Borrow-check a Def ID.
- `optimized_mir` -- Generate an optimized MIR for a Def ID; produce MIR.
- For more examples, see [Example 0](#counts-example-0).
- Query **Key**: The input/arguments to the provider. Often, this consists of a
particular [Def ID](#def-ids).
- Query **Result**: The output of the provider.
## Trace of Queries
Formally, a _trace_ of the queries consists of a _tree_, where sub-trees
represent sub-traces. In particular, the nesting structure of the trace of
queries describes how the queries depend on one another.
Even more precisely, this tree represents a directed acyclic graph (DAG), where
shared sub-graphs consist of tree nodes that occur multiple times in the tree,
first as "cache misses" and later as "cache hits".
**Cache hits and misses.** The trace is a tree with the following possible tree
nodes:
- Query, with cache **miss**: The query's result is **unknown**, and its
provider runs to compute it. In this case, the dynamic extent of the query's
trace consists of the traced behavior of its provider.
- Query, with cache **hit**: The query's result is **known**, and is reused; its
provider does not rerun. These nodes are leaves in the trace, since they have
no dynamic extent. These leaves also represent where the tree, represented as
a DAG, would _share_ a sub-graph (namely, the sub-graph of the query that was
reused from the cache).
**Tree node metrics.** To help determine how to style this tree, we define the
following tree node metrics:
- **Depth**: The number of **ancestors** of the node in its path from the tree
root.
- **Extent**: The number of **immediate children** of the node.
Intuitively, a dependency tree is "good" for incremental caching when the depth
and extent of each node is relatively small. It is pathological when either of
these metrics grows too large. For instance, a tree node whose extent consists
of 1M immediate children means that if and when this node is re-computed, all 1M
children must be re-queried, at the very least (some may also require
recomputation, too).
## External Links
Related design ideas, and tracking issues:
- Design document:
[On-demand Rustc incremental design doc](https://github.com/nikomatsakis/rustc-on-demand-incremental-design-doc/blob/master/0000-rustc-on-demand-and-incremental.md)
- Tracking Issue:
["Red/Green" dependency tracking in compiler](https://github.com/rust-lang/rust/issues/42293)
More discussion and issues:
- [GitHub issue #42633](https://github.com/rust-lang/rust/issues/42633)
- [Incremental Compilation Beta](https://internals.rust-lang.org/t/incremental-compilation-beta/4721)
- [Incremental Compilation Announcement](https://blog.rust-lang.org/2016/09/08/incremental.html)