Reorganize the guide (#651)
* start on guide reorg * backend intro * part 2, 3, 4 intros * some missing files * Fix typos Co-Authored-By: Yuki Okushi <huyuumi.dev@gmail.com> Co-Authored-By: Chris Simpkins <git.simpkins@gmail.com> * add links from intro * rename compiler source chapter and add placeholder for overview * rename parts, mv rustc_driver to part 3, make syntax chapter * update part intros * add syntax ch intro * address review comments * Better overview chapter placeholder Co-Authored-By: Chris Simpkins <git.simpkins@gmail.com> * fix link Co-authored-by: Yuki Okushi <huyuumi.dev@gmail.com> Co-authored-by: Chris Simpkins <git.simpkins@gmail.com>
This commit is contained in:
Who? Me?!
GitHub
parent
e2ea943091
commit
10a4d76ecd
|
|
@ -33,11 +33,10 @@
|
|||
- ["Cleanup Crew" ICE-breakers](ice-breaker/cleanup-crew.md)
|
||||
- [LLVM ICE-breakers](ice-breaker/llvm.md)
|
||||
- [Licenses](./licenses.md)
|
||||
- [Part 2: How rustc works](./part-2-intro.md)
|
||||
- [High-level overview of the compiler source](./high-level-overview.md)
|
||||
- [The Rustc Driver and Interface](./rustc-driver.md)
|
||||
- [Rustdoc](./rustdoc.md)
|
||||
- [Ex: Type checking through `rustc_interface`](./rustc-driver-interacting-with-the-ast.md)
|
||||
|
||||
- [Part 2: High-level Compiler Architecture](./part-2-intro.md)
|
||||
- [Overview of the Compiler](./overview.md)
|
||||
- [The compiler source code](./compiler-src.md)
|
||||
- [Queries: demand-driven compilation](./query.md)
|
||||
- [The Query Evaluation Model in Detail](./queries/query-evaluation-model-in-detail.md)
|
||||
- [Incremental compilation](./queries/incremental-compilation.md)
|
||||
|
|
@ -46,15 +45,29 @@
|
|||
- [Profiling Queries](./queries/profiling.md)
|
||||
- [Salsa](./salsa.md)
|
||||
- [Memory Management in Rustc](./memory.md)
|
||||
- [Lexing and Parsing](./the-parser.md)
|
||||
- [`#[test]` Implementation](./test-implementation.md)
|
||||
- [Panic Implementation](./panic-implementation.md)
|
||||
- [Macro expansion](./macro-expansion.md)
|
||||
- [Name resolution](./name-resolution.md)
|
||||
|
||||
- [Part 3: Source Code Representations](./part-3-intro.md)
|
||||
- [The Rustc Driver and Interface](./rustc-driver.md)
|
||||
- [Rustdoc](./rustdoc.md)
|
||||
- [Ex: Type checking through `rustc_interface`](./rustc-driver-interacting-with-the-ast.md)
|
||||
- [Syntax and the AST](./syntax-intro.md)
|
||||
- [Lexing and Parsing](./the-parser.md)
|
||||
- [`#[test]` Implementation](./test-implementation.md)
|
||||
- [Panic Implementation](./panic-implementation.md)
|
||||
- [Macro expansion](./macro-expansion.md)
|
||||
- [Name resolution](./name-resolution.md)
|
||||
- [AST Validation](./ast-validation.md)
|
||||
- [Feature Gate Checking](./feature-gate-ck.md)
|
||||
- [The HIR (High-level IR)](./hir.md)
|
||||
- [Lowering AST to HIR](./lowering.md)
|
||||
- [Debugging](./hir-debugging.md)
|
||||
- [The MIR (Mid-level IR)](./mir/index.md)
|
||||
- [HAIR and MIR construction](./mir/construction.md)
|
||||
- [MIR visitor and traversal](./mir/visitor.md)
|
||||
- [MIR passes: getting the MIR for a function](./mir/passes.md)
|
||||
- [Closure expansion](./closure.md)
|
||||
|
||||
- [Part 4: Analysis](./part-4-intro.md)
|
||||
- [The `ty` module: representing types](./ty.md)
|
||||
- [Generics and substitutions](./generics.md)
|
||||
- [`TypeFolder` and `TypeFoldable`](./ty-fold.md)
|
||||
|
|
@ -82,12 +95,7 @@
|
|||
- [Method Lookup](./method-lookup.md)
|
||||
- [Variance](./variance.md)
|
||||
- [Opaque Types](./opaque-types-type-alias-impl-trait.md)
|
||||
- [The MIR (Mid-level IR)](./mir/index.md)
|
||||
- [MIR construction](./mir/construction.md)
|
||||
- [MIR visitor and traversal](./mir/visitor.md)
|
||||
- [MIR passes: getting the MIR for a function](./mir/passes.md)
|
||||
- [MIR optimizations](./mir/optimizations.md)
|
||||
- [Debugging](./mir/debugging.md)
|
||||
- [Pattern and Exhaustiveness Checking](./pat-exhaustive-checking.md)
|
||||
- [The borrow checker](./borrow_check.md)
|
||||
- [Tracking moves and initialization](./borrow_check/moves_and_initialization.md)
|
||||
- [Move paths](./borrow_check/moves_and_initialization/move_paths.md)
|
||||
|
|
@ -100,20 +108,24 @@
|
|||
- [Closure constraints](./borrow_check/region_inference/closure_constraints.md)
|
||||
- [Error reporting](./borrow_check/region_inference/error_reporting.md)
|
||||
- [Two-phase-borrows](./borrow_check/two_phase_borrows.md)
|
||||
- [Parameter Environments](./param_env.md)
|
||||
|
||||
- [Part 5: From MIR to binaries](./part-5-intro.md)
|
||||
- [The MIR (Mid-level IR)](./mir/index.md)
|
||||
- [MIR optimizations](./mir/optimizations.md)
|
||||
- [Debugging](./mir/debugging.md)
|
||||
- [Constant evaluation](./const-eval.md)
|
||||
- [miri const evaluator](./miri.md)
|
||||
- [Parameter Environments](./param_env.md)
|
||||
- [Compiler Backend](./backend/backend.md)
|
||||
- [Monomorphization](./backend/monomorph.md)
|
||||
- [Lowering MIR](./backend/lowering-mir.md)
|
||||
- [Code Generation](./backend/codegen.md)
|
||||
- [Updating LLVM](./backend/updating-llvm.md)
|
||||
- [Debugging LLVM](./backend/debugging.md)
|
||||
- [Backend Agnostic Codegen](./backend/backend-agnostic.md)
|
||||
- [Implicit Caller Location](./codegen/implicit-caller-location.md)
|
||||
- [Profile-guided Optimization](./profile-guided-optimization.md)
|
||||
- [Sanitizers Support](./sanitizers.md)
|
||||
- [Debugging Support in Rust Compiler](./debugging-support-in-rustc.md)
|
||||
- [Monomorphization](./backend/monomorph.md)
|
||||
- [Lowering MIR](./backend/lowering-mir.md)
|
||||
- [Code Generation](./backend/codegen.md)
|
||||
- [Updating LLVM](./backend/updating-llvm.md)
|
||||
- [Debugging LLVM](./backend/debugging.md)
|
||||
- [Backend Agnostic Codegen](./backend/backend-agnostic.md)
|
||||
- [Implicit Caller Location](./codegen/implicit-caller-location.md)
|
||||
- [Profile-guided Optimization](./profile-guided-optimization.md)
|
||||
- [Sanitizers Support](./sanitizers.md)
|
||||
- [Debugging Support in Rust Compiler](./debugging-support-in-rustc.md)
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -2,14 +2,29 @@
|
|||
|
||||
This guide is meant to help document how rustc – the Rust compiler –
|
||||
works, as well as to help new contributors get involved in rustc
|
||||
development. It is not meant to replace code documentation – each
|
||||
chapter gives only high-level details, the kinds of things that
|
||||
(ideally) don't change frequently.
|
||||
development.
|
||||
|
||||
There are three parts to this guide. Part 1 contains information that should
|
||||
be useful no matter how you are contributing. Part 2 contains information
|
||||
about how the compiler works. Finally, there are some appendices at the
|
||||
end with useful reference information.
|
||||
There are six parts to this guide:
|
||||
|
||||
1. [Contributing][p1]: Contains information that should be useful no matter how
|
||||
you are contributing, such as procedures for contribution, building the
|
||||
compiler, etc.
|
||||
2. [High-level Compiler Architecture][p2]: Discusses the high-level
|
||||
architecture of the compiler, especially the query system.
|
||||
3. [The Compiler Frontend][p3]: Discusses the compiler frontend and internal
|
||||
representations.
|
||||
4. [The Type System][p4]: Discusses the type system.
|
||||
5. [The Compiler Backend][p5]: Discusses the compiler backend, code generation,
|
||||
linking, and debug info.
|
||||
6. [Appendices][app] at the end with useful reference information. There are a
|
||||
few of these with different information, inluding a glossary.
|
||||
|
||||
[p1]: ./part-1-intro.md
|
||||
[p2]: ./part-2-intro.md
|
||||
[p3]: ./part-3-intro.md
|
||||
[p4]: ./part-4-intro.md
|
||||
[p5]: ./part-5-intro.md
|
||||
[app]: ./appendix/background.md
|
||||
|
||||
The guide itself is of course open-source as well, and the sources can
|
||||
be found at the [GitHub repository]. If you find any mistakes in the
|
||||
|
|
|
|||
|
|
@ -0,0 +1,6 @@
|
|||
# AST Validation
|
||||
|
||||
AST validation is the process of checking various correctness properties about
|
||||
the AST after macro expansion.
|
||||
|
||||
**TODO**: write this chapter.
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
# Feature Gate Checking
|
||||
|
||||
**TODO**: this chapter
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
# MIR construction
|
||||
# HAIR and MIR construction
|
||||
|
||||
The lowering of [HIR] to [MIR] occurs for the following (probably incomplete)
|
||||
list of items:
|
||||
|
|
|
|||
|
|
@ -0,0 +1,3 @@
|
|||
# Overview of the Compiler
|
||||
|
||||
Coming soon! Work is in progress on this chapter. See https://github.com/rust-lang/rustc-dev-guide/pull/633 for the source and the [project README](https://github.com/rust-lang/rustc-dev-guide) for local build instructions.
|
||||
|
|
@ -1,12 +1,15 @@
|
|||
# Part 2: How rustc works
|
||||
|
||||
This part of the guide describes how the compiler works. It goes through
|
||||
everything from high-level structure of the compiler to how each stage of
|
||||
compilation works.
|
||||
The remaining parts of this guide discuss how the compiler works. They go
|
||||
through everything from high-level structure of the compiler to how each stage
|
||||
of compilation works. They should be friendly to both readers interested in the
|
||||
end-to-end process of compilation _and_ readers interested in learning about a
|
||||
specific system they wish to contribute to. If anything is unclear, feel free
|
||||
to file an issue on the [rustc-dev-guide
|
||||
repo](https://github.com/rust-lang/rustc-dev-guide/issues) or contact the compiler
|
||||
team, as detailed in [this chapter from Part 1](./compiler-team.md).
|
||||
|
||||
This section should be friendly to both readers interested in the end-to-end
|
||||
process of compilation _and_ readers interested in learning about a specific
|
||||
system they wish to contribute to. If anything is unclear, feel free to file
|
||||
an issue on the [rustc-dev-guide repo](https://github.com/rust-lang/rustc-dev-guide)
|
||||
or contact the compiler team, as detailed in [this chapter from Part
|
||||
1](./compiler-team.md).
|
||||
In this part, we will specifically look at the high-level architecture of the
|
||||
compiler. Specifically, will look at the query system, incremental compilation,
|
||||
and interning. These are three overarching design choices that impact the whole
|
||||
compiler.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,8 @@
|
|||
# Part 3: Source Code Representations
|
||||
|
||||
This part describes the process of taking raw source code from the user and
|
||||
transforming it into various forms that the compiler can work with easily.
|
||||
These are called intermediate representations.
|
||||
|
||||
This process starts with compiler understanding what the user has asked for:
|
||||
parsing the command line arguments given and determining what it is to compile.
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
# Part 4: Analysis
|
||||
|
||||
This part discusses the many analyses that the compiler uses to check various
|
||||
properties of the code and to inform later stages. Typically, this is what people
|
||||
mean when they talk about "Rust's type system". This includes the
|
||||
representation, inference, and checking of types, the trait system, and the
|
||||
borrow checker. These analyses do not happen as one big pass or set of
|
||||
contiguous passes. Rather, they are spread out throughout various parts of the
|
||||
compilation process and use different intermediate representations. For example,
|
||||
type checking happens on the HIR, while borrow checking happens on the MIR.
|
||||
Nonetheless, for the sake of presentation, we will discuss all of these
|
||||
analyses in this part of the guide.
|
||||
|
|
@ -1,19 +1,20 @@
|
|||
# The Compiler Backend
|
||||
# From MIR to binaries
|
||||
|
||||
All of the preceding chapters of this guide have one thing in common: we never
|
||||
generated any executable machine code at all! With this chapter, all of that
|
||||
changes.
|
||||
|
||||
It's often useful to think of compilers as being composed of a _frontend_ and a
|
||||
_backend_ (though in rustc, there's not a sharp line between frontend and
|
||||
backend). The _frontend_ is responsible for taking raw source code, checking it
|
||||
for correctness, and getting it into a format usable by the backend. For rustc,
|
||||
this format is the MIR. The _backend_ refers to the parts of the compiler that
|
||||
turn rustc's MIR into actual executable code (e.g. an ELF or EXE binary) that
|
||||
can run on a processor. All of the previous chapters deal with rustc's
|
||||
frontend.
|
||||
So far, we've shown how the compiler can take raw source code in text format
|
||||
and transform it into MIR. We have also shown how the compiler does various
|
||||
analyses on the code to detect things like type or lifetime errors. Now, we
|
||||
will finally take the MIR and produce some executable machine code.
|
||||
|
||||
rustc's backend does the following:
|
||||
> NOTE: This part of a compiler is often called the _backend_ the term is a bit
|
||||
> overloaded because in the compiler source, it usually refers to the "codegen
|
||||
> backend" (i.e. LLVM or Cranelift). Usually, when you see the word "backend"
|
||||
> in this part, we are refering to the "codegen backend".
|
||||
|
||||
So what do we need to do?
|
||||
|
||||
0. First, we need to collect the set of things to generate code for. In
|
||||
particular, we need to find out which concrete types to substitute for
|
||||
|
|
@ -23,15 +24,15 @@ rustc's backend does the following:
|
|||
collecting all the concrete types is called _monomorphization collection_.
|
||||
1. Next, we need to actually lower the MIR to a codegen IR
|
||||
(usually LLVM IR) for each concrete type we collected.
|
||||
2. Finally, we need to invoke the codegen backend (e.g. LLVM or Cranelift),
|
||||
which runs a bunch of optimization passes, generates executable code, and
|
||||
links together an executable binary.
|
||||
2. Finally, we need to invoke LLVM or Cranelift, which runs a bunch of
|
||||
optimization passes, generates executable code, and links together an
|
||||
executable binary.
|
||||
|
||||
[codegen1]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/base/fn.codegen_crate.html
|
||||
|
||||
The code for codegen is actually a bit complex due to a few factors:
|
||||
|
||||
- Support for multiple backends (LLVM and Cranelift). We try to share as much
|
||||
- Support for multiple codegen backends (LLVM and Cranelift). We try to share as much
|
||||
backend code between them as possible, so a lot of it is generic over the
|
||||
codegen implementation. This means that there are often a lot of layers of
|
||||
abstraction.
|
||||
|
|
@ -48,3 +49,4 @@ crate contains code specific to LLVM codegen.
|
|||
At a very high level, the entry point is
|
||||
[`rustc_codegen_ssa::base::codegen_crate`][codegen1]. This function starts the
|
||||
process discussed in the rest of this chapter.
|
||||
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
# Pattern and Exhaustiveness Checking
|
||||
|
||||
In Rust, pattern matching and bindings have a few very helpful properties. The
|
||||
compiler will check that bindings are irrefutable when made and that match arms
|
||||
are exhaustive.
|
||||
|
||||
**TODO**: write this chapter.
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
# Syntax and the AST
|
||||
|
||||
Working directly with source code is very inconvenient and error-prone. Thus,
|
||||
before we do anything else, we convert raw source code into an AST. It turns
|
||||
out that doing even this involves a lot of work, including lexing, parsing,
|
||||
macro expansion, name resolution, conditional compilation, feature-gate
|
||||
checking, and validation of the AST. In this chapter, we take a look at all
|
||||
of these steps.
|
||||
Loading…
Reference in New Issue