diff --git a/src/appendix/code-index.md b/src/appendix/code-index.md index 1b53e642..0321da19 100644 --- a/src/appendix/code-index.md +++ b/src/appendix/code-index.md @@ -8,16 +8,16 @@ Item | Kind | Short description | Chapter | ----------------|----------|-----------------------------|--------------------|------------------- `BodyId` | struct | One of four types of HIR node identifiers | [Identifiers in the HIR] | [src/librustc_hir/hir.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/struct.BodyId.html) `Compiler` | struct | Represents a compiler session and can be used to drive a compilation. | [The Rustc Driver and Interface] | [src/librustc_interface/interface.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_interface/interface/struct.Compiler.html) -`ast::Crate` | struct | A syntax-level representation of a parsed crate | [The parser] | [src/libsyntax/ast.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/struct.Crate.html) +`ast::Crate` | struct | A syntax-level representation of a parsed crate | [The parser] | [src/librustc_ast/ast.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast/ast/struct.Crate.html) `rustc_hir::Crate` | struct | A more abstract, compiler-friendly form of a crate's AST | [The Hir] | [src/librustc_hir/hir.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/struct.Crate.html) `DefId` | struct | One of four types of HIR node identifiers | [Identifiers in the HIR] | [src/librustc_hir/def_id.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/def_id/struct.DefId.html) `DiagnosticBuilder` | struct | A struct for building up compiler diagnostics, such as errors or lints | [Emitting Diagnostics] | [src/librustc_errors/diagnostic_builder.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.DiagnosticBuilder.html) `DocContext` | struct | A state container used by rustdoc when crawling through a crate to gather its documentation | [Rustdoc] | [src/librustdoc/core.rs](https://github.com/rust-lang/rust/blob/master/src/librustdoc/core.rs) `HirId` | struct | One of four types of HIR node identifiers | [Identifiers in the HIR] | [src/librustc_hir/hir_id.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir_id/struct.HirId.html) -`NodeId` | struct | One of four types of HIR node identifiers. Being phased out | [Identifiers in the HIR] | [src/libsyntax/ast.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/node_id/struct.NodeId.html) -`P` | struct | An owned immutable smart pointer. By contrast, `&T` is not owned, and `Box` is not immutable. | None | [src/syntax/ptr.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ptr/struct.P.html) +`NodeId` | struct | One of four types of HIR node identifiers. Being phased out | [Identifiers in the HIR] | [src/librustc_ast/ast.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast/node_id/struct.NodeId.html) +`P` | struct | An owned immutable smart pointer. By contrast, `&T` is not owned, and `Box` is not immutable. | None | [src/librustc_ast/ptr.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast/ptr/struct.P.html) `ParamEnv` | struct | Information about generic parameters or `Self`, useful for working with associated or generic items | [Parameter Environment] | [src/librustc/ty/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/struct.ParamEnv.html) -`ParseSess` | struct | This struct contains information about a parsing session | [The parser] | [src/libsyntax/parse/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_session/parse/struct.ParseSess.html) +`ParseSess` | struct | This struct contains information about a parsing session | [The parser] | [src/librustc_session/parse/parse.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_session/parse/struct.ParseSess.html) `Query` | struct | Represents the result of query to the `Compiler` interface and allows stealing, borrowing, and returning the results of compiler passes. | [The Rustc Driver and Interface] | [src/librustc_interface/queries.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_interface/queries/struct.Query.html) `Rib` | struct | Represents a single scope of names | [Name resolution] | [src/librustc_resolve/lib.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_resolve/late/struct.Rib.html) `Session` | struct | The data associated with a compilation session | [The parser], [The Rustc Driver and Interface] | [src/librustc/session/mod.html](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_session/struct.Session.html) @@ -25,7 +25,7 @@ Item | Kind | Short description | Chapter | `SourceMap` | struct | Maps AST nodes to their source code. It is composed of `SourceFile`s. Was previously called CodeMap | [The parser] | [src/librustc_span/source_map.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_span/source_map/struct.SourceMap.html) `Span` | struct | A location in the user's source code, used for error reporting primarily | [Emitting Diagnostics] | [src/librustc_span/span_encoding.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_span/struct.Span.html) `StringReader` | struct | This is the lexer used during parsing. It consumes characters from the raw source code being compiled and produces a series of tokens for use by the rest of the parser | [The parser] | [src/librustc_parse/lexer/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/lexer/struct.StringReader.html) -`syntax::token_stream::TokenStream` | struct | An abstract sequence of tokens, organized into `TokenTree`s | [The parser], [Macro expansion] | [src/libsyntax/tokenstream.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/tokenstream/struct.TokenStream.html) +`rustc_ast::token_stream::TokenStream` | struct | An abstract sequence of tokens, organized into `TokenTree`s | [The parser], [Macro expansion] | [src/librustc_ast/tokenstream.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast/tokenstream/struct.TokenStream.html) `TraitDef` | struct | This struct contains a trait's definition with type information | [The `ty` modules] | [src/librustc/ty/trait_def.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/trait_def/struct.TraitDef.html) `TraitRef` | struct | The combination of a trait and its input types (e.g. `P0: Trait`) | [Trait Solving: Goals and Clauses], [Trait Solving: Lowering impls] | [src/librustc/ty/sty.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/struct.TraitRef.html) `Ty<'tcx>` | struct | This is the internal representation of a type used for type checking | [Type checking] | [src/librustc/ty/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/type.Ty.html) diff --git a/src/appendix/glossary.md b/src/appendix/glossary.md index fbd76f25..f5274a8f 100644 --- a/src/appendix/glossary.md +++ b/src/appendix/glossary.md @@ -7,7 +7,7 @@ them better. Term | Meaning ------------------------|-------- arena/arena allocation | an _arena_ is a large memory buffer from which other memory allocations are made. This style of allocation is called _area allocation_. See [this chapter](../memory.md) for more info. -AST | the abstract syntax tree produced by the syntax crate; reflects user syntax very closely. +AST | the abstract syntax tree produced by the `rustc_ast` crate; reflects user syntax very closely. binder | a "binder" is a place where a variable or type is declared; for example, the `` is a binder for the generic type parameter `T` in `fn foo(..)`, and \|`a`\|` ...` is a binder for the parameter `a`. See [the background chapter for more](./background.html#free-vs-bound) bound variable | a "bound variable" is one that is declared within an expression/term. For example, the variable `a` is bound within the closure expression \|`a`\|` a * 2`. See [the background chapter for more](./background.html#free-vs-bound) codegen | the code to translate MIR into LLVM IR. diff --git a/src/appendix/stupid-stats.md b/src/appendix/stupid-stats.md index 6ed7d1cb..ea9b6e0f 100644 --- a/src/appendix/stupid-stats.md +++ b/src/appendix/stupid-stats.md @@ -61,7 +61,7 @@ Macros and syntax extensions are expanded, and `cfg` attributes will cause some code to disappear. The resulting AST won't have any macros or macro uses left in. -The code for these first two phases is in [libsyntax](https://github.com/rust-lang/rust/tree/master/src/libsyntax). +The code for these first two phases is in [librustc_ast](https://github.com/rust-lang/rust/tree/master/src/librustc_ast). After this phase, the compiler allocates ids to each node in the AST (technically not every node, but most of them). If we are writing out @@ -356,7 +356,7 @@ struct StupidVisitor { The `StupidVisitor` struct just keeps track of the number of `println!`s it has seen and the count for each number of arguments. It implements -`syntax::visit::Visitor` to walk the AST. Mostly we just use the default +`rustc_ast::visit::Visitor` to walk the AST. Mostly we just use the default methods, these walk the AST taking no action. We override `visit_item` and `visit_mac` to implement custom behaviour when we walk into items (items include functions, modules, traits, structs, and so forth, we're only interested in diff --git a/src/bug-fix-procedure.md b/src/bug-fix-procedure.md index 0f6d6c97..90d8fb48 100644 --- a/src/bug-fix-procedure.md +++ b/src/bug-fix-procedure.md @@ -65,7 +65,7 @@ described in more detail below): 1. Once the change has been in the wild for at least one cycle, we can **stabilize the change**, converting those warnings into errors. -Finally, for changes to libsyntax that will affect plugins, the general policy +Finally, for changes to `librustc_ast` that will affect plugins, the general policy is to batch these changes. That is discussed below in more detail. ### Tracking issue diff --git a/src/diagnostics.md b/src/diagnostics.md index e65893cf..c112f347 100644 --- a/src/diagnostics.md +++ b/src/diagnostics.md @@ -322,12 +322,12 @@ like normal but invokes the lint with `buffer_lint`. #### Linting even earlier in the compiler -The parser (`libsyntax`) is interesting in that it cannot have dependencies on +The parser (`librustc_ast`) is interesting in that it cannot have dependencies on any of the other `librustc*` crates. In particular, it cannot depend on `librustc::lint` or `librustc_lint`, where all of the compiler linting infrastructure is defined. That's troublesome! -To solve this, `libsyntax` defines its own buffered lint type, which +To solve this, `librustc_ast` defines its own buffered lint type, which `ParseSess::buffer_lint` uses. After macro expansion, these buffered lints are then dumped into the `Session::buffered_lints` used by the rest of the compiler. @@ -358,7 +358,7 @@ the structured JSON and see the "human" output (well, _sans_ colors) without having to compile everything twice. The "human" readable and the json format emitter can be found under -librustc_errors, both were moved from the libsyntax crate to the +librustc_errors, both were moved from the `librustc_ast` crate to the [librustc_errors crate](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/index.html). The JSON emitter defines [its own `Diagnostic` diff --git a/src/high-level-overview.md b/src/high-level-overview.md index e4b9ff37..76f4bc95 100644 --- a/src/high-level-overview.md +++ b/src/high-level-overview.md @@ -7,7 +7,7 @@ there live many crates. These crates contain the sources for the standard library and the compiler. This document, of course, focuses on the latter. -Rustc consists of a number of crates, including `syntax`, +Rustc consists of a number of crates, including `rustc_ast`, `rustc`, `rustc_target`, `rustc_codegen`, `rustc_driver`, and many more. The source for each crate can be found in a directory like `src/libXXX`, where `XXX` is the crate name. @@ -33,7 +33,7 @@ rustc_codegen rustc_borrowck ... rustc_metadata rustc | v - syntax + rustc_ast / \ / \ rustc_span rustc_builtin_macros @@ -92,7 +92,7 @@ take: 1. **Parsing input** - this processes the `.rs` files and produces the AST ("abstract syntax tree") - - the AST is defined in `src/libsyntax/ast.rs`. It is intended to match the lexical + - the AST is defined in `src/librustc_ast/ast.rs`. It is intended to match the lexical syntax of the Rust language quite closely. 2. **Name resolution, macro expansion, and configuration** - once parsing is complete, we process the AST recursively, resolving diff --git a/src/hir.md b/src/hir.md index 28cc83c5..2c3e44a7 100644 --- a/src/hir.md +++ b/src/hir.md @@ -100,7 +100,7 @@ sorts of identifiers in active use: [`DefId`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/def_id/struct.DefId.html [`HirId`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir_id/struct.HirId.html [`BodyId`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/struct.BodyId.html -[`NodeId`]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/node_id/struct.NodeId.html +[`NodeId`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast/node_id/struct.NodeId.html We also have an internal map to go from `DefId` to what’s called "Def path". "Def path" is like a module path but a bit more rich. For example, it may be `crate::foo::MyStruct` that identifies diff --git a/src/macro-expansion.md b/src/macro-expansion.md index bc3b9dc5..eb9800c6 100644 --- a/src/macro-expansion.md +++ b/src/macro-expansion.md @@ -1,6 +1,6 @@ # Macro expansion -> `libsyntax`, `librustc_expand`, and `librustc_builtin_macros` are all undergoing +> `librustc_ast`, `librustc_expand`, and `librustc_builtin_macros` are all undergoing > refactoring, so some of the links in this chapter may be broken. Macro expansion happens during parsing. `rustc` has two parsers, in fact: the @@ -361,24 +361,24 @@ be accessed from any Ident without any context) librustc_span/lib.rs - some secondary methods like macro backtrace using primary methods from hygiene.rs librustc_builtin_macros - implementations of built-in macros (including macro attributes and derives) and some other early code generation facilities like injection of -standard library imports or generation of test harness. libsyntax/config.rs - +standard library imports or generation of test harness. librustc_ast/config.rs - implementation of cfg/cfg_attr (they treated specially from other macros), -should probably be moved into libsyntax/ext. libsyntax/tokenstream.rs + -libsyntax/parse/token.rs - structures for compiler-side tokens, token trees, -and token streams. libsyntax/ext - various expansion-related stuff -libsyntax/ext/base.rs - basic structures used by expansion -libsyntax/ext/expand.rs - some expansion structures and the bulk of expansion +should probably be moved into librustc_ast/ext. librustc_ast/tokenstream.rs + +librustc_ast/parse/token.rs - structures for compiler-side tokens, token trees, +and token streams. librustc_ast/ext - various expansion-related stuff +librustc_ast/ext/base.rs - basic structures used by expansion +librustc_ast/ext/expand.rs - some expansion structures and the bulk of expansion infrastructure code - collecting macro invocations, calling into resolve for them, calling their expanding functions, and integrating the results back into -AST libsyntax/ext/placeholder.rs - the part of expand.rs responsible for +AST librustc_ast/ext/placeholder.rs - the part of expand.rs responsible for "integrating the results back into AST" basicallly, "placeholder" is a temporary AST node replaced with macro expansion result nodes -libsyntax/ext/builer.rs - helper functions for building AST for built-in macros +librustc_ast/ext/builer.rs - helper functions for building AST for built-in macros in librustc_builtin_macros (and user-defined syntactic plugins previously), can probably -be moved into librustc_builtin_macros these days libsyntax/ext/proc_macro.rs + -libsyntax/ext/proc_macro_server.rs - interfaces between the compiler and the +be moved into librustc_builtin_macros these days librustc_ast/ext/proc_macro.rs + +librustc_ast/ext/proc_macro_server.rs - interfaces between the compiler and the stable proc_macro library, converting tokens and token streams between the two -representations and sending them through C ABI libsyntax/ext/tt - +representations and sending them through C ABI librustc_ast/ext/tt - implementation of macro_rules, turns macro_rules DSL into something with signature Fn(TokenStream) -> TokenStream that can eat and produce tokens, @mark-i-m knows more about this librustc_resolve/macros.rs - resolving macro @@ -408,8 +408,8 @@ piece of AST, etc), this is an enum that lists them ProcMacro/TTMacroExpander/AttrProcMacro/MultiItemModifier - traits representing the expander signatures (TODO: change and rename the signatures into something more consistent) trait Resolver - a trait used to break crate dependencies (so -resolver services can be used in libsyntax, despite librustc_resolve and pretty -much everything else depending on libsyntax) ExtCtxt/ExpansionData - various +resolver services can be used in librustc_ast, despite librustc_resolve and pretty +much everything else depending on librustc_ast) ExtCtxt/ExpansionData - various intermediate data kept and used by expansion infra in the process of its work AstFragment - a piece of AST that can be produced by a macro (may include multiple homogeneous AST nodes, like e.g. a list of items) Annotatable - a diff --git a/src/name-resolution.md b/src/name-resolution.md index 50820d97..6dfbd91a 100644 --- a/src/name-resolution.md +++ b/src/name-resolution.md @@ -20,7 +20,7 @@ namespaces and therefore can co-exist. The name resolution in Rust is a two-phase process. In the first phase, which runs during macro expansion, we build a tree of modules and resolve imports. Macro expansion and name resolution communicate with each other via the `Resolver` -trait, defined in `libsyntax`. +trait, defined in `librustc_ast`. The input to the second phase is the syntax tree, produced by parsing input files and expanding macros. This phase produces links from all the names in the diff --git a/src/test-implementation.md b/src/test-implementation.md index 59cc7c5b..e5bcedf1 100644 --- a/src/test-implementation.md +++ b/src/test-implementation.md @@ -31,14 +31,14 @@ How does any sort of `main` function invoke these tests if they're not visible? What exactly is `rustc --test` doing? `#[test]` is implemented as a syntactic transformation inside the compiler's -[`libsyntax` crate][libsyntax]. Essentially, it's a fancy macro, that +[`librustc_ast` crate][librustc_ast]. Essentially, it's a fancy macro, that rewrites the crate in 3 steps: #### Step 1: Re-Exporting As mentioned earlier, tests can exist inside private modules, so we need a way of exposing them to the main function, without breaking any existing -code. To that end, `libsyntax` will create local modules called +code. To that end, `librustc_ast` will create local modules called `__test_reexports` that recursively reexport tests. This expansion translates the above example into: @@ -78,7 +78,7 @@ hygiene. #### Step 2: Harness Generation Now that our tests are accessible from the root of our crate, we need to do -something with them. `libsyntax` generates a module like so: +something with them. `librustc_ast` generates a module like so: ```rust,ignore #[main] @@ -115,7 +115,7 @@ fn foo() { This means our tests are more than just simple functions, they have configuration information as well. `test` encodes this configuration data into a struct called [`TestDesc`][TestDesc]. For each test function in a -crate, `libsyntax` will parse its attributes and generate a `TestDesc` +crate, `librustc_ast` will parse its attributes and generate a `TestDesc` instance. It then combines the `TestDesc` and test function into the predictably named `TestDescAndFn` struct, that `test_main_static` operates on. For a given test, the generated `TestDescAndFn` instance looks like so: @@ -146,7 +146,7 @@ $ rustc my_mod.rs -Z unpretty=hir [test]: https://doc.rust-lang.org/test/index.html [TestDesc]: https://doc.rust-lang.org/test/struct.TestDesc.html -[Symbol]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/struct.Ident.html -[Ident]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/struct.Ident.html +[Symbol]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast/ast/struct.Ident.html +[Ident]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast/ast/struct.Ident.html [eRFC]: https://github.com/rust-lang/rfcs/blob/master/text/2318-custom-test-frameworks.md -[libsyntax]: https://github.com/rust-lang/rust/tree/master/src/libsyntax +[librustc_ast]: https://github.com/rust-lang/rust/tree/master/src/librustc_ast diff --git a/src/the-parser.md b/src/the-parser.md index 3584d73e..8b8ef77c 100644 --- a/src/the-parser.md +++ b/src/the-parser.md @@ -18,7 +18,7 @@ form which is easier for the compiler to work with, usually called an [*Abstract Syntax Tree*][ast] (AST). An AST mirrors the structure of a Rust program in memory, using a `Span` to link a particular AST node back to its source text. -The AST is defined in [`libsyntax`][libsyntax], along with some definitions for +The AST is defined in [`librustc_ast`][librustc_ast], along with some definitions for tokens and token streams, data structures/traits for mutating ASTs, and shared definitions for other AST-related parts of the compiler (like the lexer and macro-expansion). @@ -46,18 +46,18 @@ Code for lexical analysis is split between two crates: constituting tokens. Although it is popular to implement lexers as generated finite state machines, the lexer in `rustc_lexer` is hand-written. -- [`StringReader`] from [libsyntax] integrates `rustc_lexer` with `rustc` +- [`StringReader`] from [`librustc_ast`][librustc_ast] integrates `rustc_lexer` with `rustc` specific data structures. Specifically, it adds `Span` information to tokens returned by `rustc_lexer` and interns identifiers. -[libsyntax]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/index.html +[librustc_ast]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast/index.html [rustc_errors]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/index.html [ast]: https://en.wikipedia.org/wiki/Abstract_syntax_tree [`SourceMap`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_span/source_map/struct.SourceMap.html -[ast module]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/index.html +[ast module]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast/ast/index.html [librustc_parse]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/index.html [parser]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/parser/index.html -[`Parser`]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/parse/parser/struct.Parser.html +[`Parser`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast/parse/parser/struct.Parser.html [`StringReader`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/lexer/struct.StringReader.html -[visit module]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/visit/index.html +[visit module]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_ast/visit/index.html [sourcefile]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_span/struct.SourceFile.html