e9b51bd0ec
Avoid no-op unlink+link dances in incr comp Incremental compilation scales quite poorly with the number of CGUs. This PR improves one reason for that. The incr comp process hard-links all the files from an old session into a new one, then it runs the backend, which may just hard-link the new session files into the output directory. Then codegen hard-links all the output files back to the new session directory. This PR (perhaps unimaginatively) fixes the silliness that ensues in the last step. The old `link_or_copy` implementation would be passed pairs of paths which are already the same inode, then it would blindly delete the destination and re-create the hard-link that it just deleted. This PR lets us skip both those operations. We don't skip the other two hard-links. `cargo +stage1 b && touch crates/core/main.rs && strace -cfw -elink,linkat,unlink,unlinkat cargo +stage1 b` before and then after on `ripgrep-13.0.0`: ``` % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 52.56 0.024950 25 978 485 unlink 34.38 0.016318 22 727 linkat 13.06 0.006200 24 249 unlinkat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.047467 24 1954 485 total ``` ``` % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 42.83 0.014521 57 252 unlink 38.41 0.013021 26 486 linkat 18.77 0.006362 25 249 unlinkat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.033904 34 987 total ``` This reduces the number of hard-links that are causing perf troubles, noted in https://github.com/rust-lang/rust/issues/64291 and https://github.com/rust-lang/rust/issues/137560 |
||
|---|---|---|
| .github/workflows | ||
| ci | ||
| examples | ||
| josh-sync | ||
| src | ||
| .editorconfig | ||
| .gitattributes | ||
| .gitignore | ||
| .mailmap | ||
| CITATION.cff | ||
| CNAME | ||
| CODE_OF_CONDUCT.md | ||
| LICENSE-APACHE | ||
| LICENSE-MIT | ||
| README.md | ||
| book.toml | ||
| mermaid-init.js | ||
| mermaid.min.js | ||
| rust-version | ||
| triagebot.toml | ||
README.md
This is a collaborative effort to build a guide that explains how rustc works. The aim of the guide is to help new contributors get oriented to rustc, as well as to help more experienced folks in figuring out some new part of the compiler that they haven't worked on before.
You can read the latest version of the guide here.
You may also find the rustdocs for the compiler itself useful. Note that these are not intended as a guide; it's recommended that you search for the docs you're looking for instead of reading them top to bottom.
For documentation on developing the standard library, see
std-dev-guide.
Contributing to the guide
The guide is useful today, but it has a lot of work still to go.
If you'd like to help improve the guide, we'd love to have you! You can find plenty of issues on the issue tracker. Just post a comment on the issue you would like to work on to make sure that we don't accidentally duplicate work. If you think something is missing, please open an issue about it!
In general, if you don't know how the compiler works, that is not a problem! In that case, what we will do is to schedule a bit of time for you to talk with someone who does know the code, or who wants to pair with you and figure it out. Then you can work on writing up what you learned.
In general, when writing about a particular part of the compiler's code, we recommend that you link to the relevant parts of the rustc rustdocs.
Build Instructions
To build a local static HTML site, install mdbook with:
> cargo install mdbook mdbook-linkcheck2 mdbook-toc mdbook-mermaid
and execute the following command in the root of the repository:
> mdbook build --open
The build files are found in the book/html directory.
Link Validations
We use mdbook-linkcheck2 to validate URLs included in our documentation. Link
checking is not run by default locally, though it is in CI. To enable it
locally, set the environment variable ENABLE_LINKCHECK=1 like in the
following example.
$ ENABLE_LINKCHECK=1 mdbook serve
Table of Contents
We use mdbook-toc to auto-generate TOCs for long sections. You can invoke the preprocessor by
including the <!-- toc --> marker at the place where you want the TOC.
Synchronizing josh subtree with rustc
This repository is linked to rust-lang/rust as a josh subtree. You can use the following commands to synchronize the subtree in both directions.
You'll need to install josh-proxy locally via
cargo +stable install josh-proxy --git https://github.com/josh-project/josh --tag r24.10.04
Older versions of josh-proxy may not round trip commits losslessly so it is important to install this exact version.
Pull changes from rust-lang/rust into this repository
- Checkout a new branch that will be used to create a PR into
rust-lang/rustc-dev-guide - Run the pull command
$ cargo run --manifest-path josh-sync/Cargo.toml rustc-pull - Push the branch to your fork and create a PR into
rustc-dev-guide
Push changes from this repository into rust-lang/rust
- Run the push command to create a branch named
<branch-name>in arustcfork under the<gh-username>account$ cargo run --manifest-path josh-sync/Cargo.toml rustc-push <branch-name> <gh-username> - Create a PR from
<branch-name>intorust-lang/rust
Minimal git config
For simplicity (ease of implementation purposes), the josh-sync script simply calls out to system git. This means that the git invocation may be influenced by global (or local) git configuration.
You may observe "Nothing to pull" even if you know rustc-pull has something to pull if your global git config sets fetch.prunetags = true (and possibly other configurations may cause unexpected outcomes).
To minimize the likelihood of this happening, you may wish to keep a separate minimal git config that only has [user] entries from global git config, then repoint system git to use the minimal git config instead. E.g.
$ GIT_CONFIG_GLOBAL=/path/to/minimal/gitconfig GIT_CONFIG_SYSTEM='' cargo +stable run --manifest-path josh-sync/Cargo.toml -- rustc-pull