A guide to how rustc works and how to contribute to it.
Go to file
Jacob Pratt 506ee27985
Rollup merge of #133429 - EnzymeAD:autodiff-middle, r=oli-obk
Autodiff Upstreaming - rustc_codegen_ssa, rustc_middle

This PR should not be merged until the rustc_codegen_llvm part is merged.
I will also alter it a little based on what get's shaved off from the cg_llvm PR,
and address some of the feedback I received in the other PR (including cleanups).

I am putting it already up to
1) Discuss with `@jieyouxu` if there is more work needed to add tests to this and
2) Pray that there is someone reviewing who can tell me why some of my autodiff invocations get lost.

Re 1: My test require fat-lto. I also modify the compilation pipeline. So if there are any other llvm-ir tests in the same compilation unit then I will likely break them. Luckily there are two groups who currently have the same fat-lto requirement for their GPU code which I have for my autodiff code and both groups have some plans to enable support for thin-lto. Once either that work pans out, I'll copy it over for this feature. I will also work on not changing the optimization pipeline for functions not differentiated, but that will require some thoughts and engineering, so I think it would be good to be able to run the autodiff tests isolated from the rest for now. Can you guide me here please?
For context, here are some of my tests in the samples folder: https://github.com/EnzymeAD/rustbook

Re 2: This is a pretty serious issue, since it effectively prevents publishing libraries making use of autodiff: https://github.com/EnzymeAD/rust/issues/173. For some reason my dummy code persists till the end, so the code which calls autodiff, deletes the dummy, and inserts the code to compute the derivative never gets executed. To me it looks like the rustc_autodiff attribute just get's dropped, but I don't know WHY? Any help would be super appreciated, as rustc queries look a bit voodoo to me.

Tracking:

- https://github.com/rust-lang/rust/issues/124509

r? `@jieyouxu`
2025-01-31 00:26:30 -05:00
.github/workflows Fix rustc-pull CI's bash commands 2025-01-27 23:34:43 +01:00
ci Opt into, rather than out of, linkcheck (#2180) 2024-12-30 17:22:22 +08:00
examples Make crate AST mutation accessible for driver callback 2025-01-28 19:45:20 +00:00
josh-sync Revert "Add `@bors rollup=never` to rustc-push PR body" 2025-01-24 13:59:05 +01:00
src Auto merge of #136038 - compiler-errors:outlives, r=lcnr 2025-01-30 11:40:32 +00:00
.editorconfig Set max line length in `.editorconfig` to 100 (#1788) 2023-09-05 23:24:59 +09:00
.gitattributes .gitattributes: Mark minified javascript as binary to filter greps 2022-10-07 18:34:51 +02:00
.gitignore add josh-sync build dir to gitignore (#2196) 2025-01-06 02:57:03 +08:00
.mailmap add a mailmap 2023-12-17 18:21:38 +01:00
CITATION.cff Add a citation file 2023-02-11 08:41:56 +02:00
CNAME cname (#606) 2020-03-09 18:10:52 -03:00
CODE_OF_CONDUCT.md Fix some links (#1865) 2024-01-28 19:44:41 -03:00
LICENSE-APACHE add code-of-conduct, licensing material, and a README 2018-01-16 16:36:21 -05:00
LICENSE-MIT add code-of-conduct, licensing material, and a README 2018-01-16 16:36:21 -05:00
README.md docs: document how to install a suitable `josh-proxy` locally 2025-01-20 12:35:45 +08:00
book.toml Save linkcheck cache always 2025-01-08 17:05:11 +01:00
mermaid-init.js add mdbook-mermaid 2022-07-17 23:34:12 +02:00
mermaid.min.js add mdbook-mermaid 2022-07-17 23:34:12 +02:00
rust-version Preparing for merge from rustc 2025-01-28 11:57:04 +00:00
triagebot.toml Add CI workflow for performing rustc-pull 2025-01-11 17:50:07 +01:00

README.md

CI

This is a collaborative effort to build a guide that explains how rustc works. The aim of the guide is to help new contributors get oriented to rustc, as well as to help more experienced folks in figuring out some new part of the compiler that they haven't worked on before.

You can read the latest version of the guide here.

You may also find the rustdocs for the compiler itself useful. Note that these are not intended as a guide; it's recommended that you search for the docs you're looking for instead of reading them top to bottom.

For documentation on developing the standard library, see std-dev-guide.

Contributing to the guide

The guide is useful today, but it has a lot of work still to go.

If you'd like to help improve the guide, we'd love to have you! You can find plenty of issues on the issue tracker. Just post a comment on the issue you would like to work on to make sure that we don't accidentally duplicate work. If you think something is missing, please open an issue about it!

In general, if you don't know how the compiler works, that is not a problem! In that case, what we will do is to schedule a bit of time for you to talk with someone who does know the code, or who wants to pair with you and figure it out. Then you can work on writing up what you learned.

In general, when writing about a particular part of the compiler's code, we recommend that you link to the relevant parts of the rustc rustdocs.

Build Instructions

To build a local static HTML site, install mdbook with:

> cargo install mdbook mdbook-linkcheck2 mdbook-toc mdbook-mermaid

and execute the following command in the root of the repository:

> mdbook build --open

The build files are found in the book/html directory.

We use mdbook-linkcheck2 to validate URLs included in our documentation. Link checking is not run by default locally, though it is in CI. To enable it locally, set the environment variable ENABLE_LINKCHECK=1 like in the following example.

$ ENABLE_LINKCHECK=1 mdbook serve

Table of Contents

We use mdbook-toc to auto-generate TOCs for long sections. You can invoke the preprocessor by including the <!-- toc --> marker at the place where you want the TOC.

Synchronizing josh subtree with rustc

This repository is linked to rust-lang/rust as a josh subtree. You can use the following commands to synchronize the subtree in both directions.

You'll need to install josh-proxy locally via

cargo +stable install josh-proxy --git https://github.com/josh-project/josh --tag r24.10.04

Older versions of josh-proxy may not round trip commits losslessly so it is important to install this exact version.

Pull changes from rust-lang/rust into this repository

  1. Checkout a new branch that will be used to create a PR into rust-lang/rustc-dev-guide
  2. Run the pull command
    $ cargo run --manifest-path josh-sync/Cargo.toml rustc-pull
    
  3. Push the branch to your fork and create a PR into rustc-dev-guide

Push changes from this repository into rust-lang/rust

  1. Run the push command to create a branch named <branch-name> in a rustc fork under the <gh-username> account
    $ cargo run --manifest-path josh-sync/Cargo.toml rustc-push <branch-name> <gh-username>
    
  2. Create a PR from <branch-name> into rust-lang/rust