From 23d77abfc0e24f9f4e4d520d35830900fb2a9701 Mon Sep 17 00:00:00 2001 From: xizheyin Date: Mon, 16 Jun 2025 23:38:15 +0800 Subject: [PATCH] Add Section How queries interact with external crate metadata Signed-off-by: xizheyin --- src/query.md | 47 ++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 42 insertions(+), 5 deletions(-) diff --git a/src/query.md b/src/query.md index 9f7154f0..7a94b526 100644 --- a/src/query.md +++ b/src/query.md @@ -78,9 +78,6 @@ during compiler initialization. The macro system generates both structs, which act as function tables for all query implementations, where each field is a function pointer to the actual provider. -[providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.Providers.html -[extern_providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.ExternProviders.html - **Note:** Both the `Providers` and `ExternProviders` structs are generated by macros and act as function tables for all query implementations. They are **not** Rust traits, but plain structs with function pointer fields. @@ -174,11 +171,51 @@ Suppose you want to add a new query called `fubar`. You would: ```rust,ignore pub fn provide(providers: &mut rustc_middle::util::Providers) { providers.queries.fubar = fubar; - // If you need an external provider: - providers.extern_queries.fubar = extern_fubar; } ``` +### How queries interact with external crate metadata + +When a query is made for an external crate (i.e., a dependency), the query system needs to load the information from that crate's metadata. +This is handled by the [`rustc_metadata` crate][rustc_metadata], which is responsible for decoding and providing the information stored in the `.rmeta` files. + +The process works like this: + +1. When a query is made, the query system first checks if the `DefId` refers to a local or external crate by checking if `def_id.krate == LOCAL_CRATE`. + This determines whether to use the local provider from [`Providers`][providers_struct] or the external provider from [`ExternProviders`][extern_providers_struct]. + +2. For external crates, the query system will look for a provider in the [`ExternProviders`][extern_providers_struct] struct. + The `rustc_metadata` crate registers these external providers through the `provide_extern` function in `rustc_metadata/src/rmeta/decoder/cstore_impl.rs`. Just like: + ```rust + pub fn provide_extern(providers: &mut ExternProviders) { + providers.foo = |tcx, def_id| { + // Load and decode metadata for external crate + let cdata = CStore::from_tcx(tcx).get_crate_data(def_id.krate); + cdata.foo(def_id.index) + }; + // Register other external providers... + } + ``` + +3. The metadata is stored in a binary format in `.rmeta` files that contains pre-computed information about the external crate, such as types, function signatures, trait implementations, and other information needed by the compiler. When an external query is made, the `rustc_metadata` crate: + - Loads the `.rmeta` file for the external crate + - Decodes the metadata using the `Decodable` trait + - Returns the decoded information to the query system + +This approach avoids recompiling external crates, allows for faster compilation of dependent crates, and enables incremental compilation to work across crate boundaries. +Here is a simplified example, when you call `tcx.type_of(def_id)` for a type defined in an external crate, the query system will: +1. Detect that the `def_id` refers to an external crate by checking `def_id.krate != LOCAL_CRATE` +2. Call the appropriate provider from `ExternProviders` which was registered by `rustc_metadata` +3. The provider will load and decode the type information from the external crate's metadata +4. Return the decoded type to the caller + +This is why most `rustc_*` crates only need to provide local providers - the external providers are handled by the metadata system. +The only exception is when a crate needs to provide special handling for external queries, in which case it would implement both local and external providers. + +[rustc_metadata]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_metadata/index.html +[providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.Providers.html +[extern_providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.ExternProviders.html + --- ## Adding a new query