improve CI cache docs
This commit is contained in:
parent
03a1548373
commit
8a6fd47a85
|
|
@ -322,7 +322,7 @@ Our CI workflow uses various caching mechanisms, mainly for two things:
|
|||
### Docker images caching
|
||||
|
||||
The Docker images we use to run most of the Linux-based builders take a *long*
|
||||
time to fully build. To speed up the build, we cache it using [Docker registry
|
||||
time to fully build. To speed up the build, we cache them using [Docker registry
|
||||
caching], with the intermediate artifacts being stored on [ghcr.io]. We also
|
||||
push the built Docker images to ghcr, so that they can be reused by other tools
|
||||
(rustup) or by developers running the Docker build locally (to speed up their
|
||||
|
|
@ -334,6 +334,13 @@ override the cache for the others. Instead, we store the images under different
|
|||
tags, identifying them with a custom hash made from the contents of all the
|
||||
Dockerfiles and related scripts.
|
||||
|
||||
The CI calculates a hash key, so that the cache of a Docker image is
|
||||
invalidated if one of the following changes:
|
||||
|
||||
- Dockerfile
|
||||
- Files copied into the Docker image in the Dockerfile
|
||||
- The architecture of the GitHub runner (x86 or ARM)
|
||||
|
||||
[ghcr.io]: https://github.com/rust-lang-ci/rust/pkgs/container/rust-ci
|
||||
[Docker registry caching]: https://docs.docker.com/build/cache/backends/registry/
|
||||
|
||||
|
|
@ -341,9 +348,18 @@ Dockerfiles and related scripts.
|
|||
|
||||
We build some C/C++ stuff in various CI jobs, and we rely on [sccache] to cache
|
||||
the intermediate LLVM artifacts. Sccache is a distributed ccache developed by
|
||||
Mozilla, which can use an object storage bucket as the storage backend. In our
|
||||
case, the artefacts are uploaded to an S3 bucket that we control
|
||||
(`rust-lang-ci-sccache2`).
|
||||
Mozilla, which can use an object storage bucket as the storage backend.
|
||||
|
||||
With sccache there's no need to calculate the hash key ourselves. Sccache
|
||||
invalidates the cache automatically when it detects changes to relevant inputs,
|
||||
such as the source code, the version of the compiler, and important environment
|
||||
variables.
|
||||
So we just pass the sccache wrapper on top of cargo and sccache does the rest.
|
||||
|
||||
We store the persistent artifacts on the S3 bucket `rust-lang-ci-sccache2`. So
|
||||
when the CI runs, if sccache sees that LLVM is being compiled with the same C/C++
|
||||
compiler and the LLVM source code is the same, sccache retrieves the individual
|
||||
compiled translation units from S3.
|
||||
|
||||
[sccache]: https://github.com/mozilla/sccache
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue