GithubHelp home page GithubHelp logo

swatinem / rust-cache Goto Github PK

View Code? Open in Web Editor NEW
1.2K 6.0 99.0 3.47 MB

A GitHub Action that implements smart caching for rust/cargo projects

License: GNU Lesser General Public License v3.0

TypeScript 97.60% Rust 2.40%

rust-cache's Introduction

Rust Cache Action

A GitHub Action that implements smart caching for rust/cargo projects with sensible defaults.

Example usage

- uses: actions/checkout@v4

# selecting a toolchain either by action or manual `rustup` calls should happen
# before the plugin, as the cache uses the current rustc version as its cache key
- run: rustup toolchain install stable --profile minimal

- uses: Swatinem/rust-cache@v2
  with:
    # The prefix cache key, this can be changed to start a new cache manually.
    # default: "v0-rust"
    prefix-key: ""

    # A cache key that is used instead of the automatic `job`-based key,
    # and is stable over multiple jobs.
    # default: empty
    shared-key: ""

    # An additional cache key that is added alongside the automatic `job`-based
    # cache key and can be used to further differentiate jobs.
    # default: empty
    key: ""

    # A whitespace separated list of env-var *prefixes* who's value contributes
    # to the environment cache key.
    # The env-vars are matched by *prefix*, so the default `RUST` var will
    # match all of `RUSTC`, `RUSTUP_*`, `RUSTFLAGS`, `RUSTDOC_*`, etc.
    # default: "CARGO CC CFLAGS CXX CMAKE RUST"
    env-vars: ""

    # The cargo workspaces and target directory configuration.
    # These entries are separated by newlines and have the form
    # `$workspace -> $target`. The `$target` part is treated as a directory
    # relative to the `$workspace` and defaults to "target" if not explicitly given.
    # default: ". -> target"
    workspaces: ""

    # Additional non workspace directories to be cached, separated by newlines.
    cache-directories: ""

    # Determines whether workspace `target` directories are cached.
    # If `false`, only the cargo registry will be cached.
    # default: "true"
    cache-targets: ""

    # Determines if the cache should be saved even when the workflow has failed.
    # default: "false"
    cache-on-failure: ""

    # Determines which crates are cached.
    # If `true` all crates will be cached, otherwise only dependent crates will be cached.
    # Useful if additional crates are used for CI tooling.
    # default: "false"
    cache-all-crates: ""

    # Determiners whether the cache should be saved.
    # If `false`, the cache is only restored.
    # Useful for jobs where the matrix is additive e.g. additional Cargo features,
    # or when only runs from `master` should be saved to the cache.
    # default: "true"
    save-if: ""
    # To only cache runs from `master`:
    save-if: ${{ github.ref == 'refs/heads/master' }}

    # Specifies what to use as the backend providing cache
    # Can be set to either "github" or "buildjet"
    # default: "github"
    cache-provider: ""

Further examples are available in the .github/workflows directory.

Outputs

cache-hit

This is a boolean flag that will be set to true when there was an exact cache hit.

Cache Effectiveness

This action only caches the dependencies of a crate, so is more effective if the dependency / own code ratio is higher.

It is also most effective for repositories with a Cargo.lock file. Library repositories with only a Cargo.toml file have limited benefits, as cargo will always use the most up-to-date dependency versions, which may not be cached.

Usage with Stable Rust is most effective, as a cache is tied to the Rust version. Using it with Nightly Rust is less effective as it will throw away the cache every day, unless a specific nightly build is being pinned.

Cache Details

This action currently caches the following files/directories:

  • ~/.cargo (installed binaries, the cargo registry, cache, and git dependencies)
  • ./target (build artifacts of dependencies)

This cache is automatically keyed by:

  • the github job_id,
  • the rustc release / host / hash,
  • the value of some compiler-specific environment variables (eg. RUSTFLAGS, etc), and
  • a hash of all Cargo.lock / Cargo.toml files found anywhere in the repository (if present).
  • a hash of all rust-toolchain / rust-toolchain.toml files in the root of the repository (if present).
  • a hash of all .cargo/config.toml files in the root of the repository (if present).

An additional input key can be provided if the builtin keys are not sufficient.

Before being persisted, the cache is cleaned of:

  • Any files in ~/.cargo/bin that were present before the action ran (for example rustc).
  • Dependencies that are no longer used.
  • Anything that is not a dependency.
  • Incremental build artifacts.
  • Any build artifacts with an mtime older than one week.

In particular, the workspace crates themselves are not cached since doing so is generally not effective. For this reason, this action automatically sets CARGO_INCREMENTAL=0 to disable incremental compilation, so that the Rust compiler doesn't waste time creating the additional artifacts required for incremental builds.

The ~/.cargo/registry/src directory is not cached since it is quicker for Cargo to recreate it from the compressed crate archives in ~/.cargo/registry/cache.

The action will try to restore from a previous Cargo.lock version as well, so lockfile updates should only re-build changed dependencies.

The action invokes cargo metadata to determine the current set of dependencies.

Additionally, the action automatically works around cargo#8603 / actions/cache#403 which would otherwise corrupt the cache on macOS builds.

Cache Limits and Control

This specialized cache action is built on top of the upstream cache action maintained by GitHub. The same restrictions and limits apply, which are documented here: Caching dependencies to speed up workflows

In particular, caches are currently limited to 10 GB in total and exceeding that limit will cause eviction of older caches.

Caches from base branches are available to PRs, but not across unrelated branches.

The caches can be controlled using the Cache API which allows listing existing caches and manually removing entries.

Debugging

The action prints detailed information about which information it considers for its cache key, and it outputs more debug-only information about which cleanup steps it performs before persisting the cache.

You can read up on how to enable debug logging to see those details as well as further details related to caching operations.

Known issues

  • The cache cleaning process currently removes all the files from ~/.cargo/bin that were present before the action ran (for example rustc).

rust-cache's People

Contributors

alerque avatar austinjones avatar carsten-wenderdel avatar dnaka91 avatar edmorley avatar friegger avatar hamirmahal avatar jongwooo avatar joroshiba avatar kamilogorek avatar lucasfernog avatar max-heller avatar mkatychev avatar neysofu avatar nicholastmosher avatar nobodyxu avatar palfrey avatar psibi avatar rhysd avatar rukai avatar rxminus avatar sergnikitin avatar stevenh avatar swatinem avatar turbo87 avatar vlad-shcherbina avatar wyatt-herkamp avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

rust-cache's Issues

Can (Should?) this cache be leveraged by a docker build?

I know this may be out of scope, but I am looking to build my docker images without downloading all the dependencies every time. It is pretty expensive and slows down the process considerably, and given rust uses a lock file it feels unnecessary. The use case is specifically when copying the source to and building from the Dockerfile.

Is this supported by this tool?

yes - how can I make that work?
no - can it be supported?

Thank you

If `$CARGO_HOME/.crates2.json` is not found, cache restore does not appear to occur

It appears that getCargoBins throws an unhandled exception when .crates2.json is not present, which appears to result in failure to restore the cache.

This occurs for me in a build on a self-hosted runner where the first execution of cargo only occurs after the cache would have been restored.

I see this type of message in my action logs:

Warning:  ENOENT: no such file or directory, open '/home/gha/gh-runner/work/project_name/project_name/.cargo/.crates2.json'

cargo can't find crate dependencies when using actions-rs/toolchain and non-`stable` release channel

I tried adding rust-cache to @rustsec and I'm encountering the following build failures when an explicit version (i.e. MSRV) is specified rather than stable:

https://github.com/rustsec/rustsec/runs/6259271728?check_suite_focus=true

Run cargo test --release
    Updating crates.io index
error: failed to select a version for the requirement `libgit2-sys = "^0.13.3"`
candidate versions found which didn't match: 0.13.2+1.4.2, 0.13.1+1.4.2, 0.13.0+1.4.1, ...
location searched: crates.io index
required by package `git2 v0.14.3`
    ... which satisfies dependency `git2 = "^0.14"` (locked to 0.14.3) of package `rustsec v0.25.2 (/Users/runner/work/rustsec/rustsec/rustsec)`
    ... which satisfies path dependency `rustsec` (locked to 0.25.2) of package `rustsec-admin v0.6.1 (/Users/runner/work/rustsec/rustsec/admin)`
Error: Process completed with exit code [10](https://github.com/rustsec/rustsec/runs/6259271728?check_suite_focus=true#step:5:10)1.

Curiously everything is fine on stable.

We have override: true set. I'm curious if that's implicated?

Caching throws an error on windows GitHub Actions runners

Hi! Thanks for this GitHub Action. I'd like to use it in a build matrix that tests on Windows, macOS, and Linux. Things work great on macOS and Linux, but Windows fails to create a cache artifact.

GNU tar throws the following error on Windows:

Post job cleanup.
Saving paths:
    C:\Users\runneradmin\.cargo\registry\index
    C:\Users\runneradmin\.cargo\registry\cache
    C:\Users\runneradmin\.cargo\git
    target
Using key "v0-rust-v2-build-1.49.0-x86_64-pc-windows-msvc-e1884a8e3c3e-bb9536f437cb47ed7a45".
C:\msys64\usr\bin\tar.exe --posix --use-compress-program "zstd -T0" -cf cache.tzst -P -C D:/a/artichoke/artichoke --files-from manifest.txt --force-local
/usr/bin/tar: C\:\\Users\runneradmin\\.cargo\registry\\index: Cannot stat: No such file or directory
/usr/bin/tar: C\:\\Users\runneradmin\\.cargo\registry\\cache: Cannot stat: No such file or directory
/usr/bin/tar: Exiting with failure status due to previous errors
Warning:  Tar failed with error: The process 'C:\msys64\usr\bin\tar.exe' failed with exit code 2

Example run: https://github.com/artichoke/artichoke/runs/1666497488?check_suite_focus=true

tar permission issue macos

Caching is failing on macos.

Issue seems to be that target folder has the wrong permissions, and therefore tar fails.

Possibly related issues from github cache action:

I suspect this may get fixed by updating your github cache action dependency (based on some comments in the linked issues)? I'm a novice at JS though so I may have misinterpreted the lock file.

Logs:

Post job cleanup.
Saving paths:
    /Users/runner/.cargo/bin
    /Users/runner/.cargo/.crates2.json
    /Users/runner/.cargo/.crates.toml
    /Users/runner/.cargo/git
    /Users/runner/.cargo/registry/cache
    /Users/runner/.cargo/registry/index
    ./target
In directory:
    /Users/runner/work/rust-ipfs/rust-ipfs
Using key:
    v0-rust-ci-matrix-1.54.0-x86_64-apple-darwin-a178d0322ce2-5bae624ed82b92875943
/usr/local/bin/gtar --posix --use-compress-program zstd -T0 -cf cache.tzst -P -C /Users/runner/work/rust-ipfs/rust-ipfs --files-from manifest.txt --delay-directory-restore
/usr/local/bin/gtar: target/debug/incremental/build_script_build-23o2sgzkctxm0/s-g1dofmvbnr-gnj1zd.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/ipfs_bitswap-34znwa6y256w6/s-g1dofphzx0-7ibob8.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/cat-11nw5twjd4a3l/s-g1dofrydxo-1ja79uq.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/pubsub-2mpm11mg579k9/s-g1dogk1fi3-104csm.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/resolve-3ouqffba8syei/s-g1dofqy9dx-zfjap5.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/add-3c5ar87ixwsxt/s-g1dofqk3h9-scmnvy.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/ipfs_http-3v72h6wvf7zpx/s-g1dohhu9l0-107r6vw.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/common_tests-wgaobe2mlb9n/s-g1dojedh0p-1mzvcfv.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/build_script_build-bql0v9ysv9rd/s-g1dofkmrob-a3eafz.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/kademlia-1rxrt1sgjuzr1/s-g1dogiw1a1-7sux10.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/build_script_build-k6i17hfcgm3o/s-g1dofkmrn8-10l8krb.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/bitswap-39x5p1b7mgdmn/s-g1dohq5v1t-1tf7var.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/ipfs_unixfs-o092r12zmxyl/s-g1dofkn3ep-7kgwpx.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/connectivity-1wl1urmk9lwdz/s-g1dohj2rh7-z8e7nq.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/ipfs_http-1318rjcvbh66f/s-g1doh4pq1v-ilefgt.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/wantlist_and_cancellation-g2yimljyp3vu/s-g1doh5gjx1-rcbe08.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/block_exchange-3fkjfznzjre7b/s-g1doicyxmj-lmo5a5.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/fetch_and_cat-3layqel49eux0/s-g1doi0geeo-1l37f6o.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/ipfs_http-1y4zv2xqxf4av/s-g1dog1kguc-8bwh8b.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/get-2hpgt42s0g17w/s-g1dofr33ys-dgq3wf.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/ipfs-1z0i5zq4lrong/s-g1dofryr39-hghtpk.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/ipfs_bitswap-2xqw89znd31ai/s-g1dofolej9-r1t69s.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/ipfs_unixfs-32i2aaiiazbej/s-g1dofmvhpi-vzhcj1.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/listening_addresses-29alxe0k3q4uw/s-g1doiq889w-19q3ji1.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/ipfs-cjlsa7c4ba54/s-g1dofsp5yn-muzxbm.lock: Cannot open: Permission denied
/usr/local/bin/gtar: target/debug/incremental/dag_creation-1u0h8fkxb2wx6/s-g1doj1egr3-91uu7v.lock: Cannot open: Permission denied
/usr/local/bin/gtar: Exiting with failure status due to previous errors
Warning:  Tar failed with error: The process '/usr/local/bin/gtar' failed with exit code 2

Additionally cache ~/.rustup folder

Doesn't the actions-rs/toolchain action end up installing a toolchain in ~/.rustup and not caching it? Shouldn't / couldn't that be cached by default here as well?

cache cleanup failure

I'm getting post-job cleanup failure that causes the cache operation to exit and nothing is cached. But it is unclear what caused the failure.

Post job cleanup.
Warning:  The process '/home/runner/.cargo/bin/cargo' failed with exit code 101

I'm invoking with

 44       - name: Cache Nydus
 45         uses: Swatinem/rust-cache@v1
 46         with:
 47           target-dir: ./target-fusedev
 48           cache-on-failure: true

So looks like cache-on-failure is not working, which is another issue though.

Question: Should this action reuse cache from master?

I was rolling up my own cache solution before and I started using this action because it looks cleaner and honestly better than my previous setup. That said, although it works perfectly for cache within a PR (first one takes a while, following builds go quick), I think the first build of every PR is always cache-miss. Is this supposed to be the behavior?

I'm not implying anything, I'm just asking to understand better the expected behavior so I can adapt accordingly.
Thanks a lot!

No cache found.

Hi, @Swatinem.

My rust project locates in root/path_a/path_b, and I set working-directory: path_a/path_b. But the action always returns No cache found. Could help to guide me on how to set the right arguments?

Cache of different branches

I want to know if a cache created in one branch workflow can be used in another branch workflow provided lock and dependencies don't change.

cargo test SIGILL - not sure if this is related

I am using rust-cache in my GitHub Actions with default options, like this:

      - name: Checkout
         uses: actions/checkout@v2

      - uses: Swatinem/[email protected]

      - name: Install latest stable
        uses: actions-rs/toolchain@v1
        with:
          profile: minimal
          toolchain: stable
          override: true
          components: rustfmt, clippy

      # Cargo actions.
      - name: Run rustfmt and fail if any warnings
        run: cargo fmt -- --check
      - name: Build
        run: cargo build --color=always
      - name: Run clippy and fail if any warnings
        run: cargo clippy -- -D warnings
      - name: Run tests
        run: cargo test

Sometimes, cargo build or cargo test would crash due to an error like this:

Caused by:
  process didn't exit successfully: `<my_path>/target/debug/deps/integration_tests-a0b2fd9edc2b1e73` (signal: 4, SIGILL: illegal instruction)

If I remove the use of rust-cache and re-run Actions, it would succeed. This happened with or without Cargo.toml changes.

Is there any known issue that could cause SIGILL during cargo build or cargo test?

`cargo` and `rustup` get deleted when saving cache

Since we switched to running on self-hosted runners, we noticed that some jobs failed because they couldn't find the cargo executable anymore. After some digging, we noticed that after a successful run with this action on the same machine, .cargo/bin was empty, which is normally where cargo and rustup are installed (when installing the rustup.rs way at least).

Looking through the code, it seems that .cargo/bin gets cleaned apart for the binaries it finds a matching crate for, which would end up removing cargo and rustup. Was this intentional? If so, should I be doing something differently?

Use separate cache for registry

As the crates.io registry doesn't depend on anything and does not vary between versions, would it make sense to store the registry separately? This would avoid duplicating the registry across N caches.

Mode for just saving source and registry

For those of us on nightly, it would be nice to have an option so that the action does not key the cache with the job or the toolchain, and saves only the downloaded source and registry. This would be nice for avoiding extra source downloads specifically.

new minor release

I would like to use the latest change to provide custom target dir. Could we get another minor release pls ?

Do not append `process.env.GITHUB_JOB` to the cache key if `key` is set

If the key property is set, don't append process.env.GITHUB_JOB to the cache key. This allows you to share a cache between jobs in a workflow.

In my config, I have a job that prepares dependencies, and others that consume the cache key. Adding job to the key prevents this from working.

This could be enabled by another flag (e.g. sharedKey) if you are concerned about backwards compatibility.

Feature request: support 'with:' parameter for working directory

Rationale: it would be possible to use this on repositories where a Rust module is in subdir, not in root.

Example: this workflow builds a Rust library which is stored in subdirectory.

The "Post Run" step of the action is failing with a warning (example):
Warning: The process '/usr/share/rust/.cargo/bin/cargo' failed with exit code 101

cache installed tools

Eg. cargo install cargo-hack should not install fresh each time.

This can be done transparently just by caching $HOME/.cargo/bin (minus a few items probably) since Cargo does the necessary checks to prevent duplicate installs of the most recent version.

This would reduce CI times from installing cargo-hack and cargo-tarpaulin on actix-web by 20%.

Open Source License

I'm not seeing a license for this code mentioned anywhere (e.g., in a LICENSE file). Is this code licensed under one of the common open source licenses (e.g., the popular Apache-2/MIT combo most Rust projects seem to use)? If not, would you consider doing so?

The reason I ask is that I'd like to try and replicate your work here in a form I can use in GitLab CI, but I, of course, don't want to violate your copyright and I'd like to give you proper attribution.

Thanks for this lovely project!!

Include github actions `runs-on` field in key.

I just hit a confusing compilation issue when downgrading from runs-on: ubuntu-20.04 to runs-on: ubuntu-18.04 caused because the cache didnt clear.
I'm not sure if you have access to this field, maybe something similar could be done by looking at something like lsb_release -a

Is cache miss on any Cargo.lock change ideal?

Thank you for writing this!

I am curious about whether or not it's necessary to include the hash of Cargo.lock files in the cache key. If some dependencies have changed but others haven't, the hash of Cargo.lock files will change, which will cause a cache miss. However, since some dependencies haven't changed, it might better to populate the cache, and let cargo ignore things that are out of date.

Add ability to specify multiple `target-dir`s

Hi @Swatinem thanks for rust-cache, it's very useful! I was wondering whether it's possible to specify more than one target directory to be cached. I have a project with a top-level workspace that shares one target directory, and another independent, nested workspace (used because we build for WASM targets by default there), and the nested workspace has a separate target/ directory that I don't believe gets cached by this action.

So if our folder structure looked like this:

project/
  Cargo.toml
  src/
  target/
  wasm-project/
    Cargo.toml
    src/
    target/

Then I believe only project/target/ is getting cached, and I'm wondering if it'd be possible somehow to add multiple entries to target-dir so that project/wasm-project/target/ would also be cached.

Include env vars into the cache key

Hey there, thanks for this great tool! Just found out about it in matklad's article and it works really well for me.

I was just wondering whether it makes sense to include environment variables when creating the cache key?

For example I added the cache first and then later added the following to disable incremental builds and debug info but the cache was still considered up-to-date.

env:
  CARGO_INCREMENTAL: 0
  RUSTFLAGS: "-C debuginfo=0 -D warnings"

I saw that you already expose CARGO_INCREMENTAL=0 as part of restoring the cache. Still, as other env vars can influence the generated files during compilation it might make sense to include either all or some Rust/cargo specific vars that are known to affect compilation.

Caching errors

Since yesterday we started seeing those errors:

Warning:  Cache service responded with 400 during upload chunk.
Warning:  Cache upload failed because file read failed with EBADF: bad file descriptor, read
Error: Cache upload failed because file read failed with EBADF: bad file descriptor, read
    at ReadStream.<anonymous> (/home/runner/work/_actions/Swatinem/rust-cache/v1/dist/save/index.js:359:[31](https://github.com/caido/proxy-backend/runs/5080976100?check_suite_focus=true#step:15:31))
    at ReadStream.emit (events.js:210:5)
    at internal/fs/streams.js:167:12
    at FSReqCallback.wrapper [as oncomplete] (fs.js:470:5)
Warning:  Cache upload failed because file read failed with ESPIPE: invalid seek, read
Error: Cache upload failed because file read failed with ESPIPE: invalid seek, read
    at ReadStream.<anonymous> (/home/runner/work/_actions/Swatinem/rust-cache/v1/dist/save/index.js:359:31)
    at ReadStream.emit (events.js:210:5)
    at internal/fs/streams.js:167:12
    at FSReqCallback.wrapper [as oncomplete] (fs.js:470:5)

The upload takes 5-6m but then the cache is never really reused.

can it work with `cross`?

I wonder whether can this action work with cross. Can it properly cache cross build cache? Maybe it's worth mentioning in the README.md

Strip version(s) of developed crate(s) from `Cargo.toml` before caching

Cargo.toml contains the entry of the crate (crates in case of a workspace) it was generated for. For example, consider a crate foo with the following Cargo.toml:

[package]
name = "foo"
version = "0.1"

[dependencies]
...

It will have something like this in its Cargo.lock along with its dependencies:

[[package]]
name = "foo"
version = "0.1"
dependencies = [ ... ]

When you change a version of this crate this would result into a change of this entry. This will change the Cargo.lock hash although no dependencies were changed. Then an unnecessary cache miss will happen.

Caching the entire ~/.cargo/git is excessive

The subdirectory ~/.cargo/git/checkouts caches working copies of git dependencies that are entirely recoverable offline from a fresh ~/.cargo/git/db. Removing the first subdirectory from the cache can save quite a lot of cache traffic:

$ du -sh ~/.cargo/git/{checkouts,db}
170M	/home/mzabaluev/.cargo/git/checkouts
1,2M	/home/mzabaluev/.cargo/git/db

Lockfile hash trimmed to half its length

While working on #35, I noticed that the lockfile hash is using SHA-1 as algorithm. In the end, it is turned into a hex string and then trimmed to 20 characters.

As SHA-1 uses 20 bytes as output, it is doubled when turned into a hex string, resulting in a total of 40 characters.

I was just wondering what's the reasoning behind trimming the hash to half its actual length, as it can increase the chance for clashes (duplicate hash keys for different content).

Maybe the cache key that GitHub accepts has an upper length limit, or maybe this was done by mistake?

Cache own code with integrated cargo-sweep-like solution

cargo-sweep is a tool to be used around any build actions to timestamp and remove unvisited cache. It does so by checking the atime (access time) of the files in the target directory.

cargo sweep -s

<Insert any number of cargo build, cargo test etc...>

cargo sweep -f

With this method, incremental build can be left on, and build of own code can be cached for greater speed.
Unfortunately, cargo-sweep doesn't publish any binary builds nor github actions.

#37 mentions the benefits of not caching one's own code:

  1. it would only cache the first invocation when it generates a new cache key.
  2. with time the code will diverge more, causing longer builds.
  3. incremental needs a lot of disk space, and thus cache space and time to fetch & extract, lowering the benefits.
  1. is no longer an issue with the kind of smart diffing of cargo-sweep.

Support for excluding paths from the cache

Is it possible to explicitly exclude one of the default-cached paths?

We just hit an issue using this plugin where CI failed to detect that a build was broken in the following case:

  1. Patch was made to a dependency
  2. During PR process, project was pointed to the patch, CI ran, patch was cached
  3. Patch was merged into dependency, branch deleted
  4. Local project continued building in CI just fine because the cache found the patch, even though the referenced revision no longer existed after the PR merge

This can cause CI to report false-positive "PASS" on builds. I know this is a somewhat special case, so I was curious if it would be possible to explicitly exclude git dependencies from caching, even if it forces a longer build time.

Overwrite the Cache Key

Just imagine that I have a project with a lot of dependencies and a Cargo.lock which changes every couple of days.

Does it make sense that we are able to overwrite the generated key with completely something else, other than Cargo.lock?

For example, for me, it completely makes sense to cache by rust version.

PS: I might be completely wrong, I don't have so much knowledge in this.

cross targetting support

I wrote workflow setup like:

      - uses: actions-rs/toolchain@v1
        with:
          target: i686-unknown-linux-gnu

      - uses: Swatinem/rust-cache@v1

But it seems it doesn't work for cross target.

When looking for cache:

    v0-rust-exotic_targets-1.55.0-x86_64-unknown-linux-gnu-c8dfcfe046a7
No cache found.

When trying cache:

Warning:  Unable to locate executable file: rustc. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.

Add support for the `cache-hit` output

The actions/cache@v2 plugin supports an output variable called cache-hit, which is set to true if there is an exact match. This is really useful for conditional steps later in the job, such as a cargo build call:

      - name: cargo build | dependencies
        uses: actions-rs/cargo@v1
        if: steps.cargo-cache.outputs.cache-hit != 'true'
        with:
          command: build

This would be a great output variable for rust-cache, and would make it a drop-in replacement for actions/cache@v2.

Warning if there is only a release build

When I do a job with only a release build and no debug build, I see the following warning in the log

Warning: ENOENT: no such file or directory, opendir 'target/debug'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.