Caching your build results for faster pipelines

  • When you build locally, your compiler will just rebuild the parts you changed since you last hit compile - this should mean even huge projects build pretty quickly if you’re making small incremental changes. However, on a CI, you’ll find that it builds everything every time. This is fine for tiny projects, but as soon as they start getting more complex, and you start adding lots of platforms your CI build times can rocket.

  • The solution to this is to cache the results of your last build, so the next build doesn’t have to start from fresh each time - this is easily achieved setting the cache fields in your .gitlab-ci.yml

    cache:
key: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
untracked: true

  • The key field talls it to keep a seperate cache for each seperate build task (so, your x86 Linux build won’t share its cache with the x64 build) and also per git branch being built (so if multiple branches are being worked in, it won’t rebuild everything each time it builds a different branch)

  • Caching in most build systems relies on the timestamps of your files - if this.cpp is newer, then recompile

  • Gitlab will pull in a fresh copy of your repository each time a runner runs, and git by default will just give all the files the current time as their timestamp - this means all source files will be newer than the cache (even if they’ve not actually changed since the last build) - this will trigger a complete recompile of everything, rather than just your new changes.

  • The solution is to change the file timestamps to match the commit times

include