Local Caching In CI

[bestickley]

Summary

Is anyone using turbo's local cache in CI by having CI reuse it's filesystem across builds? In other words, the output of your CI would become the input of your next CI's run.

Additional information

No response

Example

No response

[anthonyshew]

This sounds like a description of Remote Caching but on Hard Mode™️. Can you say why you don't want to use Remote Caching for this?

[bestickley]

@anthonyshew, thanks for the response! Local caching in CI just seemed simpler to get started. Nothing against Remote Caching.

[anthonyshew]

I see. Well, if your CI machines copy disk between each other, then the .turbo directory would come along with it and you'd have effectively what you're describing. That'd almost be a very limited imitation implementation of Remote Caching, so you could get at least something out of it.

[bestickley]

That'd almost be a very limited imitation implementation of Remote Caching

What would this be missing from Remote Caching?

[anthonyshew]

• No shared cache between CI machines and developer machines. For example, I sometimes check that my lints/tests/etc. are going to hit cache before I push my code for CI, and, when I do that, I hit cache in CI. The reverse direction is true as well.
• You won't be able to easily share cache across CI providers if you use more than one.
• The disk location of local cache is not part of our public API, so we reserve the right to move it or change it in some way that could break your copying scheme.

There's more that's mostly internal (also not part of our API, can break any time) that, if you do cover all the gaps, you've re-invented Remote Caching with a ton more complexity for no real benefit.

You're welcome to re-implement it, of course, but there are plenty of nuances, gotchas, and difficulties that we've already handled. We are, after all, just uploading and downloading files. But, as with many things in programming, even things that look just so simple on the surface can contain many complexities.