Speed up initial clone with --filter=blob:none

[fsackur]

On my box, cloning the tools repo and running make update takes 45 minutes. (My dev box is terrible and its on a terrible LAN.)

By adding --filter=blob:none, I've reduced that to 22 minutes.

Yes, it's twice as fast.

The tradeoff is that some git operations will take longer, as blobs will be downloaded on demand (e.g. doing git bisect or checking out earlier commits).

However, I expect this cost will never be incurred for the majority of workflows.

For workflows that dive into git history, I expect that the cost of fetching blobs on-demand will still be much less than the cost of fetching everything at clone time.

My understanding of the build system is very superficial, so I am open to having my assumptions corrected.

Ref: https://github.blog/open-source/git/get-up-to-speed-with-partial-clone-and-shallow-clone/

To implement, I think we just need to add the --filter=blob:none arg to the git_clone function in build/common.sh. I attach a PR to demonstrate: #468

[fichtner]

Just a question do you intend to build all from scratch? Partial updates are also an option with prefetching sets for kernel, base and packages even. If you even need an image...

[fsackur]

Just a question do you intend to build all from scratch?

No, I'm all about writing a plugin. The reason I'm running make update is because it's on the way to make plugins PLUGINSLIST="devel/my_plugin".

I've already captured the repos into a VM template, so I personally don't need to clone again.

[fichtner]

Ok I’m asking because you can use an OPNsense VM to build plugins just by cloning core and plugins into the VM.