Can't enable prebuild using file URL

[miyagawa]

I'm experimenting some combinations of using Carmel and cpm in development and production environments.

The goal is to manage the dependencies and snapshot with Carmel, then package the vendor/cache directory to install them with cpm on CI. To replicate what I would get on the CI, I'd like to install the modules the same way using cpm in development as well:

> cpm install -L /path/to/local --resolver 02packages,file://$PWD/vendor/cache --prebuilt

this is not as fast as I want with 200+ deps, because apparently prebuilt is not enabled when the mirror URL is file://. I needed to patch Installer.pm to allow the file:/// URL to force enable the build cache.

Can there be an option to enable this per URL basis, or rather, trust it if the URL is given as a CLI argument?

[skaji]

Currently I restrict cpm from using prebuilds if the mirror is not CPAN

cpm/lib/App/cpm/Worker/Installer.pm

Line 29 in f437699

!!( $uri =~ m{^https?://(?:www.cpan.org|backpan.perl.org|cpan.metacpan.org)} );

I think we should keep this by default,
but yeah, we need an option to tell cpm "this mirror is CPAN".

One thing I'm worried about is that
cpanfile allows users to specify dist/url for modules, and in that case, vender/cache will contain modules not only from CPAN but also from some custom mirrors.
This might be not a real problem right now,
but in the future, we need to determine how to cache modules from custom mirrors, I think.

[miyagawa]

Yep, that's the line I patched and returned 1 if the given URL starts with a file://.

i think ultimately the build cache needs to record where the tarball came from, whether it was CPAN or not. And might be some metadata of integrity, like SHA sum of the tarball. Then if the 02packages contained the sha sum of the tarball (needs an upgrade of the format) you're sure that the build cache was generated using the same archive.

Building and bundling vendor/cache to installs modules solely from the vendor cache is a common technique I recommend in Carmel and Carton, and would be great to be able to speed this up with build caches (i.e. with GitHub actions).

[skaji]

i think ultimately the build cache needs to record where the tarball came from, whether it was CPAN or not. And might be some metadata of integrity, like SHA sum of the tarball. Then if the 02packages contained the sha sum of the tarball (needs an upgrade of the format) you're sure that the build cache was generated using the same archive.

Completely agreed!