Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only use URI from mirror where we found the module and not from all m… #518

Open
wants to merge 1 commit into
base: devel
Choose a base branch
from

Conversation

perlancar
Copy link

…irrors.

When multiple --mirrors are used, cpan_dist() will construct URIs from all
mirrors, regardless or whether this mirror contains the module or not. This can
cause a lot of noise warnings of failing to fetch files from the mirrors,
particularly when darkpan(s) are involved. For example:

% cpanm --mirror http://darkpan1/ --mirror http://darkpan2/ \
    --mirror http://darkpan3/ --mirror http://cpan/ --mirror-only \
    A-CPAN-Module-1.23.tar.gz
! Finding A::CPAN::Module () on mirror http://darkpan1/ failed.
! Finding A::CPAN::Module () on mirror http://darkpan2/ failed.
! Finding A::CPAN::Module () on mirror http://darkpan3/ failed.
--> Working on A::CPAN::Module
Fetching http://darkpan1/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz ... FAIL
! Download http://darkpan1/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
! Download http://darkpan1/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
! Download http://darkpan1/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
Fetching http://darkpan2/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz ... FAIL
! Download http://darkpan2/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
! Download http://darkpan2/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
! Download http://darkpan2/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
Fetching http://darkpan3/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz ... FAIL
! Download http://darkpan3/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
! Download http://darkpan3/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
! Download http://darkpan3/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
Fetching http://cpan/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz ... OK
...

With this small commit, the distribution A-CPAN-Module will only get the
http://cpan/ URL as that is where the associated module is found. So, the noise
is much reduced:

% cpanm --mirror http://darkpan1/ --mirror http://darkpan2/ \
    --mirror http://darkpan3/ --mirror http://cpan/ --mirror-only \
    A-CPAN-Module-1.23.tar.gz
! Finding A::CPAN::Module () on mirror http://darkpan1/ failed.
! Finding A::CPAN::Module () on mirror http://darkpan2/ failed.
! Finding A::CPAN::Module () on mirror http://darkpan3/ failed.
--> Working on A::CPAN::Module
Fetching http://cpan/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz ... OK
...

…irrors.

When multiple --mirrors are used, cpan_dist() will construct URIs from all
mirrors, regardless or whether this mirror contains the module or not. This can
cause a lot of noise warnings of failing to fetch files from the mirrors,
particularly when darkpan(s) are involved. For example:

    % cpanm --mirror http://darkpan1/ --mirror http://darkpan2/ \
        --mirror http://darkpan3/ --mirror http://cpan/ --mirror-only \
        A-CPAN-Module-1.23.tar.gz
    ! Finding A::CPAN::Module () on mirror http://darkpan1/ failed.
    ! Finding A::CPAN::Module () on mirror http://darkpan2/ failed.
    ! Finding A::CPAN::Module () on mirror http://darkpan3/ failed.
    --> Working on A::CPAN::Module
    Fetching http://darkpan1/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz ... FAIL
    ! Download http://darkpan1/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
    ! Download http://darkpan1/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
    ! Download http://darkpan1/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
    Fetching http://darkpan2/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz ... FAIL
    ! Download http://darkpan2/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
    ! Download http://darkpan2/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
    ! Download http://darkpan2/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
    Fetching http://darkpan3/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz ... FAIL
    ! Download http://darkpan3/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
    ! Download http://darkpan3/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
    ! Download http://darkpan3/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz failed. Retrying ...
    Fetching http://cpan/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz ... OK
    ...

With this small commit, the distribution A-CPAN-Module will only get the
http://cpan/ URL as that is where the associated module is found. So, the noise
is much reduced:

    % cpanm --mirror http://darkpan1/ --mirror http://darkpan2/ \
        --mirror http://darkpan3/ --mirror http://cpan/ --mirror-only \
        A-CPAN-Module-1.23.tar.gz
    ! Finding A::CPAN::Module () on mirror http://darkpan1/ failed.
    ! Finding A::CPAN::Module () on mirror http://darkpan2/ failed.
    ! Finding A::CPAN::Module () on mirror http://darkpan3/ failed.
    --> Working on A::CPAN::Module
    Fetching http://cpan/authors/id/P/PA/PAUSEID/A-CPAN-Module-1.23.tar.gz ... OK
    ...
@miyagawa
Copy link
Owner

miyagawa commented Nov 7, 2016

This looks reasonable to me - is it possible to write a unit test in xt ? there are several existing tests that uses a local directory as a mirror source.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants