Test suite does not support rootless containers.

[EvanCarroll]

If you try to run this in a rootless container you'll see a lot of failures.

~/aports/testing/perlcc/src/perl-5.32.1 # PERL5LIB=lib LD_LIBRARY_PATH=. ./perl ./utils/prove -l dist/Net-Ping/t/*
dist/Net-Ping/t/000_load.t ........... ok   
dist/Net-Ping/t/001_new.t ............ 1/? icmp socket error - Operation not permitted at dist/Net-Ping/t/001_new.t line 71.
# Looks like your test exited with 1 just after 9.
dist/Net-Ping/t/001_new.t ............ Dubious, test returned 1 (wstat 256, 0x100)
All 9 subtests passed 
dist/Net-Ping/t/010_pingecho.t ....... ok   
dist/Net-Ping/t/110_icmp_inst.t ...... 1/2 icmp socket error - Operation not permitted at dist/Net-Ping/t/110_icmp_inst.t line 24.
# Looks like your test exited with 1 just after 1.
dist/Net-Ping/t/110_icmp_inst.t ...... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/2 subtests 
dist/Net-Ping/t/120_udp_inst.t ....... ok   
dist/Net-Ping/t/130_tcp_inst.t ....... ok   
dist/Net-Ping/t/140_stream_inst.t .... ok   
dist/Net-Ping/t/150_syn_inst.t ....... ok   
dist/Net-Ping/t/190_alarm.t .......... ok   
dist/Net-Ping/t/200_ping_tcp.t ....... ok     
dist/Net-Ping/t/250_ping_hires.t ..... ok   
dist/Net-Ping/t/300_ping_stream.t .... skipped: loopback tcp echo service is off (Connection refused)
dist/Net-Ping/t/400_ping_syn.t ....... ok     
dist/Net-Ping/t/410_syn_host.t ....... ok     
dist/Net-Ping/t/420_ping_syn_port.t .. ok     
dist/Net-Ping/t/450_service.t ........ ok     
dist/Net-Ping/t/500_ping_icmp.t ...... icmp socket error - Operation not permitted at dist/Net-Ping/t/500_ping_icmp.t line 58.
dist/Net-Ping/t/500_ping_icmp.t ...... Dubious, test returned 1 (wstat 256, 0x100)
No subtests run 
dist/Net-Ping/t/501_ping_icmpv6.t .... icmp socket error - Operation not permitted at dist/Net-Ping/t/501_ping_icmpv6.t line 49.
dist/Net-Ping/t/501_ping_icmpv6.t .... Dubious, test returned 1 (wstat 256, 0x100)
No subtests run 
dist/Net-Ping/t/510_ping_udp.t ....... ok   
dist/Net-Ping/t/520_icmp_ttl.t ....... 1/? icmp socket error - Operation not permitted at dist/Net-Ping/t/520_icmp_ttl.t line 23.
# Looks like your test exited with 1 just after 1.
dist/Net-Ping/t/520_icmp_ttl.t ....... Dubious, test returned 1 (wstat 256, 0x100)
All 1 subtests passed 

Test Summary Report
-------------------
dist/Net-Ping/t/001_new.t          (Wstat: 256 Tests: 9 Failed: 0)
  Non-zero exit status: 1
dist/Net-Ping/t/110_icmp_inst.t    (Wstat: 256 Tests: 1 Failed: 0)
  Non-zero exit status: 1
  Parse errors: Bad plan.  You planned 2 tests but ran 1.
dist/Net-Ping/t/500_ping_icmp.t    (Wstat: 256 Tests: 0 Failed: 0)
  Non-zero exit status: 1
  Parse errors: No plan found in TAP output
dist/Net-Ping/t/501_ping_icmpv6.t  (Wstat: 256 Tests: 0 Failed: 0)
  Non-zero exit status: 1
  Parse errors: No plan found in TAP output
dist/Net-Ping/t/520_icmp_ttl.t     (Wstat: 256 Tests: 1 Failed: 0)
  Non-zero exit status: 1
Files=20, Tests=132, 53 wallclock secs ( 0.08 usr  0.04 sys +  1.27 cusr  0.23 csys =  1.62 CPU)
Result: FAIL

This is because the test suite is not capability aware. It assumes if it has root is has CAP_NET_RAW, but it is smart enough to at least know these tests are problematic.

ok 9 # skip icmp ping requires root privileges.
ok 10 # skip icmp ping requires root privileges.
ok 11 # skip icmp ping requires root privileges.

You can see this by building a rootless container with buildah. Use the attached build.sh script like buildah unshare ./build.sh

Then run the test suite with podman as a regular user,

podman run -ti netping /usr/bin/prove -l t

You can run it again with cap_net_raw to see the diff,

podman run --cap-add=cap_net_raw -ti netping /usr/bin/prove -l t 

build.sh.txt

[rurban]

Well, in this case the user has to be smart enough to use it appropriately. We don't have probes for caps yet, and I never have seen some.

[EvanCarroll]

It's certainly easy enough to do with getpcaps and if we want to allow the compilation in a rootless container (for devops and what not) it would be useful. Alternatively we could shell out to capsh --has-p=cap_net_raw. But even if probing is too scope creepy, I would think if the test returns "Operation not permitted" to skip it -- we don't want to fail perl's make test

[rurban]

Imho with Net-Ping it should be allowed for tests to fail. It's highly specific to the user environment. firewall's, routing, docker, ...

Only when he understands why a test fails, he can override it, eg. via cpan -f -i.

capsh seems to be linux specific only. Much better would be crosstesting via external ping/ping6.

[EvanCarroll]

Imho with Net-Ping it should be allowed for tests to fail. It's highly specific to the user environment. firewall's, routing, docker, ...

I would ask Net::Ping to be delisted with corelist at least. Currently if anything fails in Net::Ping the whole make test on Perl fails, and it hardly makes sense for Perl to fail because of something "highly specific to the user environment" to block perl's make test. Or implement a more conservative sizing for Perl with xt/ or the like. Anyway this will be a pita as distros move to containerized builds.

capsh seems to be linux specific only. Much better would be crosstesting via external ping/ping6.

Capsh is linux specific, but so are containers. Your suggestion won't work on Linux because you can ping and still have all these tests above fail. If for example your group is listed in net.ipv4.ping_group_range. (Note, that's how the tests failed in the first place.. So ping worked fine and the tests failed).