Maska989/ping_test #254

maska989 · 2023-09-18T10:19:11Z

Description

JIRA: CI-326

Motivation and Context

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

How Has This Been Tested?

Already covered by automatic testing.
New test added: (add PR link here).
Tested by hand on: (list targets here).

Checklist:

My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added tests to cover my changes.
All new and existing linter checks and tests passed.
My changes generate no new compilation warnings for any of the targets.

Special treatment

This PR needs additional PRs to work (list the PRs, preferably in merge-order).
_project: setup targets for PPP and LwiP usage phoenix-rtos-project#1021
psh: implement route management tool phoenix-rtos-utils#207
I will merge this PR by myself when appropriate.

psh/test-ping.py

github-actions · 2023-09-18T10:32:49Z

Unit Test Results

5 681 tests - 1 779 5 142 ✅ - 1 602 28m 25s ⏱️ - 10m 15s
313 suites - 107 537 💤 - 178
1 files ± 0 2 ❌ + 1

For more details on these failures, see this check.

Results for commit 0981721. ± Comparison against base commit 04b1486.

This pull request removes 1780 and adds 1 tests. Note that renamed tests count towards both.

flash ‑ armv7a9-zynq7000-zedboard:flash
phoenix-rtos-tests/cpp/hello-cpp ‑ armv7a7-imx6ull-evk:phoenix-rtos-tests/cpp/hello-cpp
phoenix-rtos-tests/cpp/hello-cpp ‑ armv7a9-zynq7000-zedboard:phoenix-rtos-tests/cpp/hello-cpp
phoenix-rtos-tests/initfini/main ‑ armv7a7-imx6ull-evk:phoenix-rtos-tests/initfini/main
phoenix-rtos-tests/initfini/main ‑ armv7a9-zynq7000-zedboard:phoenix-rtos-tests/initfini/main
phoenix-rtos-tests/ioctl/unit ‑ armv7a7-imx6ull-evk:phoenix-rtos-tests/ioctl/unit.ioctl.data_in
phoenix-rtos-tests/ioctl/unit ‑ armv7a7-imx6ull-evk:phoenix-rtos-tests/ioctl/unit.ioctl.data_in_big
phoenix-rtos-tests/ioctl/unit ‑ armv7a7-imx6ull-evk:phoenix-rtos-tests/ioctl/unit.ioctl.data_inout
phoenix-rtos-tests/ioctl/unit ‑ armv7a7-imx6ull-evk:phoenix-rtos-tests/ioctl/unit.ioctl.data_inout_big
phoenix-rtos-tests/ioctl/unit ‑ armv7a7-imx6ull-evk:phoenix-rtos-tests/ioctl/unit.ioctl.data_out
…

phoenix-rtos-tests/psh/ping_lan ‑ armv7m7-imxrt106x-evk:phoenix-rtos-tests/psh/ping_lan

♻️ This comment has been updated with latest results.

mateusz-bloch

Overall, the test looks good.

Do we know if pinging domain names works, and is there an intention to support it?
What happen if we ping broadcast address?

psh/test-ping.py

nalajcie · 2023-09-27T15:40:08Z

Generic remarks:

we agreed to set up networking by directly using ifconfig command (right now from busybox, psh applet in the future)
don't assume the networking doesn't work at the beginning of the test (we will successively add more networking tests, this might not be the first one to run)
make the test run as fast as possible (we don't want to waste too much time waiting for CI results in PRs), eventual long tests need to be a separate testcase running only in nightly workflows (but still not exceedingly long)
structure the code that basic ping test could be re-used multiple times as it's a great check if networking works (eg. we would like to assign static IP, check if networking works, assign IP from DHCP, check if ping works, enable special network firewall rules, check if incoming/outgoing packets to certain hosts are indeed blocked) - these are all networking tests which will use ping as a verification tool
no need to solve it in this PR, but take note that this is a hard task, probably would need architectural changes in trunner - how to assign unique static IP (and other properties in the future) for every runner instance individually in a generic way to be available/used in harnesses while testing
please don't reformat python files unnecessarily

maska989 · 2023-09-28T06:28:13Z

TWM for writing about generic remarks, all of this needs rethink about test structure and runner options.
About:

* please don't reformat python files unnecessarily

Rn I'm using this PR to research/configure Ethernet support on QEMU ia32 because CI if even run network.sh to set up still ping don't pass.
https://github.com/phoenix-rtos/phoenix-rtos-project/pull/852
That's why still this test isn't converted to use just ifconfig.
My fault was not converted this PR to draft, probably after full research I will delete this PR and create new one (or clear a little bit for focus strictly on ping test).

Again, thanks 4 all your help.

nalajcie · 2023-09-28T09:21:55Z

Rn I'm using this PR to research/configure Ethernet support on QEMU ia32 because CI if even run network.sh to set up still ping don't pass. https://github.com/phoenix-rtos/phoenix-rtos-project/pull/852 That's why still this test isn't converted to use just ifconfig. My fault was not converted this PR to draft, probably after full research I will delete this PR and create new one (or clear a little bit for focus strictly on ping test).

IMHO using ifconfig directly would make experimenting clearer. From what I see:

you're using DHCP setup right now - so that won't work in qemu bridged mode straight away (no DHCP server in GH network would want to give you an address due to security reasons)
the dhcp could easily be used in user qemu set-up but this won't allow us to reach WAN (still, ping 10.0.2.2 works (default for qemu server, can be configured by params as described in help))
we probably should set-up bridge networking with NAT/private DHCP (should be possible in docker, not sure about tap interface tho) if we need access to WAN
all the above network-setup tests can be done locally (in docker - you can disconnect it from network)
if network setup would still fail in CI after the local testing succeeds - I would advice adding ssh tunnel to CI and experiment interactively in runner (eg: https://github.com/mxschmitt/action-tmate)
for PING tests we don't need access to WAN, we could simply use only LAN addresses (eg. the address of the host running test runner)
nevertheless, setup with WAN access might be beneficial in the future in other tests (eg. ntp client, wget, etc. -> we won't need to set up local instances of various servers on the test runner host ;))

psh/test-hw-ping-lan.py

maska989 · 2023-11-16T12:58:05Z

Right now this PR waiting for ifconfig and route implementation.

psh/test.yaml

psh/test-hw-ping-lan.py

mateusz-bloch

Several morelike cosmetic improvements, you don't have to apply them if you don't want to

psh/test-hw-ping-lan.py

psh/test-hw-ping-wan.py

psh/tools/netset.py

mateusz-bloch · 2024-02-20T11:16:21Z

psh/tools/netset.py

+    port: Optional[str] = "/dev/ttyUSB0",
+    host_ip: Optional[str] = "10.0.0.1",
+    target_ip: Optional[str] = "10.0.0.2",
+    interface_idle_time: Optional[int] = "10",


Parameter is declared as an Optional[int] but has a default value of type str

Suggested change

interface_idle_time: Optional[int] = "10",

interface_idle_time: Optional[int] = 10,

mateusz-bloch · 2024-02-20T11:18:08Z

psh/tools/netset.py

+        time.sleep(0.01)
+        asleep += 0.01


inefficient timeout ? Is it worth invoking ifconfig every 0.01 second?

mateusz-bloch · 2024-02-20T11:19:57Z

psh/tools/netset.py

+    host_setup = inet + netmask
+    return host_setup


maybe, but do whatever you think is more readable

Suggested change

host_setup = inet + netmask

return host_setup

return inet + netmask

mateusz-bloch · 2024-02-20T11:24:13Z

psh/test-hw-ping-wan.py

+IP_LOOPBACK = "127.0.0.1"
+
+
+def ping_output_regex(target, iteration, possible_fail: Optional[bool] = False):


Is Optional needed ?

JIRA: CI-326

maska989 added the Tests label Sep 18, 2023

github-actions bot reviewed Sep 18, 2023

View reviewed changes

maska989 force-pushed the maska989/ping_test branch 2 times, most recently from 7e2369c to b5fd34f Compare September 18, 2023 10:23

maska989 marked this pull request as ready for review September 18, 2023 12:32

mateusz-bloch reviewed Sep 19, 2023

View reviewed changes

maska989 force-pushed the maska989/ping_test branch from b5fd34f to 9aee81f Compare September 25, 2023 12:41

github-actions bot reviewed Sep 25, 2023

View reviewed changes

psh/test-ping.py Outdated Show resolved Hide resolved

maska989 force-pushed the maska989/ping_test branch 9 times, most recently from 20ff96e to 4bb41a8 Compare September 27, 2023 12:53

maska989 marked this pull request as draft September 28, 2023 06:16

maska989 force-pushed the maska989/ping_test branch from 4bb41a8 to 16017af Compare September 28, 2023 07:25

maska989 force-pushed the maska989/ping_test branch 5 times, most recently from 4d581c8 to 6535925 Compare October 10, 2023 08:31

maska989 force-pushed the maska989/ping_test branch 2 times, most recently from 072f28a to 2e9f237 Compare October 26, 2023 07:15

maska989 force-pushed the maska989/ping_test branch from 6de577e to 7e1eb07 Compare November 16, 2023 10:20

github-actions bot reviewed Nov 16, 2023

View reviewed changes

psh/test-hw-ping-lan.py Outdated Show resolved Hide resolved

psh/test-hw-ping-lan.py Outdated Show resolved Hide resolved

maska989 force-pushed the maska989/ping_test branch from 7e1eb07 to 9e47669 Compare November 16, 2023 12:53

maska989 marked this pull request as ready for review November 16, 2023 12:56

maska989 assigned maska989 and unassigned maska989 Nov 16, 2023

nalajcie mentioned this pull request Nov 22, 2023

psh/ping: add hostname to IP address resolving phoenix-rtos/phoenix-rtos-utils#204

Merged

13 tasks

maska989 force-pushed the maska989/ping_test branch from 9e47669 to 02151d0 Compare November 24, 2023 11:28

mateusz-bloch reviewed Nov 27, 2023

View reviewed changes

psh/test.yaml Outdated Show resolved Hide resolved

maska989 force-pushed the maska989/ping_test branch 4 times, most recently from cc57492 to 7163e55 Compare December 7, 2023 08:25

github-actions bot reviewed Dec 7, 2023

View reviewed changes

psh/test-hw-ping-lan.py Outdated Show resolved Hide resolved

psh/test-hw-ping-lan.py Outdated Show resolved Hide resolved

maska989 force-pushed the maska989/ping_test branch 2 times, most recently from 5b3ecaf to 8724481 Compare December 7, 2023 14:52

maska989 marked this pull request as draft December 8, 2023 07:34

maska989 requested a review from damianloew December 8, 2023 07:34

maska989 force-pushed the maska989/ping_test branch 2 times, most recently from 44aeddc to b420c60 Compare December 21, 2023 12:46

mateusz-bloch reviewed Jan 3, 2024

View reviewed changes

maska989 force-pushed the maska989/ping_test branch from b420c60 to cd1494b Compare February 14, 2024 14:28

github-actions bot reviewed Feb 14, 2024

View reviewed changes

psh/tools/netset.py Outdated Show resolved Hide resolved

maska989 force-pushed the maska989/ping_test branch 3 times, most recently from d75cf35 to 2dc6ef3 Compare February 19, 2024 11:22

mateusz-bloch reviewed Feb 20, 2024

View reviewed changes

psh: local and wan ping tests

0981721

JIRA: CI-326

maska989 force-pushed the maska989/ping_test branch from 2dc6ef3 to 0981721 Compare April 29, 2024 10:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Maska989/ping_test #254

Maska989/ping_test #254

maska989 commented Sep 18, 2023 •

edited

Loading

github-actions bot commented Sep 18, 2023 •

edited

Loading

mateusz-bloch left a comment

nalajcie commented Sep 27, 2023

maska989 commented Sep 28, 2023 •

edited

Loading

nalajcie commented Sep 28, 2023

maska989 commented Nov 16, 2023

mateusz-bloch left a comment

mateusz-bloch Feb 20, 2024

mateusz-bloch Feb 20, 2024

mateusz-bloch Feb 20, 2024

mateusz-bloch Feb 20, 2024

	interface_idle_time: Optional[int] = "10",
	interface_idle_time: Optional[int] = 10,

	host_setup = inet + netmask
	return host_setup
	return inet + netmask

		IP_LOOPBACK = "127.0.0.1"


		def ping_output_regex(target, iteration, possible_fail: Optional[bool] = False):

Maska989/ping_test #254

Are you sure you want to change the base?

Maska989/ping_test #254

Conversation

maska989 commented Sep 18, 2023 • edited Loading

Description

Motivation and Context

Types of changes

How Has This Been Tested?

Checklist:

Special treatment

github-actions bot commented Sep 18, 2023 • edited Loading

Unit Test Results

mateusz-bloch left a comment

Choose a reason for hiding this comment

nalajcie commented Sep 27, 2023

maska989 commented Sep 28, 2023 • edited Loading

nalajcie commented Sep 28, 2023

maska989 commented Nov 16, 2023

mateusz-bloch left a comment

Choose a reason for hiding this comment

mateusz-bloch Feb 20, 2024

Choose a reason for hiding this comment

mateusz-bloch Feb 20, 2024

Choose a reason for hiding this comment

mateusz-bloch Feb 20, 2024

Choose a reason for hiding this comment

mateusz-bloch Feb 20, 2024

Choose a reason for hiding this comment

maska989 commented Sep 18, 2023 •

edited

Loading

github-actions bot commented Sep 18, 2023 •

edited

Loading

maska989 commented Sep 28, 2023 •

edited

Loading