Release 2020.07 - RC1 #156
Assignees
Labels
Comments
|
Currently running the tests provided by #155, but not putting my name behind them because other people should try it too :-). |
|
First batch of results are in Click on the link, to know more (e.g. why tests were skipped). So a good first run IMHO. Sadly, the
|
No it is not. After having a look with Output of 3.5 parsed with
|
|
At least 4.9 now runs successfully, due to my fix in 9e23b81. |
|
I've also ran the skipped sudo RIOTBASE=${RIOTBASE} tox -- -m sudo_only |
|
( |
|
I implicitly ran |
|
Here are the results |
|
As always 7.5 is failing (see #142 (comment)) but the framework accounts for that now. |
|
08-interop Task 11 - UDP exchange between iotlab-m3 and Zephyr: PASS RIOT Node Zephyr Node Edit: I used Zephyr v2.3.0 to test this. |
|
I've automated what I can of 8, and tasks 1 and 8 are succeeding. (the rest is more complicated to automate) |
|
11-lorawan Task 01 - LoRaWAN example: PASSTask 02 - OTAA join procedure / Task 03 - ABP join procedure: PASSTask 04 - LoRaWAN device parameters persistence: PASS |
|
08-interop: Task 03 - UDP exchange between iotlab-m3 and Contiki PASSThe testing procedures need to be adapted for RIOT-OS/RIOT#13568 though. |
See #166 |
|
I've automated the tests for 10. They require RIOT-OS/RIOT#14510, but they PASS |
|
The CoAP tests are now also fully automated (#168). Here are the passing results |
|
@cgundogan @jia200x @MrKevinWeiss @leandrolanzieri @PeterKietzmann can one of you do 4.5-8, please. The |
|
02-tests, Task 01 Unable to reproduce locally. I guess, since I am running currently however, I can reproduce. |
From the test's README
So that is the reason why it does not work on |
|
I ran all the tests that require root permissions for sudo dist/tools/tapsetup/tapsetup
for test in tests/{gnrc_dhcpv6,gnrc_ipv6_ext,gnrc_ipv6_nib_dns,gnrc_rpl_srh,gnrc_tcp}*; do
RIOT_CI_BUILD=1 make --no-print-directory -C ${test} -j16 flash &&
sudo make --no-print-directory -C ${test} test ||
break
doneIt succeeded As such, considering #156 (comment) and #156 (comment), I consider 2.1 to PASS. |
|
|
I seem to have run an old version of RIOT in #156 (comment). See RIOT-OS/RIOT#14524. |
|
I as the the error space for spec 02, task 03 overlaps with spec 02, task 02 I don't expect any new insights in trying them locally, and just refer to the latter. |
|
No big surprises in the remaining interop tests. |
|
So only |
|
And of course |
(on the other hand I think I might have configured something wrong for |
|
I don't have an arduino-zero so I could not run Task 07 and Task 08, but the other 2 look good. 04-single-hop-6lowpan-icmp Task 05 (Experimental) - ICMPv6 multicast echo with samr21-xpro/remote: PASSTask 06 (Experimental) - ICMPv6 link-local echo with samr21-xpro/remote: PASS |
I started running them now. |
|
04-single-hop-6lowpan-icmp Ran using the release test scripts. Task 07 (Experimental) - ICMPv6 multicast echo with samr21-xpro/zero + xbee PASSTask 08 (Experimental) - ICMPv6 echo with samr21-xpro/zero + xbeePASS |
|
I think I got closer to the issue in 3.5: The NIB seems to get stuck in the condition that the next scheduled NS is closer to the next retransmission, so sending the NS is just skipped, causing the retransmission timer not to go down, causing the Neighbor Cache entry to stay, causing the packet to stay in its queue. Will investigate, why it gets stuck. |
Talked offline with @fjmolinas: We won't wait for that for RC2. For all currently running there were no surprises. So only thing waiting for RC2 is |
|
There seems to be a bug in the compile test script. It says 0 errors but there are errors reported: Have to investigate what is the problem with the failing ones |
|
What I executed was btw DOCKER_MAKE_ARGS="-j16" \
DOCKER_ENV_VARS=$(env | grep "^CCACHE" | cut -d= -f1) \
DOCKER_VOLUMES_AND_ENV="-v /tmp:/tmp" \
BUILD_IN_DOCKER=1 \
./dist/tools/compile_test/compile_test.py |
|
On deleted comment: tested something ;-). |
|
Ok, most failures are just I can't reproduce this however isolated, by building those boards just with |
|
Closing in favor of #172 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This issue lists the status of all tests for the Release Candidate 1 of the 2020.07 release.
Specs tested:
iotlab-m3due tocore/cpuregression RIOT#14521, tests/gnrc_dhcpv6_client: fix expected output RIOT#14524)Task #05 (Experimental) - UDP with large payload on iotlab-m3 with three hops (RPL route)(see Release 2020.07 - RC1 #156 (comment))Task #09 - UDP between GNRC and emb6 on iotlab-m3(deprecated, see 08-interop: deprecate 9-10 #157)Task #10 - UDP between lwIP and emb6 on iotlab-m3(deprecated, see 08-interop: deprecate 9-10 #157)The text was updated successfully, but these errors were encountered: