Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added e2e tests for untagged and any new EVCs #210

Merged
merged 3 commits into from
Mar 14, 2023

Conversation

Alopalao
Copy link

@Alopalao Alopalao commented Mar 3, 2023

Related to mef_eline and of_core PRs

Summary

Tests including the recent addition to vlan available values untagged and any.

Local Tests

tests/test_e2e_20_flow_manager.py ....................                   [ 39%]
tests/test_e2e_21_flow_manager.py ..                                     [ 43%]
tests/test_e2e_22_flow_manager.py ...............                        [ 72%]
tests/test_e2e_23_flow_manager.py ..............                         [100%]

@viniarck viniarck self-requested a review March 6, 2023 16:01
Copy link
Member

@viniarck viniarck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Alopalao great e2e cov.

  • Did you also run this locally?

@Alopalao
Copy link
Author

Alopalao commented Mar 6, 2023

Yes, the results are posted on mef_eline PR too,

tests/test_e2e_01_kytos_startup.py ..                                    [  0%]
tests/test_e2e_05_topology.py ..................                         [  9%]
tests/test_e2e_10_mef_eline.py ..........ss.....x.....x........R.        [ 25%]
tests/test_e2e_11_mef_eline.py .....                                     [ 27%]
tests/test_e2e_12_mef_eline.py .....xx.                                  [ 31%]
tests/test_e2e_13_mef_eline.py .....xxXx......xxXx.XXxX.xxxx..x......... [ 50%]
...                                                                      [ 52%]
tests/test_e2e_14_mef_eline.py x                                         [ 52%]
tests/test_e2e_15_mef_eline.py ..                                        [ 53%]
tests/test_e2e_20_flow_manager.py .....RRF.RRF...........                [ 62%]
tests/test_e2e_21_flow_manager.py ...                                    [ 64%]
tests/test_e2e_22_flow_manager.py ...............                        [ 71%]
tests/test_e2e_23_flow_manager.py ..............                         [ 78%]
tests/test_e2e_30_of_lldp.py ....                                        [ 80%]
tests/test_e2e_31_of_lldp.py R...                                        [ 81%]
tests/test_e2e_32_of_lldp.py ...                                         [ 82%]
tests/test_e2e_40_sdntrace.py RRF.RRFRRF                                 [ 84%]
tests/test_e2e_41_kytos_auth.py ........                                 [ 88%]
tests/test_e2e_50_maintenance.py ........................                [100%]

h11.cmd('ip addr add 100.0.0.11/24 dev vlan100')
h2.cmd('ip link add link %s name vlan100 type vlan id 100' % (h2.intfNames()[0]))
h2.cmd('ip link set up vlan100')
h2.cmd('ip addr add 100.0.0.2/24 dev vlan100')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

symmetric hosts VLANs I think it's fair game to test this use case, but let's also assert the installed UNI flows on the switches based on your spreadsheet for completeness. @italovalcy let us know if you'd like to see this tested differently.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it is worth adding assertions on the flows besides doing the ping test. Especially since we don't have tests for asymmetric VLANs so far.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I divided the PR, here is the correct one for mef_eline

@viniarck
Copy link
Member

viniarck commented Mar 6, 2023

Yes, the results are posted on mef_eline PR too,

tests/test_e2e_01_kytos_startup.py ..                                    [  0%]
tests/test_e2e_05_topology.py ..................                         [  9%]
tests/test_e2e_10_mef_eline.py ..........ss.....x.....x........R.        [ 25%]
tests/test_e2e_11_mef_eline.py .....                                     [ 27%]
tests/test_e2e_12_mef_eline.py .....xx.                                  [ 31%]
tests/test_e2e_13_mef_eline.py .....xxXx......xxXx.XXxX.xxxx..x......... [ 50%]
...                                                                      [ 52%]
tests/test_e2e_14_mef_eline.py x                                         [ 52%]
tests/test_e2e_15_mef_eline.py ..                                        [ 53%]
tests/test_e2e_20_flow_manager.py .....RRF.RRF...........                [ 62%]
tests/test_e2e_21_flow_manager.py ...                                    [ 64%]
tests/test_e2e_22_flow_manager.py ...............                        [ 71%]
tests/test_e2e_23_flow_manager.py ..............                         [ 78%]
tests/test_e2e_30_of_lldp.py ....                                        [ 80%]
tests/test_e2e_31_of_lldp.py R...                                        [ 81%]
tests/test_e2e_32_of_lldp.py ...                                         [ 82%]
tests/test_e2e_40_sdntrace.py RRF.RRFRRF                                 [ 84%]
tests/test_e2e_41_kytos_auth.py ........                                 [ 88%]
tests/test_e2e_50_maintenance.py ........................                [100%]

Nice. sdntrace_cp failing is expected until this issue kytos-ng/sdntrace_cp#78 is addressed. I've noticed some reruns on this output here, later on I'll also recommend for you to run each test isolated here and keep eye on reruns they're not usually expected when tests are stable and written in way that favor deterministic execution.

time.sleep(10)

h11, h2 = self.net.net.get('h11', 'h2')
h11.cmd('ip link add link %s name vlan100 type vlan id 100' % (h11.intfNames()[0]))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are using the "any" VLAN here, what do you think about using a random value for the VLAN on the PING test? I suggest replacing the VLAN 100 with a random value chosen from a range from 1 to 4095. Let me know what you think.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, will do that.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I divided the PR, here is the correct one for mef_eline

@viniarck
Copy link
Member

viniarck commented Mar 8, 2023

Yes, the results are posted on mef_eline PR too,

tests/test_e2e_01_kytos_startup.py ..                                    [  0%]
tests/test_e2e_05_topology.py ..................                         [  9%]
tests/test_e2e_10_mef_eline.py ..........ss.....x.....x........R.        [ 25%]
tests/test_e2e_11_mef_eline.py .....                                     [ 27%]
tests/test_e2e_12_mef_eline.py .....xx.                                  [ 31%]
tests/test_e2e_13_mef_eline.py .....xxXx......xxXx.XXxX.xxxx..x......... [ 50%]
...                                                                      [ 52%]
tests/test_e2e_14_mef_eline.py x                                         [ 52%]
tests/test_e2e_15_mef_eline.py ..                                        [ 53%]
tests/test_e2e_20_flow_manager.py .....RRF.RRF...........                [ 62%]
tests/test_e2e_21_flow_manager.py ...                                    [ 64%]
tests/test_e2e_22_flow_manager.py ...............                        [ 71%]
tests/test_e2e_23_flow_manager.py ..............                         [ 78%]
tests/test_e2e_30_of_lldp.py ....                                        [ 80%]
tests/test_e2e_31_of_lldp.py R...                                        [ 81%]
tests/test_e2e_32_of_lldp.py ...                                         [ 82%]
tests/test_e2e_40_sdntrace.py RRF.RRFRRF                                 [ 84%]
tests/test_e2e_41_kytos_auth.py ........                                 [ 88%]
tests/test_e2e_50_maintenance.py ........................                [100%]

@Alopalao I've closed the threads that I opened. Thanks for pushing these updates. Other than the threads that you're aligning with Italo, when it's done, please rerun e2e selecting flow_manager related tests.

Also, another piece of advice will be to break this PR in two, since mef_eline PR is still in review, if you split it in two, covering flow_manager here and mef_eline on another one then of_core PR kytos-ng/of_core#98 could land soon as well once this PR here is approved. You could still keep both but it will sit longer in code review.

Copy link
Member

@viniarck viniarck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I'll leave this PR approved since Aldo managed to split the PRs to simplify the review and the open discussions will continue on PR #213

@Alopalao, can you rerun one last time test_032_on_switch_reconnection_should_recreate_untagged_any_flows to ensure it doesn't have any auto-rerun from pytest?

@Alopalao
Copy link
Author

Alopalao commented Mar 9, 2023

New build test results:

tests/test_e2e_20_flow_manager.py ....................                   [ 39%]
tests/test_e2e_21_flow_manager.py ..                                     [ 43%]
tests/test_e2e_22_flow_manager.py ...............                        [ 72%]
tests/test_e2e_23_flow_manager.py ..............                         [100%]

Copy link
Collaborator

@italovalcy italovalcy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for sending this PR to cover the changes on flow_manger and of_core, Aldo! Very well done.

@Alopalao Alopalao merged commit a6b9a56 into kytos-ng:master Mar 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants