Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Explainability with hcs #477

Merged
merged 205 commits into from
Jul 5, 2023
Merged

Explainability with hcs #477

merged 205 commits into from
Jul 5, 2023

Conversation

shmfr
Copy link
Collaborator

@shmfr shmfr commented Apr 13, 2023

No description provided.

@tanyaveksler
Initial implementation of the optimized TcpLikeProperties (and HyperC…
74ec605 
…ubeSet) holding all connections including src_peers, dst_peers and protocols

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Extended testcase3 to produce connectivity_map
c279de6 
Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Fixed a problem in HyperCubeSet (wrongly changing self in _and_aux.
1b1ebbe 
Added optimized_denied_ingress_props
and optimized_denied_egress_props (in addition to allowed ones).
Improved non_captured_conns computation

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Small fix
09cfad1 
Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Further fix of the hyper cube set
94fb117 
Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Further fix of the hyper cube set
520377a 
Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Merge remote-tracking branch 'origin/Optimized_HC_set' into Optimized…
af37b10 
…_HC_set

# Conflicts:
#	nca/CoreDS/CanonicalHyperCubeSet.py
@tanyaveksler
Avoiding redundant and heavy copy of layers.
0339215 
Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Merge branch 'master' into Optimized_HC_set
564d667
@tanyaveksler
General changes from the Optimized_HC_set branch.
b9810b3 
Signed-off-by: Tanya <[email protected]>
@tanyaveksler
General changes from the Optimized_HC_set branch.
e82ac99 
Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Avoiding lint warnings
6b3b364 
Signed-off-by: Tanya <[email protected]>

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Avoiding lint warnings
cfb5ee7 
Signed-off-by: Tanya <[email protected]>

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Merge branch 'Small-fixes-for-master' into Optimized_HC_set
01a9d40
@tanyaveksler
Added support of IpBlocks in optimized hyper cube set implementation.
28e211b 
Signed-off-by: Tanya <[email protected]>
@tanyaveksler
More accurate update of base_peer_set.
d5d97a6 
Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Added dedundant fw_rules creation for testing (to be further removed).
0a2cb45 
Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Initial implementation of Calico optimized connections handling.
0800959 
Fixed protocol handling: 0 is a valid protocol number (HOPOPT).
Allowing any protocol in the range [0...255], though ProtocolNameResolver does not contain names of all of the possible 256 protocols.
Fixed handling non-captured peers in K8S (cannot be handled as denied).

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Fixed the construction of connectivity graph (when some of src_peer o…
05a496c 
…r dst_peers dimensions is all values).

Added optimization for fw_rules_map - join different entries having the same values (fw_rules).

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Multiple fixes:
bb6842c 
1. To represent No connections, do not build TcpLikeProperties (TcpLikeProperties with no dimensions represent All connections).
2. Support subsets in query in optimized solution.
3. For comparison of optimized solution to the original one, add connections from peers to themselves.
4. More accurate comparison for 'dot' connectivity queries.
5. Generalized convert_named_ports (to not assume dimensions order)
6. Handling the possibility when projection on one dimension is empty.

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Properly handling 'False' represented by TcpLikeProperties.
0d4bd28 
Properly handling HostEPs in optimized TcpLikeProperties.

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Added support to ICMP data in optimized HC set (handled in TCPLikePro…
47d5bd0 
…perties)

Fixed handling of non-captured pods in optimized solution.
Added using True/False HC_set (make_all_properties()/make_empty_properties())

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Added command line flag optimized_run, having 3 possibilities:
0a8d53f 
'false' - only original run
'true' - only optimized run
'debug'- both runs and comparison of their results.

Printing parsing time, queries time and total run time.

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Small fixes;
183eaeb 
Workaround for the bug in HC set: using mutual contained_in, instead of ==

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Small fixes;
8e34573 
Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Fixed building tcp_like_properties form connectivity graph
9957228 
Signed-off-by: Tanya <[email protected]>

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Optimized the comparison between original and optimized connections (…
004742b 
…for -opt=debug option)

Added more debug prints.
Better handling of peer_set copying in TcpLikeProperties.

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Implemented optimized Istio policy handling.
a27899d 
Further optimization - calculating ref_ip_blocks only in non-optimized run.

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Added Ingress policy support in the optimized solution.
ebae572 
Improved comments.

Signed-off-by: Tanya <[email protected]>
@tanyaveksler
Added Istio Ingress policy support in the optimized solution.
25118de 
Signed-off-by: Tanya <[email protected]>
shmfr added 4 commits June 7, 2023 21:01
@shmfr
manually adding tests
89789d9 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
manually updating test times
c0b32de 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
fixed some testing env agnostic issues
cd9d28f 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
better handling of ipblocks and some small fixes.
92ce374 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
Copy link
Collaborator Author

shmfr commented Jun 8, 2023

output comments: for ip-block as src or dst - no need to add policy configurations or resources configurations.

Done - "ipblocks had no configurations" explanation now replace the explanation for Ipblocks.

Test run issue: when running with explain: ALL and without -opt=true , nca crashes in explain_all function, though I would expect this function should not be called.

I could not re-create the crash. Perhaps was tested on an older version? the current version protects this case.

@shmfr
removing explanation for ipblocks
c402d73 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
adding policies in one place
d835df7 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
adding some documentations and test
b7ceb53 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
after merge with master
895d75d 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
updated expected results
dea4134 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
fixed run_all_tests path problem
c4761fc 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
duplicate line removed
fd40627 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr shmfr requested a review from adisos June 22, 2023 12:59
adisos
adisos reviewed Jun 26, 2023
View reviewed changes
docs/SchemeFileFormat.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
docs/SchemeFileFormat.md Outdated Show resolved Hide resolved
shmfr added 10 commits June 26, 2023 18:33
@shmfr
updated testes and some docs.
cdaa329 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
little better explanation
658811e 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
some small fixes
7c6ac51 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
base_name added
a468689 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
better documentation for using IP-blocks
224e43b 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
added a ip-block test
cae6397 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
base_name check
7dcfc67 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
base_name check
c33d88a 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr
Merge remote-tracking branch 'origin/master' into explainability_with…
43f494b 
…_HCS
@shmfr
base_name and sync
2a6e68f 
Signed-off-by: Shmulik Froimovich <[email protected]>
@shmfr shmfr merged commit 7335253 into master Jul 5, 2023
@shmfr shmfr deleted the explainability_with_HCS branch July 6, 2023 06:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tanyaveksler tanyaveksler tanyaveksler left review comments

@adisos adisos adisos approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@shmfr @tanyaveksler @adisos