Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Initial implementation of the optimized TcpLikeProperties (and HyperCubeSet) holding all connections including src_peers, dst_peers and protocols Signed-off-by: Tanya <[email protected]> * Extended testcase3 to produce connectivity_map Signed-off-by: Tanya <[email protected]> * Fixed a problem in HyperCubeSet (wrongly changing self in _and_aux. Added optimized_denied_ingress_props and optimized_denied_egress_props (in addition to allowed ones). Improved non_captured_conns computation Signed-off-by: Tanya <[email protected]> * Small fix Signed-off-by: Tanya <[email protected]> * Further fix of the hyper cube set Signed-off-by: Tanya <[email protected]> * Further fix of the hyper cube set Signed-off-by: Tanya <[email protected]> * Avoiding redundant and heavy copy of layers. Signed-off-by: Tanya <[email protected]> * General changes from the Optimized_HC_set branch. Signed-off-by: Tanya <[email protected]> * General changes from the Optimized_HC_set branch. Signed-off-by: Tanya <[email protected]> * Avoiding lint warnings Signed-off-by: Tanya <[email protected]> Signed-off-by: Tanya <[email protected]> * Avoiding lint warnings Signed-off-by: Tanya <[email protected]> Signed-off-by: Tanya <[email protected]> * Added support of IpBlocks in optimized hyper cube set implementation. Signed-off-by: Tanya <[email protected]> * More accurate update of base_peer_set. Signed-off-by: Tanya <[email protected]> * Added dedundant fw_rules creation for testing (to be further removed). Signed-off-by: Tanya <[email protected]> * Initial implementation of Calico optimized connections handling. Fixed protocol handling: 0 is a valid protocol number (HOPOPT). Allowing any protocol in the range [0...255], though ProtocolNameResolver does not contain names of all of the possible 256 protocols. Fixed handling non-captured peers in K8S (cannot be handled as denied). Signed-off-by: Tanya <[email protected]> * Fixed the construction of connectivity graph (when some of src_peer or dst_peers dimensions is all values). Added optimization for fw_rules_map - join different entries having the same values (fw_rules). Signed-off-by: Tanya <[email protected]> * Multiple fixes: 1. To represent No connections, do not build TcpLikeProperties (TcpLikeProperties with no dimensions represent All connections). 2. Support subsets in query in optimized solution. 3. For comparison of optimized solution to the original one, add connections from peers to themselves. 4. More accurate comparison for 'dot' connectivity queries. 5. Generalized convert_named_ports (to not assume dimensions order) 6. Handling the possibility when projection on one dimension is empty. Signed-off-by: Tanya <[email protected]> * Properly handling 'False' represented by TcpLikeProperties. Properly handling HostEPs in optimized TcpLikeProperties. Signed-off-by: Tanya <[email protected]> * Added support to ICMP data in optimized HC set (handled in TCPLikeProperties) Fixed handling of non-captured pods in optimized solution. Added using True/False HC_set (make_all_properties()/make_empty_properties()) Signed-off-by: Tanya <[email protected]> * Added command line flag optimized_run, having 3 possibilities: 'false' - only original run 'true' - only optimized run 'debug'- both runs and comparison of their results. Printing parsing time, queries time and total run time. Signed-off-by: Tanya <[email protected]> * Small fixes; Workaround for the bug in HC set: using mutual contained_in, instead of == Signed-off-by: Tanya <[email protected]> * Small fixes; Signed-off-by: Tanya <[email protected]> * Fixed building tcp_like_properties form connectivity graph Signed-off-by: Tanya <[email protected]> Signed-off-by: Tanya <[email protected]> * Optimized the comparison between original and optimized connections (for -opt=debug option) Added more debug prints. Better handling of peer_set copying in TcpLikeProperties. Signed-off-by: Tanya <[email protected]> * Implemented optimized Istio policy handling. Further optimization - calculating ref_ip_blocks only in non-optimized run. Signed-off-by: Tanya <[email protected]> * Added Ingress policy support in the optimized solution. Improved comments. Signed-off-by: Tanya <[email protected]> * Added Istio Ingress policy support in the optimized solution. Signed-off-by: Tanya <[email protected]> * Further optimization: converting HC set directly to fw rules. Signed-off-by: Tanya <[email protected]> * Small bug fix Signed-off-by: Tanya <[email protected]> * Fixed printing peer sets in FWRules. Signed-off-by: Tanya <[email protected]> * More released comparison between original and optimized fw-rules (allowing differences in auto-connections). Signed-off-by: Tanya <[email protected]> * Splitting istio opt properties to tcp and non-tcp properties. Fixed handling non captured peers in Istio policy. Signed-off-by: Tanya <[email protected]> * Fixed creation TcpLikeProperties with empty methods or protocols. Signed-off-by: Tanya <[email protected]> * Optimizing sidecar priorities handling by refinement of sidecar.selected_peers in parse time. Istio sidecar optimized connectivity implementation. Removing unused methods. Better non-captured handling. Signed-off-by: Tanya <[email protected]> * Adding newline at the end of connectivity test expected results. Signed-off-by: Tanya <[email protected]> * Handling exclude_ipv6 print in optimized calculation. Signed-off-by: Tanya <[email protected]> * Fixing initialization of MethodSet in HTTPRoute (None means no methods, MethodSet() means empty method set, which would create empty connections). Signed-off-by: Tanya <[email protected]> * Changed output format of ICMP data. Signed-off-by: Tanya <[email protected]> * Making default the original (not optimized) implementation in run_all_tests Signed-off-by: Tanya <[email protected]> * 1. Merge with master 2. Improved filtering of ipv6 blocks in optimized solution. Signed-off-by: Tanya <[email protected]> * Simplifying and improving make_tcp_like_properties function. Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Removed unised classes ConnectivityGraphPrototype and ConnectivityGraphOptimized. In creating TcpLikeProperties, methods and protocols cannot be None. Signed-off-by: Tanya <[email protected]> Signed-off-by: Tanya <[email protected]> * Avoid using creation of TcpLikeProperties directly with init; using make_tcp_like_properties, make_tcp_like_properties_from_dict, make_empty_properties or make_all_properties instead. Use icmp_type and icmp_code full domain intervals instead of None in parameters to TcpLikeProperties creation methods. Removed unused or commented-out code. Fixed project_on_one_dimension to return None (to represent all values) for non-active dimensions. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Fixing ConnectionSet.__str__ to be accurate, since it is used in sorting functions. Signed-off-by: Tanya <[email protected]> * Fixed excluding unused ipv6 blocks in the optimized solution. Signed-off-by: Tanya <[email protected]> * Renamed TcpLikeProperties to ConnectivityProperties. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * track expl data Signed-off-by: Shmulik Froimovich <[email protected]> * explain connectivity Signed-off-by: Shmulik Froimovich <[email protected]> * explain connectivity Signed-off-by: Shmulik Froimovich <[email protected]> * Simplified and cleaned interfaces. Signed-off-by: Tanya <[email protected]> * Fixed lint errors. Removed assertion which is no longer correct. Signed-off-by: Tanya <[email protected]> * Fixed lint errors. Signed-off-by: Tanya <[email protected]> * Added a new class ConnectivityCube that manages forth and back translations of all dimensions of ConnectivityProperties, (translations between input format and internal format). Consequently, changed API of ConnectivityProperties methods to use the above new class. Improved documentation. Signed-off-by: Tanya <[email protected]> * Added set_dims method to set multiple dimensions at once. Signed-off-by: Tanya <[email protected]> * Added get_protocol_set_with_single_protocol function to ProtocolSet. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * Fixed connectivity properties unit tests to match the new API. Signed-off-by: Tanya <[email protected]> * Aligned get_cube_dict to return str for all dimensions. Fixed resolving named ports for excluded_named_ports in the optimized solution. Signed-off-by: Tanya <[email protected]> * Removed unused ICMPDAtaSet class and its unit tests. Optimized ConnectivityProperties properties creation for semantic True (all full dimensions). Removed unused methods. Signed-off-by: Tanya <[email protected]> * Cleaner code using the new ConnectivityCube API. Signed-off-by: Tanya <[email protected]> * Added missing copy() in ConnectionSet. Fixed lint errors. Signed-off-by: Tanya <[email protected]> * Added missing copy() in ConnectionSet. Fixed lint errors. Signed-off-by: Tanya <[email protected]> * When running with -opt=debug, printing the original results of ConnectivityMap query. Signed-off-by: Tanya <[email protected]> * Made cleaner interface of ConectivityCube class, using __setitem__, __getitem__ and update functions. Made cleaner interface of creating empty/full/by cube ConnectivityProperties. Signed-off-by: Tanya <[email protected]> * explain connectivity Signed-off-by: Shmulik Froimovich <[email protected]> * Made cleaner interface of ConectivityCube class, using __setitem__, __getitem__ and update functions. Made cleaner interface of creating empty/full/by cube ConnectivityProperties. Signed-off-by: Tanya <[email protected]> * explain connectivity Signed-off-by: Shmulik Froimovich <[email protected]> * Fixed lint error. Signed-off-by: Tanya <[email protected]> * Small fix Signed-off-by: Tanya <[email protected]> * Added documentation and small code beautifications. Signed-off-by: Tanya <[email protected]> * Improved documentation. Simplified interface by adding ConnectivityCube.make_from_dict method. Signed-off-by: Tanya <[email protected]> * Small fix. Signed-off-by: Tanya <[email protected]> * Moved empty dimension values to DimensionsManager. Fixed project_on_one_dimension method. Signed-off-by: Tanya <[email protected]> * Moved empty dimension values to DimensionsManager. Fixed lint errors. Signed-off-by: Tanya <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Fixed small errors. Signed-off-by: Tanya <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectivityProperties.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/Parsers/CalicoPolicyYamlParser.py Co-authored-by: Adi Sosnovich <[email protected]> * Fixed lint errors. Signed-off-by: Tanya <[email protected]> * Removed base_peer_set from ConnectivityProperties and ConnectivityCube. Instead, added a singleton class BasePeerSet that keeps all peers and translates PeerSets to CanonicalIntervalSets and vice versa. Signed-off-by: Tanya <[email protected]> * Removed base_peer_set from ConnectivityProperties and ConnectivityCube. Instead, added a singleton class BasePeerSet that keeps all peers and translates PeerSets to CanonicalIntervalSets and vice versa. Signed-off-by: Tanya <[email protected]> * Added check to BasePeerSet.get_peer_interval_of that all peers are translated to intervals. Signed-off-by: Tanya <[email protected]> * updates after merge with HC branch Signed-off-by: Shmulik Froimovich <[email protected]> * updates after merge with HC branch Signed-off-by: Shmulik Froimovich <[email protected]> * removed unused function Signed-off-by: Shmulik Froimovich <[email protected]> * added some function descriptions Signed-off-by: Shmulik Froimovich <[email protected]> * txt_no_fe_rules format added Signed-off-by: Shmulik Froimovich <[email protected]> * Added OptimizedPolicyConnections class to hold allowed, denied and pass connections and captured peers. Signed-off-by: Tanya <[email protected]> * Fixed lint errors. Signed-off-by: Tanya <[email protected]> * The BasePeerSet singleton should be reset in the main (for the cases when running multiple tests, for example from run_all_tests) Signed-off-by: Tanya <[email protected]> * Added support to calico PASS rules in optimized solution. Signed-off-by: Tanya <[email protected]> * Added support to calico PASS rules in optimized solution. Signed-off-by: Tanya <[email protected]> * Simplified calico parsing method to avoid lint error. Fixed typo in the code. Signed-off-by: Tanya <[email protected]> * handling ipBlocks and base ip range * Generalized ServiceEntry implementation for optimized solution. Signed-off-by: Tanya <[email protected]> * Fixing lint errors. Signed-off-by: Tanya <[email protected]> * xml support for explain_all and default-policy fix Signed-off-by: Shmulik Froimovich <[email protected]> * Removed unused functions. Optimized BasePeerSet.get_peer_interval_of method. Signed-off-by: Tanya <[email protected]> * Separated ConnectivityCube class to its own file. Removed unused methods/params/imports. Signed-off-by: Tanya <[email protected]> * Update nca/NetworkConfig/NetworkConfig.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/Resources/NetworkPolicy.py Co-authored-by: Adi Sosnovich <[email protected]> * Added assertions avoiding incorrect comparisons of "src_peers" and "dst_peers" dimensions. Split complex ConnectivityMapQuery.exec method to multiple methods. Signed-off-by: Tanya <[email protected]> * Update nca/CoreDS/Peer.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectionSet.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/CoreDS/ConnectionSet.py Co-authored-by: Adi Sosnovich <[email protected]> * Added shortcut function ConnectivityProperties.make_conn_props_from_dict. Moved BasePeerSet.reset() to run_args. Signed-off-by: Tanya <[email protected]> * Fixed lint errors. Signed-off-by: Tanya <[email protected]> * Code reuse optimization. Added documentation. Removed unused method. Improved ConnectivityProperties.project_on_one_dimension() method. Signed-off-by: Tanya <[email protected]> * Update nca/NetworkConfig/NetworkLayer.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/NetworkConfig/NetworkLayer.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/NetworkConfig/NetworkLayer.py Co-authored-by: Adi Sosnovich <[email protected]> * Update nca/NetworkConfig/NetworkLayer.py Co-authored-by: Adi Sosnovich <[email protected]> * Removed unused functions and imports. Avoid including IpBlocks in non-captured peers. Signed-off-by: Tanya <[email protected]> * Making more accurate default all properties, according to all peers in the current config. Signed-off-by: Tanya <[email protected]> * output_endpoints support. Signed-off-by: Shmulik Froimovich <[email protected]> * support ep modes Signed-off-by: Shmulik Froimovich <[email protected]> * Documentation added Signed-off-by: Shmulik Froimovich <[email protected]> * minor fix Signed-off-by: Shmulik Froimovich <[email protected]> * use Expl' functions only when activated by user Signed-off-by: Shmulik Froimovich <[email protected]> * some lintings Signed-off-by: Shmulik Froimovich <[email protected]> * supporting scheme files Signed-off-by: Shmulik Froimovich <[email protected]> * support only text output formats and '[',']' in peer names Signed-off-by: Shmulik Froimovich <[email protected]> * linting Signed-off-by: Shmulik Froimovich <[email protected]> * small fixes Signed-off-by: Shmulik Froimovich <[email protected]> * small fixes Signed-off-by: Shmulik Froimovich <[email protected]> * adding beautifulsoup4 to requirements Signed-off-by: Shmulik Froimovich <[email protected]> * small fix Signed-off-by: Shmulik Froimovich <[email protected]> * use TCP conns when istio layer is present Signed-off-by: Shmulik Froimovich <[email protected]> * some fixes and improvements. Signed-off-by: Shmulik Froimovich <[email protected]> * extra line Signed-off-by: Shmulik Froimovich <[email protected]> * adding test Signed-off-by: Shmulik Froimovich <[email protected]> * adding test Signed-off-by: Shmulik Froimovich <[email protected]> * changed 'run_all_tests' so opt parameter can be override by tests Signed-off-by: Shmulik Froimovich <[email protected]> * added expected results Signed-off-by: Shmulik Froimovich <[email protected]> * parsing xml without the need of lxml Signed-off-by: Shmulik Froimovich <[email protected]> * parsing xml without the need of lxml Signed-off-by: Shmulik Froimovich <[email protected]> * update expected results (for new xml parser) Signed-off-by: Shmulik Froimovich <[email protected]> * update expected results and make it deterministic Signed-off-by: Shmulik Froimovich <[email protected]> * some fixes and new tests Signed-off-by: Shmulik Froimovich <[email protected]> * temp support in setting peers from the peer container, till fix will be merged to master Signed-off-by: Shmulik Froimovich <[email protected]> * new expected results Signed-off-by: Shmulik Froimovich <[email protected]> * new expected results Signed-off-by: Shmulik Froimovich <[email protected]> * manually adding tests Signed-off-by: Shmulik Froimovich <[email protected]> * manually updating test times Signed-off-by: Shmulik Froimovich <[email protected]> * fixed some testing env agnostic issues Signed-off-by: Shmulik Froimovich <[email protected]> * better handling of ipblocks and some small fixes. Signed-off-by: Shmulik Froimovich <[email protected]> * removing explanation for ipblocks Signed-off-by: Shmulik Froimovich <[email protected]> * adding policies in one place Signed-off-by: Shmulik Froimovich <[email protected]> * adding some documentations and test Signed-off-by: Shmulik Froimovich <[email protected]> * updated expected results Signed-off-by: Shmulik Froimovich <[email protected]> * fixed run_all_tests path problem Signed-off-by: Shmulik Froimovich <[email protected]> * duplicate line removed Signed-off-by: Shmulik Froimovich <[email protected]> * updated testes and some docs. Signed-off-by: Shmulik Froimovich <[email protected]> * little better explanation Signed-off-by: Shmulik Froimovich <[email protected]> * some small fixes Signed-off-by: Shmulik Froimovich <[email protected]> * base_name added Signed-off-by: Shmulik Froimovich <[email protected]> * better documentation for using IP-blocks Signed-off-by: Shmulik Froimovich <[email protected]> * added a ip-block test Signed-off-by: Shmulik Froimovich <[email protected]> * base_name check Signed-off-by: Shmulik Froimovich <[email protected]> * base_name check Signed-off-by: Shmulik Froimovich <[email protected]> * base_name and sync Signed-off-by: Shmulik Froimovich <[email protected]> --------- Signed-off-by: Tanya <[email protected]> Signed-off-by: Shmulik Froimovich <[email protected]> Co-authored-by: Tanya <[email protected]> Co-authored-by: Adi Sosnovich <[email protected]>
- Loading branch information
3 people
authored
Jul 5, 2023
1 parent
36661e4
commit 7335253