Skip to content

Commit

Permalink
after merge with expl branch
Browse files Browse the repository at this point in the history 
Signed-off-by: Shmulik Froimovich <[email protected]>
  • Loading branch information
@shmfr
shmfr committed Jul 6, 2023
2 parents cef2195 + 2a6e68f commit f4f7394
Show file tree
Hide file tree
Showing 29 changed files with 2,694 additions and 23,198 deletions.
6 changes: 3 additions & 3 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -102,10 +102,10 @@ The arguments to `--resource_list` and to `--base_resource_list` should be one o
Choose endpoints type in output (pods/deployments).\
*default:* deployments
- `--explain`\
A pair of node names to explain the policies affecting their connection or lack of it. Relevant only for connectivity query.\
A pair of node names (comma separated) to explain the policies affecting their connection or lack of it. Relevant only for connectivity query.\
Connections including IP-Blocks will show only the configurations of the node in that connection (since, IP-Blocks does
not have configurations).\
e.g. default/deployment-A1,default/deployment-B1.
not have configurations). IP-Blocks should be places in CIDR format as seen in the query results (run the connectivity query first, to see the nodes there).\
e.g. default/pod-A1,default/deployment-B1.
- `--print_ipv6`\
include IPv6 range in the query results even when the policies of the config do not contain any IPv6 addresses.

Expand Down
18 changes: 9 additions & 9 deletions docs/SchemeFileFormat.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -78,15 +78,15 @@ For example: `my_set/prod_ns/deny_all_policy`. If there are multiple policies na
#### <a name="outputconfig"></a>Output Configuration object
The supported entries in the outputConfiguration object are as follows:

| Field | Description | Value |
|------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------|
| outputFormat | Output format specification. | string [ txt / yaml / csv / md / dot / jpg/ txt_no_fw_rules] |
| outputPath | A file path to redirect output into. | string |
| outputEndpoints | Choose endpoints type in output. | string [ pods / deployments ] |
| subset | A dict object with the defined subset elements to display in the output | [subset](#subset) object |
| fullExplanation | Choose if to print all counterexamples causing the query result in the output | bool |
| excludeIPv6Range | If the policies of the config do not contain any IPv6 addresses, do not include IPv6 range in the query results | bool [default: True] |
| explain | A pair of node names to explain the policies affecting their connection or lack of it. Relevant only for connectivity query. Connections including IP-Blocks will show only the configurations of the node in that connection (since, IP-Blocks does not have configurations). | string [ ns/node1,ns/node2 ] |
| Field | Description | Value |
|------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------|
| outputFormat | Output format specification. | string [ txt / yaml / csv / md / dot / jpg/ txt_no_fw_rules] |
| outputPath | A file path to redirect output into. | string |
| outputEndpoints | Choose endpoints type in output. | string [ pods / deployments ] |
| subset | A dict object with the defined subset elements to display in the output | [subset](#subset) object |
| fullExplanation | Choose if to print all counterexamples causing the query result in the output | bool |
| excludeIPv6Range | If the policies of the config do not contain any IPv6 addresses, do not include IPv6 range in the query results | bool [default: True] |
| explain | A pair of node names (comma separated) to explain the policies affecting their connection or lack of it. Relevant only for connectivityMap query. Connections including IP-Blocks will show only the configurations of the node in that connection (since, IP-Blocks does not have configurations). IP-Blocks should be places in CIDR format as seen in the query results (run the connectivity query first, to see the nodes there). | string [ ns/node1,ns/node2 ] |

#### <a name="subset"></a>Subset object
The supported entries in the subset object are as follows:
Expand Down
11 changes: 11 additions & 0 deletions nca/CoreDS/ConnectivityProperties.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -445,6 +445,17 @@ def get_all_conns_props_per_config_peers(peer_container):
return ConnectivityProperties.make_conn_props_from_dict({"src_peers": all_peers_and_ips_and_dns,
"dst_peers": all_peers_and_ips_and_dns})

@staticmethod
def get_all_conns_props_per_domain_peers():
"""
Return all possible between-peers connections.
This is a compact way to represent all peers connections, but it is an over-approximation also containing
IpBlock->IpBlock connections. Those redundant connections will be eventually filtered out.
"""
src_peers = BasePeerSet().get_peer_set_by_indices(DimensionsManager().get_dimension_domain_by_name("src_peers"))
dst_peers = BasePeerSet().get_peer_set_by_indices(DimensionsManager().get_dimension_domain_by_name("dst_peers"))
return ConnectivityProperties.make_conn_props_from_dict({"src_peers": src_peers, "dst_peers": dst_peers})

@staticmethod
def make_empty_props():
"""
Expand Down
13 changes: 11 additions & 2 deletions nca/NetworkConfig/NetworkConfig.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -289,8 +289,17 @@ def allowed_connections_optimized(self, layer_name=None):
# all possible connections involving hostEndpoints
conn_hep = ConnectivityProperties.make_conn_props_from_dict({"src_peers": host_eps}) | \
ConnectivityProperties.make_conn_props_from_dict({"dst_peers": host_eps})
conns_res = OptimizedPolicyConnections()
conns_res.all_allowed_conns = ConnectivityProperties.get_all_conns_props_per_config_peers(self.peer_container)
if host_eps and NetworkLayerName.K8s_Calico not in self.policies_container.layers:
# maintain K8s_Calico layer as active if peer container has hostEndpoint
conns_res = \
self.policies_container.layers.empty_layer_allowed_connections_optimized(self.peer_container,
NetworkLayerName.K8s_Calico)
conns_res.allowed_conns &= conn_hep
conns_res.denied_conns &= conn_hep
conns_res.pass_conns &= conn_hep
else:
conns_res = OptimizedPolicyConnections()
conns_res.all_allowed_conns = ConnectivityProperties.get_all_conns_props_per_config_peers(self.peer_container)
for layer, layer_obj in self.policies_container.layers.items():
conns_per_layer = layer_obj.allowed_connections_optimized(self.peer_container)
# only K8s_Calico layer handles host_eps
Expand Down
5 changes: 4 additions & 1 deletion nca/NetworkConfig/NetworkConfigQuery.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -806,11 +806,14 @@ def compute_connectivity_output_optimized(self):
"dst_peers": opt_peers_to_compare})
base_peers_num = len(opt_peers_to_compare)
subset_peers = self.compute_subset(opt_peers_to_compare)
all_peers = subset_peers
if len(subset_peers) != base_peers_num:
# remove connections where both of src_peers and dst_peers are out of the subset
subset_conns = ConnectivityProperties.make_conn_props_from_dict({"src_peers": subset_peers}) | \
ConnectivityProperties.make_conn_props_from_dict({"dst_peers": subset_peers})
all_conns_opt &= subset_conns
src_peers, dst_peers = ExplTracker().extract_peers(all_conns_opt)
all_peers = src_peers | dst_peers
all_conns_opt = self.config.filter_conns_by_peer_types(all_conns_opt, opt_peers_to_compare)
expl_conns = all_conns_opt
if self.config.policies_container.layers.does_contain_layer(NetworkLayerName.Istio):
Expand All @@ -820,7 +823,7 @@ def compute_connectivity_output_optimized(self):
else:
output_res, opt_fw_rules = self.get_props_output_full(all_conns_opt, opt_peers_to_compare)
if ExplTracker().is_active():
ExplTracker().set_connections_and_peers(expl_conns, subset_peers)
ExplTracker().set_connections_and_peers(expl_conns, all_peers)
return output_res, opt_fw_rules, opt_fw_rules_tcp, opt_fw_rules_non_tcp

def exec(self):
Expand Down
2 changes: 1 addition & 1 deletion nca/NetworkConfig/NetworkConfigQueryRunner.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ def _run_query_for_each_config(self):
expl_out = ''
if ExplTracker().is_active() and self.output_configuration.explain and \
ExplTracker().is_output_format_supported(self.output_configuration.outputFormat):
expl_out = ExplTracker().explain(self.output_configuration.explain.split(','))
expl_out = '\n\nExplainability results:\n'+ExplTracker().explain(self.output_configuration.explain.split(','))
numerical_result, output, num_not_executed = query_result.compute_final_results(self.output_configuration.outputFormat)
return numerical_result, output + expl_out, num_not_executed

Expand Down
10 changes: 3 additions & 7 deletions nca/Parsers/CalicoPolicyYamlParser.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -558,7 +558,7 @@ def _parse_xgress_rule(self, rule, is_ingress, policy_selected_eps, is_profile):
if not dst_res_pods and policy_selected_eps and (not is_ingress or not is_profile):
self.warning('Rule selects no destination endpoints', rule)

return CalicoPolicyRule(src_res_pods, dst_res_pods, connections, action), conn_props
return CalicoPolicyRule(src_res_pods, dst_res_pods, connections, action, conn_props)

def _verify_named_ports(self, rule, rule_eps, rule_conns):
"""
Expand Down Expand Up @@ -698,16 +698,12 @@ def parse_policy(self):
self.syntax_error('order is not allowed in the spec of a Profile', policy_spec)

for ingress_rule in policy_spec.get('ingress', []):
rule, optimized_props = self._parse_xgress_rule(ingress_rule, True, res_policy.selected_peers, is_profile)
rule = self._parse_xgress_rule(ingress_rule, True, res_policy.selected_peers, is_profile)
res_policy.add_ingress_rule(rule)
if self.optimized_run != 'false':
res_policy.update_and_add_optimized_props(optimized_props, rule.action, True)

for egress_rule in policy_spec.get('egress', []):
rule, optimized_props = self._parse_xgress_rule(egress_rule, False, res_policy.selected_peers, is_profile)
rule = self._parse_xgress_rule(egress_rule, False, res_policy.selected_peers, is_profile)
res_policy.add_egress_rule(rule)
if self.optimized_run != 'false':
res_policy.update_and_add_optimized_props(optimized_props, rule.action, False)

self._apply_extra_labels(policy_spec, is_profile, res_policy.name)
res_policy.findings = self.warning_msgs
Expand Down
32 changes: 12 additions & 20 deletions nca/Parsers/GenericIngressLikeYamlParser.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from nca.CoreDS.DimensionsManager import DimensionsManager
from nca.CoreDS.Peer import PeerSet
from nca.CoreDS.PortSet import PortSet
from nca.CoreDS.ProtocolSet import ProtocolSet
from nca.CoreDS.ConnectivityProperties import ConnectivityProperties
from nca.CoreDS.ConnectionSet import ConnectionSet
from nca.Resources.IngressPolicy import IngressPolicyRule
Expand Down Expand Up @@ -56,40 +57,31 @@ def parse_regex_host_value(self, regex_value, rule):
regex_value = regex_value.replace("*", allowed_chars + '*')
return MinDFA.dfa_from_regex(regex_value)

def _make_allow_rules(self, allowed_conns):
"""
Make deny rules from the given connections
:param ConnectivityProperties allowed_conns: the given allowed connections
:return: the list of deny IngressPolicyRules
"""
return self._make_rules_from_conns(allowed_conns)

@staticmethod
def _make_rules_from_conns(conn_props):
def _make_allow_rules(conn_props, src_peers):
"""
Make IngressPolicyRules from the given connections
:param ConnectivityProperties conn_props: the given connections
:param PeerSet src_peers: the source peers to add to optimized props
:return: the list of IngressPolicyRules
"""
assert not conn_props.named_ports
assert not conn_props.excluded_named_ports
peers_to_conns = {}
res = []
# extract peers dimension from cubes
assert not conn_props.is_active_dimension("src_peers")
# extract dst_peers dimension from cubes
tcp_protocol = ProtocolSet.get_protocol_set_with_single_protocol('TCP')
for cube in conn_props:
conn_cube = conn_props.get_connectivity_cube(cube)
dst_peer_set = conn_cube["dst_peers"]
conn_cube.unset_dim("dst_peers")
new_props = ConnectivityProperties.make_conn_props(conn_cube)
new_conn_cube = conn_cube.copy()
conn_cube.update({"src_peers": src_peers, "protocols": tcp_protocol})
rule_opt_props = ConnectivityProperties.make_conn_props(conn_cube)
dst_peer_set = new_conn_cube["dst_peers"]
new_conn_cube.unset_dim("dst_peers")
new_props = ConnectivityProperties.make_conn_props(new_conn_cube)
new_conns = ConnectionSet()
new_conns.add_connections('TCP', new_props)
if dst_peer_set in peers_to_conns:
peers_to_conns[dst_peer_set] |= new_conns # optimize conns for the same peers
else:
peers_to_conns[dst_peer_set] = new_conns
for peer_set, conns in peers_to_conns.items():
res.append(IngressPolicyRule(peer_set, conns))
res.append(IngressPolicyRule(dst_peer_set, new_conns, rule_opt_props))
return res

@staticmethod
Expand Down
7 changes: 1 addition & 6 deletions nca/Parsers/IngressPolicyYamlParser.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@
from nca.CoreDS.PortSet import PortSet
from nca.CoreDS.ConnectivityCube import ConnectivityCube
from nca.CoreDS.ConnectivityProperties import ConnectivityProperties
from nca.CoreDS.ProtocolSet import ProtocolSet
from nca.Resources.IngressPolicy import IngressPolicy
from nca.Resources.NetworkPolicy import NetworkPolicy
from .GenericIngressLikeYamlParser import GenericIngressLikeYamlParser
Expand Down Expand Up @@ -287,10 +286,6 @@ def parse_policy(self):
# allowed_conns = none means that services referenced by this Ingress policy are not found,
# then no connections rules to add (Ingress policy has no effect)
if allowed_conns:
res_policy.add_rules(self._make_allow_rules(allowed_conns))
protocols = ProtocolSet.get_protocol_set_with_single_protocol('TCP')
allowed_conns &= ConnectivityProperties.make_conn_props_from_dict({"protocols": protocols,
"src_peers": res_policy.selected_peers})
res_policy.add_optimized_allow_props(allowed_conns, False)
res_policy.add_rules(self._make_allow_rules(allowed_conns, res_policy.selected_peers))
res_policy.findings = self.warning_msgs
return res_policy
Loading

0 comments on commit f4f7394

Please sign in to comment.