Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Command Status and Report Generations #1854

Open
wants to merge 90 commits into
base: AV-104274
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
ca41392
Command Status and Report Generations
sagarpsalvi Nov 25, 2021
a42054c
Minor Changes
sagarpsalvi Nov 25, 2021
94caa28
Added prefix and migration of profile converter
agarwalme Nov 25, 2021
67da3e6
Command Status and Reports for Http Application Profile
sagarpsalvi Nov 29, 2021
5dce37e
report generation for network profile
agarwalme Nov 30, 2021
2c4d3b6
changed command_status.yaml
agarwalme Dec 1, 2021
15fe81e
Minor Changes
sagarpsalvi Dec 1, 2021
902b790
converted config to ansible playbook , added progress bar and made so…
agarwalme Dec 3, 2021
6709a45
minor changes
agarwalme Dec 3, 2021
5ce775d
Unit testing for profile and pool conversion
agarwalme Dec 7, 2021
51c1aad
#AV-103663: NSX-T to ALB migration pep8 fixes
Dec 8, 2021
85888e5
#AV-103663: NSX-T to ALB migration fix excel path
Dec 8, 2021
a9f9c6b
unit-testing for monitor convertion
agarwalme Dec 8, 2021
2023439
unit-test for monitor conversion
agarwalme Dec 8, 2021
562c69d
Code Reafactor & Log Generation
sagarpsalvi Dec 8, 2021
8dd89a8
#AV-103663: NSX-T to ALB migration support for migrate to option
Dec 8, 2021
f6401d0
Minor changes
sagarpsalvi Dec 8, 2021
b7804f1
Minor change in Testcases
sagarpsalvi Dec 8, 2021
baf10d9
#AV-103663: NSX-T to ALB migration fixed ansible migration issues
Dec 8, 2021
78b721a
Added Logs
sagarpsalvi Dec 8, 2021
905b62f
added test cases and minor changes in nsxt converter code
agarwalme Dec 14, 2021
90f3578
implemented object merging for HealthMonitor , Application and networ…
agarwalme Dec 16, 2021
7e962b4
Added Avi object for merged objects of application profile
agarwalme Dec 21, 2021
eb3ce29
Added the migration of vs
agarwalme Dec 22, 2021
4b88c16
Migration of vs and test cases
agarwalme Dec 24, 2021
371b7c9
added code for overall skipped setting in vs
agarwalme Jan 12, 2022
f432ea8
SSL Cert Changes
sagarpsalvi Jan 5, 2022
2404109
Final Changes SSL merges with AVI
sagarpsalvi Jan 12, 2022
bef7caf
Minor changes
sagarpsalvi Jan 16, 2022
0527c0c
Added test cases for vs and have done some modification in earlier te…
agarwalme Jan 17, 2022
83140b5
minor modifications
agarwalme Jan 18, 2022
e6fdac9
minor change in ssl and persistence file
agarwalme Jan 19, 2022
f6e3c8e
added test cases for ssl and persistance profile
agarwalme Jan 19, 2022
97e5492
added vs_filter
agarwalme Jan 20, 2022
f0cb717
Minor Fixes Demo code to upload alb config to ALB
sagarpsalvi Jan 21, 2022
b0a57a9
Upload Avi Config
sagarpsalvi Jan 25, 2022
9f2e52c
SSL Configuration Changes
sagarpsalvi Jan 25, 2022
9d1ca23
added code for patch , config , minor changes in not applicable list .
agarwalme Feb 9, 2022
e500fb6
added code for preserve client ip
agarwalme Feb 10, 2022
dd54b9f
HttpPolicySet Rules
sagarpsalvi Feb 11, 2022
31f367c
Code for PKI profile , minor changes in not applicable list
agarwalme Feb 15, 2022
f8aa9df
Rule Integration added skipped settings and na setings
sagarpsalvi Feb 15, 2022
d99b45b
Minor Changes
sagarpsalvi Feb 15, 2022
d9740e6
added code for changes in avi mapping
agarwalme Feb 17, 2022
3e83d23
completed the changes as per the changes in avi mapping sheet , added…
agarwalme Feb 23, 2022
2a6c58e
added code for nsx-cleanup
agarwalme Feb 28, 2022
d8bcffe
Policy Rules Minor changes
sagarpsalvi Mar 2, 2022
f2d2159
added code for new mapping
agarwalme Mar 16, 2022
8151c5f
minor changes
agarwalme Mar 16, 2022
7233b7d
minor changes in conversion_util file
agarwalme Mar 16, 2022
7af35ae
Added code for new mapping for ssl , monitor and persistence and modi…
agarwalme Mar 22, 2022
fed0c82
Added Discovery tool code
agarwalme Apr 1, 2022
4b6d9df
Discovery tool code changes
agarwalme Apr 1, 2022
00b1850
Code for vlan pool config , added cloud ref , discovery tool script
agarwalme Apr 18, 2022
116c286
Added pool group configuration
agarwalme Apr 28, 2022
03d72b4
Minor changes
agarwalme May 19, 2022
da27119
Done Changes in the skipped and indirect setting , vs filter
agarwalme May 20, 2022
fe23308
Done changes in the code regarding ids and name
agarwalme May 24, 2022
89d2f7b
Added filters for nsx cleanup code
agarwalme May 25, 2022
53d63a9
Added migration code for
shardullatkar May 30, 2022
9f25c96
Added support for vs_state and traffic_enabled
Rohan-sss1 May 31, 2022
5dfe57e
Merge branch 'SS-AV-104274' of https://github.com/vmware/alb-sdk into…
Rohan-sss1 May 31, 2022
c849b5a
Fixed minor changes
Rohan-sss1 May 31, 2022
9023a25
Fixed tenant and cloud ref , fixed the vs filter issues
agarwalme May 31, 2022
6bb7dd9
fixed cloud ref
agarwalme Jun 1, 2022
399953a
minor fixes
agarwalme Jun 1, 2022
ff866af
minor fixed for traffic
Rohan-sss1 Jun 1, 2022
8c8dc45
minor fix regarding cloud
agarwalme Jun 1, 2022
25c416a
fix in nsxt_util
agarwalme Jun 1, 2022
944de19
fixes regarding cloud , pools
agarwalme Jun 2, 2022
f5fe462
fixed overlay cloud
agarwalme Jun 2, 2022
f1a997a
minor changes
agarwalme Jun 2, 2022
d0980e0
changes for vs filter
agarwalme Jun 2, 2022
ab7b3d0
Added certificate handling, fixed script parameters, rollback and cut…
shardullatkar Jun 3, 2022
d79b05b
Fixed known hosts issue in paramiko, fixed cert logger
shardullatkar Jun 3, 2022
4fe2dcf
added placement network script to vs vip
agarwalme Jun 3, 2022
20a1ad8
Added fixes for following
shardullatkar Jun 5, 2022
19e875a
fixed test cases and fixes vs script
agarwalme Jun 5, 2022
9e7ad3e
Updated example
Rohan-sss1 Jun 6, 2022
a598604
Fixed following points
shardullatkar Jun 7, 2022
2be3c26
Added tenant ref
agarwalme Jun 7, 2022
91f583f
Fixed paramiko ssh connect error, removed duplicate parameter
shardullatkar Jun 7, 2022
5259648
changes for tenant ref in ssl profile
agarwalme Jun 14, 2022
9bc854c
Added script for pool sharing and pool group
agarwalme Jun 15, 2022
4e6ef56
minor changes in pool sharing script
agarwalme Jun 15, 2022
0b68dc0
change in network ref
agarwalme Jun 16, 2022
12262ea
added passwords in environment variables , added ssh root password
agarwalme Jun 16, 2022
ee83560
minor changes
agarwalme Jun 16, 2022
d297d4c
changes in console data format
agarwalme Jun 17, 2022
48d776f
minor changes in pool and vs
agarwalme Jun 20, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
202 changes: 202 additions & 0 deletions python/avi/migrationtools/nsxt_converter/alb_converter.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
import argparse
import json
import os
from pprint import PrettyPrinter

SUPPORTED_ALB_OBJECTS = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this information to yaml file and read from file

"AlertScriptConfig",
"AnalyticsProfile",
"ApplicationPersistenceProfile",
"ApplicationProfile",
"Authprofiles",
"AutoScaleLaunchConfig",
"DnsPolicy",
"ErrorPageBody",
"ErrorPageProfile",
"HardwareSecurityModuleGroup",
"HealthMonitor",
"HTTPPolicySet",
"IpAddrGroup",
"L4PolicySet",
"NetworkProfile",
"NetworkSecurityPolicy",
"ObjectAccessPolicy",
"PingAccessAgent",
"PKIProfile",
"Pool",
"PoolGroup",
"PoolGroupDeploymentPolicy",
"PriorityLabels",
"ProtocolParser",
"SecurityPolicy",
"ServerAutoScalePolicy",
"CertificateManagementProfile",
"SSLKeyAndCertificate",
"SSLProfile",
"SSOPolicy",
"StringGroup",
"TrafficCloneProfile",
"VirtualService",
"VSDataScriptSet",
"VsVip",
"WafCRS",
"WafPolicy",
"WafPolicyPSMGroup",
"WafProfile",
"Webhook"
]

albObjectType = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this to command status yaml

"alertscriptconfig" : "alb-alert-script-configs",
"analyticsprofile" : "alb-analytics-profiles",
"applicationpersistenceprofile" : "alb-application-persistence-profiles",
"applicationprofile" : "alb-application-profiles",
"authprofile" : "alb-auth-profiles",
"autoscalelaunchconfig" : "alb-auto-scale-launch-configs",
"certificatemanagementprofile" : "alb-certificate-management-profiles",
"dnspolicy" : "alb-dns-policies",
"errorpagebody" : "alb-error-page-bodies",
"errorpageprofile" : "alb-error-page-profiles",
"httppolicyset" : "alb-http-policy-sets",
"hardwaresecuritymodulegroup" : "alb-hardware-security-module-groups",
"healthmonitor" : "alb-health-monitors",
"ipaddrgroup" : "alb-ip-addr-groups",
"l4policyset" : "alb-l4-policy-sets",
"networkprofile" : "alb-network-profiles",
"networksecuritypolicy" : "alb-network-security-policies",
"pkiprofile" : "alb-pki-profiles",
"pingaccessagent" : "alb-ping-access-agents",
"pool" : "alb-pools",
"poolgroup" : "alb-pool-groups",
"poolgroupdeploymentpolicy" : "alb-pool-group-deployment-policies",
"prioritylabels" : "alb-priority-labels",
"protocolparser" : "alb-protocol-parsers",
"sslkeyandcertificate" : "alb-ssl-key-and-certificates",
"sslprofile" : "alb-ssl-profiles",
"ssopolicy" : "alb-sso-policies",
"securitypolicy" : "alb-security-policies",
"serverautoscalepolicy" : "alb-server-auto-scale-policies",
"stringgroup" : "alb-string-groups",
"trafficcloneprofile" : "alb-traffic-clone-profiles",
"vsdatascriptset" : "alb-vs-data-script-sets",
"virtualservice" : "alb-virtual-services",
"vsvip" : "alb-vs-vips",
"wafcrs" : "alb-waf-crs",
"wafpolicy" : "alb-waf-policies",
"wafpolicypsmgroup" : "alb-waf-policy-psm-groups",
"wafprofile" : "alb-waf-profiles",
"objectaccesspolicy" : "alb-object-access-policies",
"webhook" : "alb-webhooks"
}

# SUPPORTED_ALB_OBJECTS = ['VirtualService']

NOT_APPLICABLE = ['url', 'uuid', 'tenant_ref']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move to command status yaml

REPLACE_KEYS = ['name', 'cloud_ref', 'vrf_ref', 'vrf_context_ref', 'tier1_lr']

class ALBConverter:
def __init__(self, args):
'''

'''
self.avi_config_file = args.avi_config_file
self.output_file_path = args.output_file_path if args.output_file_path \
else 'output-alb'

def convert(self):
if not os.path.exists(self.output_file_path):
os.mkdir(self.output_file_path)
output_dir = os.path.normpath(self.output_file_path)
input_path = output_dir + os.path.sep + os.path.sep + "input-alb"
if not os.path.exists(input_path):
os.makedirs(input_path)
input_config = input_path + os.path.sep + "config.json"
with open(self.avi_config_file, "r") as read_file:
avi_config = json.load(read_file)

with open(input_config, "w", encoding='utf-8') as text_file:
json.dump(avi_config, text_file, indent=4)

alb_config = self.convert_to_alb(avi_config)
pp = PrettyPrinter()
pp.pprint(alb_config)

output_path = output_dir + os.path.sep + os.path.sep + "output-alb"
if not os.path.exists(output_path):
os.makedirs(output_path)
output_config = output_path + os.path.sep + "alb_config.json"
with open(output_config, "w", encoding='utf-8') as text_file:
json.dump(alb_config, text_file, indent=4)

def convert_to_alb(self, avi_config):
alb_config = dict()
for key in avi_config.keys():
if key in SUPPORTED_ALB_OBJECTS:
config = []
supported_obj = avi_config[key]
for obj in supported_obj:
data = self.recursive_items(obj, {})
config.append(data)
alb_config[key] = config
return alb_config

def recursive_items(self, obj, data):
for k, v in obj.items():
if k not in NOT_APPLICABLE:
if type(v) is dict:
data[k] = self.recursive_items(v, {})
elif not k.endswith("_refs") and type(v) is list:
tmp = []
for iter_over_obj in v:
if type(iter_over_obj) is dict:
tmp.append(self.recursive_items(iter_over_obj, {}))
else:
tmp.append(iter_over_obj)
data[k] = tmp
elif k in REPLACE_KEYS:
if k == REPLACE_KEYS[0]:
data['display_name'] = v
data['id'] = v
if k == REPLACE_KEYS[1]:
data['cloud_name'] = v.split("name=")[1]
if k == REPLACE_KEYS[2]:
data['vrf_name'] = v.split("name=")[1].split("&")[0]
if k == REPLACE_KEYS[3]:
data['vrf_context_name'] = v.split("name=")[1].split("&")[0]
if k == REPLACE_KEYS[4]:
data["tier1_path"] = v
elif k.endswith("_ref"):
if v.split('/')[2] not in albObjectType.keys():
continue
object_type = albObjectType[v.split('/')[2]]
data[k.replace("_ref", "_path")] = "/infra/" + object_type + "/" + v.split("name=")[1]
elif k.endswith("_refs"):
list_of_paths = [ "/infra/" + albObjectType[data.split('/')[2]] + "/" + data.split("name=")[1] for data in v ]
data[k.replace("_refs", "_paths")] = list_of_paths
else:
data[k] = v
return data


if __name__ == "__main__":
HELP_STR = '''
Converts AVI Config to ALB config.
Example to convert AVI config file to ALB config json:
alb_converter.py -f tmp_exported_config.json
'''

parser = argparse.ArgumentParser(
formatter_class=argparse.RawTextHelpFormatter,
description=(HELP_STR))

# Added args for baseline profile json file
parser.add_argument('-f', '--avi_config_file',
help='absolute path for avi config file')
parser.add_argument('-o', '--output_file_path',
help='Folder path for output files to be created in',
)

args = parser.parse_args()

alb_converter = ALBConverter(args)
alb_converter.convert()
Empty file modified python/avi/migrationtools/nsxt_converter/base_client.py
100755 → 100644
Empty file.
111 changes: 111 additions & 0 deletions python/avi/migrationtools/nsxt_converter/command_status.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# This fie defines the list of F5 commands.
# Supported commands defines that these commands are supported by F5 tool
# Not Applicable commands are not applicable in AVI. Those commands are F5 specific commands
# Indirect commands are supported indirectly in AVI
# Not Supported commands are not supported by this tool yet

# Attributes for f5 version 11
VERSION_11:
Monitor_Supported_Types:
- "LBHttpMonitorProfile"
- "LBHttpsMonitorProfile"
- "LBIcmpMonitorProfile"
- "LBTcpMonitorProfile"
- "LBUdpMonitorProfile"

Monitor_Supported_Attributes:
- "monitor_port"
- "interval"
- "timeout"
- "rise_count"
- "fall_count"
- "display_name"
- "resource_type"
- "id"

Monitor_http_attr:
- "request_url"
- "request_method"
- "request_version"
- "request_body"
- "response_status_codes"
- "response_body"

Monitor_https_attr:
- "request_url"
- "request_method"
- "request_version"
- "request_body"
- "response_status_codes"
- "response_body"
- "server_ssl_profile_binding"

Monitor_tcp_attr:
- "send"
- "receive"

Monitor_udp_attr:
- "send"
- "receive"

Monitor_ping_attr:

Pool_supported_attr:
- "display_name"
- "members"
- "algorithm"
- "member_group"
- "active_monitor_paths"
- "snat_translation"
- "tcp_multiplexing_enabled"
- "tcp_multiplexing_number"
- "min_active_members"
- "resource_type"
- "id"

Pool_supported_attr_convert_servers_config:
- "display_name"
- "ip_address"
- "port"
- "admin_state"
- "weight"
- "max_concurrent_connections"

Pool_supported_attr_convert_member_group:
- "group_path"
- "port"

Application_Http_Profile_supported_attr:
- "x_forwarded_for"
- "http_redirect_to_https"
- "idle_timeout"
- "request_header_size"
- "request_body_size"
- "description"
- "display_name"
- "resource_type"
- "server_keep_alive"
- "id"

Network_Profile_supported_attr:
- "display_name"
- "resource_type"
- "id"
- "idle_timeout"


Common_Na_List:
- "path"
- "relative_path"
- "parent_path"
- "unique_id"
- "realization_id"
- "marked_for_delete"
- "overridden"
- "_create_time"
- "_create_user"
- "_last_modified_time"
- "_last_modified_user"
- "_system_owned"
- "_protection"
- "_revision"
Loading