diff --git a/docs/images/initial_setup_of_velos_system_controllers/image24.png b/docs/images/initial_setup_of_velos_system_controllers/image24.png index 66cf651..82bbada 100644 Binary files a/docs/images/initial_setup_of_velos_system_controllers/image24.png and b/docs/images/initial_setup_of_velos_system_controllers/image24.png differ diff --git a/docs/images/initial_setup_of_velos_system_controllers/image25.png b/docs/images/initial_setup_of_velos_system_controllers/image25.png index 4d1482c..6103535 100644 Binary files a/docs/images/initial_setup_of_velos_system_controllers/image25.png and b/docs/images/initial_setup_of_velos_system_controllers/image25.png differ diff --git a/docs/images/initial_setup_of_velos_system_controllers/image30.png b/docs/images/initial_setup_of_velos_system_controllers/image30.png index 8ab9e7b..b0ae21f 100644 Binary files a/docs/images/initial_setup_of_velos_system_controllers/image30.png and b/docs/images/initial_setup_of_velos_system_controllers/image30.png differ diff --git a/docs/images/initial_setup_of_velos_system_controllers/image33.png b/docs/images/initial_setup_of_velos_system_controllers/image33.png index fd20ab7..45aa1fb 100644 Binary files a/docs/images/initial_setup_of_velos_system_controllers/image33.png and b/docs/images/initial_setup_of_velos_system_controllers/image33.png differ diff --git a/docs/images/initial_setup_of_velos_system_controllers/image34.png b/docs/images/initial_setup_of_velos_system_controllers/image34.png index b6dc797..83eefbc 100644 Binary files a/docs/images/initial_setup_of_velos_system_controllers/image34.png and b/docs/images/initial_setup_of_velos_system_controllers/image34.png differ diff --git a/docs/images/initial_setup_of_velos_system_controllers/image5.png b/docs/images/initial_setup_of_velos_system_controllers/image5.png index d08f58a..cc0178a 100644 Binary files a/docs/images/initial_setup_of_velos_system_controllers/image5.png and b/docs/images/initial_setup_of_velos_system_controllers/image5.png differ diff --git a/docs/images/initial_setup_of_velos_system_controllers/image6.png b/docs/images/initial_setup_of_velos_system_controllers/image6.png index df57797..20ece4f 100644 Binary files a/docs/images/initial_setup_of_velos_system_controllers/image6.png and b/docs/images/initial_setup_of_velos_system_controllers/image6.png differ diff --git a/docs/images/initial_setup_of_velos_system_controllers/image6sub.png b/docs/images/initial_setup_of_velos_system_controllers/image6sub.png new file mode 100644 index 0000000..b0f2a3a Binary files /dev/null and b/docs/images/initial_setup_of_velos_system_controllers/image6sub.png differ diff --git a/docs/images/initial_setup_of_velos_system_controllers/snmp.png b/docs/images/initial_setup_of_velos_system_controllers/snmp.png new file mode 100644 index 0000000..fc51d0c Binary files /dev/null and b/docs/images/initial_setup_of_velos_system_controllers/snmp.png differ diff --git a/docs/images/velos_high_availability/image3.png b/docs/images/velos_high_availability/image3.png index 5989d16..eec237b 100644 Binary files a/docs/images/velos_high_availability/image3.png and b/docs/images/velos_high_availability/image3.png differ diff --git a/docs/images/velos_high_availability/image3a.png b/docs/images/velos_high_availability/image3a.png new file mode 100644 index 0000000..fa29a68 Binary files /dev/null and b/docs/images/velos_high_availability/image3a.png differ diff --git a/docs/images/velos_high_availability/image3b.png b/docs/images/velos_high_availability/image3b.png new file mode 100644 index 0000000..62a4c11 Binary files /dev/null and b/docs/images/velos_high_availability/image3b.png differ diff --git a/docs/images/velos_high_availability/image4.png b/docs/images/velos_high_availability/image4.png index 04bfffe..f25ea55 100644 Binary files a/docs/images/velos_high_availability/image4.png and b/docs/images/velos_high_availability/image4.png differ diff --git a/docs/initial_setup_of_velos_chassis_partitions.rst b/docs/initial_setup_of_velos_chassis_partitions.rst index c10614d..54f05ca 100644 --- a/docs/initial_setup_of_velos_chassis_partitions.rst +++ b/docs/initial_setup_of_velos_chassis_partitions.rst @@ -2,7 +2,7 @@ Initial Setup Within the Chassis Partition ========================================== -Chassis partitions are completely separate management entities that are managed outside of the system controllers but are still considered part of the F5OS platform layer. If you have properly setup a chassis partition and assigned an out-of-band management IP address, you will be able to access it via its own F5OS-C CLI, webUI, and API. The chassis partitions only have a single out-of-band IP address and the system is resilient in that the single IP address should be reachable as long as one blade in the partition is active. There is no way to access the chassis partition via in-band networks, as the chassis partition does not have an option for in-band IP interfaces. At first login you will be prompted to change the default chassis partition password. +Chassis partitions are completely separate management entities that are managed outside of the system controllers but are still considered part of the F5OS platform layer. If you have properly setup a chassis partition and assigned an out-of-band management IP address, you will be able to access it via its own F5OS-C CLI, webUI, and API. The chassis partitions only have a single out-of-band IP address, and the system is resilient in that the single IP address should be reachable as long as one blade in the partition is active. There is no way to access the chassis partition via in-band networks, as the chassis partition does not have an option for in-band IP interfaces. At first login you will be prompted to change the default chassis partition password. .. image:: images/initial_setup_of_velos_chassis_partitions/image1.png :align: center @@ -12,7 +12,7 @@ Chassis partitions are completely separate management entities that are managed Chassis Partition Dashboard --------------------------- -The chassis partition **Dashboard** will provide a visual system summary of the partition and which slots are assigned to it. It will also list the total number of vCPUs available for multitenancy and how many are currently in use. There is also a tenant overview showing a quick summary of tenant status and basic parameters. Lastly it will display **System Summary** stats under the tab of that name.. +The chassis partition **Dashboard** will provide a visual system summary of the partition and which slots are assigned to it. It will also list the total number of vCPUs available for multitenancy and how many are currently in use. There is also a tenant overview showing a quick summary of tenant status and basic parameters. Lastly it will display **System Summary** stats under the tab of that name. .. image:: images/initial_setup_of_velos_chassis_partitions/image2.png :align: center @@ -24,7 +24,7 @@ If you click on the **Network** tab, then each front panel port will be displaye :align: center :scale: 70% -If you click on the **CPU** tab, then CPU utlization across different time periods will be displyed for each of the blades in this chassis partition. +If you click on the **CPU** tab, then CPU utilization across different time periods will be displayed for each of the blades in this chassis partition. .. image:: images/initial_setup_of_velos_chassis_partitions/image4.png :align: center @@ -57,7 +57,7 @@ Before configuring any interfaces, VLANs, or LAG’s you’ll need to configure :width: 45% -**NOTE: Both ports on the BX110 blade must be configured in the same mode in current F5OS versions i.e. both ports must be configured for 100Gb, or 40Gb, or 4 x 25GB, or 4 x 10Gb. You cannot mix different port group settings on the same blade currently. A future release may provide more granular options.** +**NOTE: Both ports on the BX110 blade must be configured in the same mode in current F5OS versions i.e., both ports must be configured for 100Gb, or 40Gb, or 4 x 25GB, or 4 x 10Gb. You cannot mix different port group settings on the same blade currently. A future release may provide more granular options.** Configuring PortGroups from the webUI ----------------------------------- @@ -68,7 +68,7 @@ To configure Portgroups go to **Network Settings > Port Groups** in the chassis :align: center :scale: 70% -If you do make a change the blade will be forced to reboot to load a new bitstream image into the FPGA. +If you make a change the blade will be forced to reboot to load a new bitstream image into the FPGA. Configuring PortGroups from the CLI ----------------------------------- @@ -242,7 +242,7 @@ To list the current portgroup configuration issue the following API call: Network Settings -> Interfaces ------------------------------ -Interface numbering will vary depending on the current portgroup configuration. Interfaces will always be numbered by **/**. The number of ports on a blade will change depending on if the portgroup is configured as bundled or unbundled. If the ports are bundled then ports will be **1/1.0** and **1/2.0** for slot 1, and **2/1.0** and **2/2.0** for slot 2 etc…. If ports are unbundled then the port numbering will be **1/1.1, 1/1.2, 1/1.3, and 1/1.4** for the first physical port and **1/2.1, 1/2.2, 1/2.3, and 1/2.4** for the second physical port. A breakout cable is require to separate the different ports. Even when multiple chassis partitions are used, the port numbering will stay consistent starting with the blade number. Below is an example of port numbering with all bundled interfaces. +Interface numbering will vary depending on the current portgroup configuration. Interfaces will always be numbered by **/**. The number of ports on a blade will change depending on if the portgroup is configured as bundled or unbundled. If the ports are bundled then ports will be **1/1.0** and **1/2.0** for slot 1, and **2/1.0** and **2/2.0** for slot 2 etc…. If ports are unbundled then the port numbering will be **1/1.1, 1/1.2, 1/1.3, and 1/1.4** for the first physical port and **1/2.1, 1/2.2, 1/2.3, and 1/2.4** for the second physical port. A breakout cable is required to separate the different ports. Even when multiple chassis partitions are used, the port numbering will stay consistent starting with the blade number. Below is an example of port numbering with all bundled interfaces. .. image:: images/initial_setup_of_velos_chassis_partitions/image9.png :align: center @@ -263,7 +263,7 @@ Within the chassis partition webUI the physical ports of all blades within that :align: center :scale: 70% -You can click on any interface to view its settings or edit them. You can currently change the interface State via the webUI or the **Native VLAN** (untagged) and **Trunk VLANs** (tagged) as long as the interface is not part of a LAG. If the interface is part of the LAG then the VLAN configuration is done within the LAG rather than the interface. +You can click on any interface to view its settings or edit them. You can currently change the interface State via the webUI or the **Native VLAN** (untagged) and **Trunk VLANs** (tagged) as long as the interface is not part of a LAG. If the interface is part of the LAG, then the VLAN configuration is done within the LAG rather than the interface. .. image:: images/initial_setup_of_velos_chassis_partitions/image12.png :align: center @@ -638,7 +638,7 @@ The **show vlan-listeners** command will show the current state: Configuring VLANs from the API ------------------------------ -To configure VLANs use the following API command and JSON body. This will configure mul;tiple VLANs along with their VLAN ID’s. After the VLANs are created you will be able to assign then to either interfaces or LAGs. +To configure VLANs use the following API command and JSON body. This will configure multiple VLANs along with their VLAN IDs. After the VLANs are created you will be able to assign then to either interfaces or LAGs. .. code-block:: bash @@ -824,13 +824,15 @@ Link Aggregation Groups (LAGs) can be configured in the chassis partition webUI You can add a new LAG or edit an existing one. For **LAG Type** the options are **LACP** or **STATIC**. If you choose LACP, then you have additional options for **LACP Interval** (**SLOW** or **FAST**) and **LACP Mode** (**ACTIVE** or **PASSIVE**). LACP best practices should follow previous BIG-IP examples as outlined in the links below. Note in BIG-IP the term **Trunks** is used in place of **LAG** which is used in the F5OS layer in VELOS: -https://support.f5.com/csp/article/K1689 +`K1689: Overview of trunks on BIG-IP platforms `_ + +`K13142: Configure the BIG-IP system to interface with Cisco virtual PortChannel `_ -https://support.f5.com/csp/article/K13142 The following solution article provides guidance for setting up VELOS LAG interfaces and LACP with Cisco Nexus 9000 series switches: -https://support.f5.com/csp/article/K33431212 + +`K33431212: Configure LAGs with LACP between the VELOS system and Cisco Nexus 9000 series switches `_ Once you have configured the LAG Type and LACP options, you can add any physical interfaces within this chassis partition to be part of a LAG. Note you cannot add physical interfaces that reside in other chassis partitions as they are completely isolated from each other. Finally, you can configure the **Native VLAN** (for untagged VLAN), and what **Trunked VLANs** (tagged) you’d like to add to this LAG interface. @@ -1212,7 +1214,7 @@ The final step is adding LACP configuration for each LAG: } } -To view the final LAG configuration via the API use the following API call: +To view the final LAG configuration via the API, use the following API call: .. code-block:: bash diff --git a/docs/initial_setup_of_velos_system_controllers.rst b/docs/initial_setup_of_velos_system_controllers.rst index 6714977..f174c66 100644 --- a/docs/initial_setup_of_velos_system_controllers.rst +++ b/docs/initial_setup_of_velos_system_controllers.rst @@ -5,7 +5,7 @@ Initial Setup of VELOS System Controllers System Controller Setup ======================= -Connect a console or terminal server to each of the system controllers console ports. Login as admin/admin (you’ll be prompted to change the password), and access the F5OS CLI. F5OS utilizes **ConfD** for configuration management and will be a familiar navigation experience if you have used it on other products. The CLI supports **** command completion and online help via **?**, and is easy to navigate. There are **show** commands to display current configurations and status, and a **config** mode to alter current configuration. +Connect a console or terminal server to each of the system controllers console ports. Login as admin/admin (you’ll be prompted to change the password) and access the F5OS CLI. F5OS utilizes **ConfD** for configuration management and will be a familiar navigation experience if you have used it on other products. The CLI supports **** command completion and online help via **?**, and is easy to navigate. There are **show** commands to display current configurations and status, and a **config** mode to alter current configuration. Once logged in, you can display the current running configuration by issuing the command **show running-config**. @@ -82,9 +82,9 @@ VELOS systems ship with a default internal RFC6598 address space of 100.64.0.0/1 system network state active-chassis-id 1 syscon-2-active# -This address range never leaves the inside of the chassis, and will not interfere with any communication outside the VELOS chassis. There can however be address collisions if a device trying to manage VELOS via the out-of-band management port falls within this range, or an external management device or service falls within this range, and communicates with VELOS over its out-of-band networking. This may result in VELOS not able to communicate with those devices. +This address range never leaves the inside of the chassis and will not interfere with any communication outside the VELOS chassis. There can however be address collisions if a device trying to manage VELOS via the out-of-band management port falls within this range, or an external management device or service falls within this range and communicates with VELOS over its out-of-band networking. This may result in VELOS not able to communicate with those devices. -Some examples would be; any client trying to access the F5OS-C platform layer (system controller or chassis partition), or tenant out-of-band interfaces to reach its’ CLI, webUI, or API. Other examples would be; external services such as SNMP, DNS, NTP, SNMP, Authentication that have addresses that fall within the RFC6598 address space. You may experience connectivity problems with these types of clients/services, if there is any address space overlap. Note, this does not affect the data plane / in-band interfaces, it only affects communication to the out-of-band interfaces. +Some examples would be any client trying to access the F5OS-C platform layer (system controller or chassis partition), or tenant out-of-band interfaces to reach its’ CLI, webUI, or API. Other examples would be external services such as SNMP, DNS, NTP, SNMP, Authentication that have addresses that fall within the RFC6598 address space. You may experience connectivity problems with these types of clients/services, if there is any address space overlap. Note, this does not affect the data plane / in-band interfaces, it only affects communication to the out-of-band interfaces. If there is the potential for conflict with external devices that fall within this range that need to communicate with F5OS, then there are options to change the configured **network-range-type** to one of sixteen different blocks within the RFC1918 address space. Changing this will require a complete chassis power-cycle, rebooting is not sufficient, a **commit** must occur before the reboot. Please consult with F5 prior to making any changes to the internal addresses. @@ -136,7 +136,7 @@ If changing to one of the RFC1918 address spaces, you will need to choose from o IP Address Assignment & Routing ------------------------------- -Each system controller requires its own unique IP address, and a floating IP address also needs to be configured. The floating IP address will follow the primary system controller. The IP addresses can be statically defined, or acquired via DHCP. In addition to the IP addresses, a default route and subnet mask/prefix length is defined. For the initial 1.1.x versions of F5OS-C only IPv4 IP addresses were supported on the out-of-band interfaces of the system controllers. IPv6 and dual stack IPv4/v6 support requires F5OS-C be running 1.2.x or later. Note the tenants themselves support IPv4/IPv6 management reagrdless of F5OS version. +Each system controller requires its own unique IP address, and a floating IP address also needs to be configured. The floating IP address will follow the primary system controller. The IP addresses can be statically defined or acquired via DHCP. In addition to the IP addresses, a default route and subnet mask/prefix length is defined. For the initial 1.1.x versions of F5OS-C only IPv4 IP addresses were supported on the out-of-band interfaces of the system controllers. IPv6 and dual stack IPv4/v6 support requires F5OS-C be running 1.2.x or later. Note the tenants themselves support IPv4/IPv6 management regardless of F5OS version. .. image:: images/initial_setup_of_velos_system_controllers/image1.png :align: center @@ -152,13 +152,13 @@ Once logged in, you will configure the static IP addresses (unless DHCP is prefe syscon-2-active(config)# system mgmt-ip config ipv4 prefix-length 24 syscon-2-active(config)# system mgmt-ip config ipv4 gateway 10.255.0.1 -In order to make these changes active, you must commit the changes. No configuration changes are executed until the commit command is issued. +To make these changes active, you must commit the changes. No configuration changes are executed until the commit command is issued. .. code-block:: bash syscon-2-active(config)# commit -Now that the out-of-band addresses and routing are configured, you can attempt to access the system controller webUI via the floating IP address that has been defined. The floating IP address should always be used to onitor and configure the system as it will always follow the active controller. Using the static IP addresses is best saved for diagnosing a problem, as the secondary controller will not allow config changes to be made, and monitoring may be limited when in stnady state. After logging into the floating IP address, you should see a screen similar to the one below, and you can verify your management interface settings. +Now that the out-of-band addresses and routing are configured, you can attempt to access the system controller webUI via the floating IP address that has been defined. The floating IP address should always be used to monitor and configure the system as it will always follow the active controller. Using the static IP addresses is best saved for diagnosing a problem, as the secondary controller will not allow config changes to be made, and monitoring may be limited when in standby state. After logging into the floating IP address, you should see a screen like the one below, and you can verify your management interface settings. .. image:: images/initial_setup_of_velos_system_controllers/image2.png :align: center @@ -168,7 +168,7 @@ Now that the out-of-band addresses and routing are configured, you can attempt t Interface Aggregation for System Controllers (Optional) ------------------------------------------------------- -As seen in previous diagrams, each system controller has its own independent out-of-band 10Gb ethernet connection. These can run independently of each other, and should be connected to the same layer2 VLAN so that the floating IP address can move from primary to standby in the event of a failure. You may optionally configure these two interfaces into a single Link Aggregation Group (LAG) for added resiliency, which is recommended. This would allow direct access to either static IP address on the system controllers in the event one link should fail. Below is a depiction of each system controllers out-of-band management interface bonded together in a single LAG: +As seen in previous diagrams, each system controller has its own independent out-of-band 10Gb ethernet connection. These can run independently of each other and should be connected to the same layer2 VLAN so that the floating IP address can move from primary to standby in the event of a failure. You may optionally configure these two interfaces into a single Link Aggregation Group (LAG) for added resiliency, which is recommended. This would allow direct access to either static IP address on the system controllers in the event one link should fail. Below is a depiction of each system controllers out-of-band management interface bonded together in a single LAG: .. image:: images/initial_setup_of_velos_system_controllers/image3.png :align: center @@ -176,7 +176,7 @@ As seen in previous diagrams, each system controller has its own independent out To enable this feature, you would need to enable link aggregation on the system controllers via the CLI, webUI or API, and then make changes to your upstream layer2 switching infrastructure to ensure the two ports are put into the same LAG. To configure the management ports of both system controllers to run in a LAG, configure as follows: -On the active controller, create a managment LACP interface: +On the active controller, create a management LACP interface: .. code-block:: bash @@ -250,16 +250,39 @@ Configuring Network Time Protocol is highly recommended, so that the VELOS syste :align: center :scale: 70% -It’s also a good idea to have the VELOS system controllers send logs to an external syslog server. This can be configured in the **System Settings > Log Settings** screen. Here, you can configure remote servers, the logging facility, and severity levels. You can also configure logging subsystem levels individually. The remote logging severity level will override the component logging levels if they are configured higher, but only for logs sent remotely. Local logging levels will always follow the component levels that are configured here. +It’s also a good idea to have the VELOS system controllers send logs to an external syslog server. This can be configured in the **System Settings > Log Settings** screen. Here, you can configure remote servers, the logging facility, and severity levels. + .. image:: images/initial_setup_of_velos_system_controllers/image6.png :align: center :scale: 70% +You can also configure logging subsystem levels individually. The remote logging severity level will override the component logging levels if they are configured higher, but only for logs sent remotely. Local logging levels will always follow the component levels that are configured here. + +.. image:: images/initial_setup_of_velos_system_controllers/image6sub.png + :align: center + :scale: 70% + + Configure System Settings From the API --------------------------------------- -If you would prefer to automate the setup of the VELOS chassis, there are F5OS-C API calls for all of the examples above. VELOS supports token based authentication for the F5OS API's. You can send a standard API call with user/password based authentication (basic auth), and then store the token for subsequent API calls. The X-Auth-Token has a lifetime of fifteen minutes and can be renewed a maximum of five times before you need to authenticate again using basic auth. The renewal period begins at the ten-minute point, where the API will start sending a new X-Auth-Token in the response for the next five minutes. If your API calls fail to start using the new token by the 15-minute point, API calls will start returning 401 Not Authorized. All the API examples in this guide were generated using the Postman utility. Below is an example of using password based authentication to the system controller floating IP address. Be sure to go to the **Auth** tab and set the *Type** to **Basic Auth**, and enter the username and password to log into your system controller. +If you would prefer to automate the setup of the VELOS chassis, there are F5OS-C API calls for all the examples above. F5OS supports token-based authentication for the F5OS API’s. You may send API calls to either port 8888 or port 443. The URI path will change slightly depending on which TCP port you choose to use. For API calls sent to port 443, the initial path will be /api, while API calls to port 888 will start with /restconf. F5OS also listens on port 80 and will redirect to TCP port 443. + +Example of API call using port 8888. + +.. code-block:: bash + + https://{{rseries_rseries_appliance1_ip}}:8888/restconf/data/openconfig-system:system/aaa + + +Example of API call using port 443. Replace /restconf with /api. + +.. code-block:: bash + + https://{{rseries_rseries_appliance1_ip}}/api/data/openconfig-system:system/aaa + +You can send a standard API call with user/password-based authentication (basic auth), and then store the token for subsequent API calls. The X-Auth-Token has a lifetime of fifteen minutes and can be renewed a maximum of five times before you need to authenticate again using basic auth. The renewal period begins at the ten-minute point, where the API will start sending a new X-Auth-Token in the response for the next five minutes. If your API calls fail to start using the new token by the 15-minute point, API calls will start returning 401 Not Authorized. All the API examples in this guide were generated using the Postman utility. Below is an example of using password-based authentication to the system controller floating IP address. Be sure to go to the **Auth** tab and set the *Type** to **Basic Auth** and enter the username and password to log into your system controller. .. image:: images/initial_setup_of_velos_system_controllers/image6a.png :align: center @@ -291,7 +314,7 @@ Once the variable is stored with the auth token, it can be used instead of using :align: center :scale: 70% -You must also add some required headers to any API calls sent to F5OS. It is important to include the header **Content-Type** **application/yang-data+json** and the Token header **X-Auth-Token** with a value of **{{x-auth-token_chassis1_system_controller}}**. The variable and header will change depening on the destination of the API call. It can be send to a second chassis, or to chassis partitions within the chassis depending on the parameter being configured. +You must also add some required headers to any API calls sent to F5OS. It is important to include the header **Content-Type** **application/yang-data+json** and the Token header **X-Auth-Token** with a value of **{{x-auth-token_chassis1_system_controller}}**. The variable and header will change depending on the destination of the API call. It can be sent to a second chassis, or to chassis partitions within the chassis depending on the parameter being configured. .. image:: images/initial_setup_of_velos_system_controllers/image6e.png :align: center @@ -419,7 +442,7 @@ To set a Remote Logging destination, use the following API call: Licensing the VELOS Chassis --------------------------- -Licensing for the VELOS system is handled at the chassis level. This is similar to how VIPRION licensing is implemented, where the system is licensed once, and all subsystems inherit their licensing from the chassis. With VELOS, licensing is applied at the system controller level, and all chassis partitions and tenants will inherit their licenses from the base system. There is no need to procure add-on licenses for MAX SSL/Compression, or for tenancy/vCMP. This is different than VIPRION, where there was an extra charge for virtualization/vCMP, and in some cases for MAX SSL/Compression. For VELOS, these are included in the base license at no extra cost. VELOS does not run vCMP, and instead runs tenancy. +Licensing for the VELOS system is handled at the chassis level. This is like how VIPRION licensing is implemented, where the system is licensed once, and all subsystems inherit their licensing from the chassis. With VELOS, licensing is applied at the system controller level, and all chassis partitions and tenants will inherit their licenses from the base system. There is no need to procure add-on licenses for MAX SSL/Compression, or for tenancy/vCMP. This is different than VIPRION, where there was an extra charge for virtualization/vCMP, and in some cases for MAX SSL/Compression. For VELOS, these are included in the base license at no extra cost. VELOS does not run vCMP, and instead runs tenancy. Licenses can be applied via the F5OS-C CLI, webUI, or API. A base registration key and optional add-on keys are needed, and it follows the same manual or automatic licensing capabilities of other BIG-IP systems. @@ -439,7 +462,7 @@ You can activate and display the current license in the webUI, CLI or API. Apply License via CLI --------------------- -You can license the VELOS system automatically if F5's licesing server is reachable from the VELOS system, and it can resolve the licensing servers name via DNS. If this is not possible, manual licensing may be used. To license the VELOS chassis automatically from the F5OS CLI: +You can license the VELOS system automatically if F5's licensing server is reachable from the VELOS system, and it can resolve the licensing servers name via DNS. If this is not possible, manual licensing may be used. To license the VELOS chassis automatically from the F5OS CLI: .. code-block:: bash @@ -602,16 +625,16 @@ In the body of the API call, add the following JSON payload which references the Chassis Partition Creation -------------------------- -Once the base level networking, licensing, and system settings are defined, the next step is to create the chassis partitions that will be used. The system ships with all 8 slots defined within the **default** chassis partition. If there is no need for more than one chassis partition, you can just utilize the default partition and any blades installed will automatically cluster together. Multiple tenants can be defined within the chassis partition, and they can be restricted to specific vCPUs as well as restricted to a single blade, or be allowed to span across multiple blades. Conceptually this is similar to how vCMP guests are defined on VIPRION, but the underlying technology in F5OS is different. +Once the base level networking, licensing, and system settings are defined, the next step is to create the chassis partitions that will be used. The system ships with all 8 slots defined within the **default** chassis partition. If there is no need for more than one chassis partition, you can just utilize the default partition and any blades installed will automatically cluster together. Multiple tenants can be defined within the chassis partition, and they can be restricted to specific vCPUs as well as restricted to a single blade or be allowed to span across multiple blades. Conceptually this is like how vCMP guests are defined on VIPRION, but the underlying technology in F5OS is different. -If you decide to utilize the default partition you will need to assign an out-of-band management IP address, prefix, and default route so that it can be managed. You must also define what release of F5OS-C software the chassis partition should run. It is recommended you check downloads.f5.com for the latest F5OS-C software, as the version that shipped on the system may not be the latest. Note, this is different than the software that the tenants will actually run. Once the management IP address is assigned, you would then connect directly to that chassis partition to manage its networking, users and authentication, and tenants. +If you decide to utilize the default partition you will need to assign an out-of-band management IP address, prefix, and default route so that it can be managed. You must also define what release of F5OS-C software the chassis partition should run. It is recommended you check downloads.f5.com for the latest F5OS-C software, as the version that shipped on the system may not be the latest. Note, this is different than the software that the tenants will run. Once the management IP address is assigned, you would then connect directly to that chassis partition to manage its networking, users and authentication, and tenants. -You may want to leave the default partition, and create new partitions with names that make sense for your environment. As an example, you may want to create different chassis partitions for different environments like Development and Production, or for different business units, or departments, or different security zones. If you want to utilize other chassis partitions so that you can isolate all the networking for use by different groups, or for specific use cases, you must first edit the default partition and remove any slots that you want to add to other partitions. Once a slot is removed from the default partition an option to create new chassis partitions is enabled, slots that are not currently tied to an existing chassis partition may be added to a new chassis partition. +You may want to leave the default partition and create new partitions with names that make sense for your environment. As an example, you may want to create different chassis partitions for different environments like Development and Production, or for different business units, or departments, or different security zones. If you want to utilize other chassis partitions so that you can isolate all the networking for use by different groups, or for specific use cases, you must first edit the default partition and remove any slots that you want to add to other partitions. Once a slot is removed from the default partition an option to create new chassis partitions is enabled, slots that are not currently tied to an existing chassis partition may be added to a new chassis partition. Creating Chassis Partitions via the webUI ----------------------------------------- -The term **Slot** and **Blade** may be used interchangably, in the F5OS-C configuration interfaces (CLI, webUI, API) the term slot is used as you may want to assign empty slots to a chassis partition. If the term blade were used, this may lead to confusion, as a blade may not be installed yet. In the webUI screen below, note that there is only the default partition, and all 8 slots are assigned to it. There are 3 blades installed in the chassis (in slots 1-3), and the rest show as **Empty**. +The term **Slot** and **Blade** may be used interchangeably, in the F5OS-C configuration interfaces (CLI, webUI, API) the term slot is used as you may want to assign empty slots to a chassis partition. If the term blade were used, this may lead to confusion, as a blade may not be installed yet. In the webUI screen below, note that there is only the default partition, and all 8 slots are assigned to it. There are 3 blades installed in the chassis (in slots 1-3), and the rest show as **Empty**. .. image:: images/initial_setup_of_velos_system_controllers/image12.png :align: center @@ -672,7 +695,7 @@ Once the partitions are started and operational, you can log into each one direc :scale: 70% -Below is an example of the CLI prompting for a new password. You'll then be disconnected, and will have to log in with the new password. +Below is an example of the CLI prompting for a new password. You'll then be disconnected and will have to log in with the new password. .. code-block:: bash @@ -696,7 +719,7 @@ Below is an example of the CLI prompting for a new password. You'll then be disc Creating a Chassis Partition via the CLI ---------------------------------------- -Before creating a chassis partition via the CLI, you must first ensure that there are available slots in order to create a partition. You can issue the command **show running-config slots** to see what slots are available. If all the slots are assigned to a partition like **default**, then you’ll need to move some of the slots to the partition **none** before they can be added to a new partition. The **none** chassis partition is a special hidden partition that is used to temporarily remove slots from an existing chassis partition. In the webUI the none partition is hidden and this process is handled automatically. In the CLI and API you must first move slots to the none partition, before they are eligible to be added to a new partition. +Before creating a chassis partition via the CLI, you must first ensure that there are available slots to create a partition. You can issue the command **show running-config slots** to see what slots are available. If all the slots are assigned to a partition like **default**, then you’ll need to move some of the slots to the partition **none** before they can be added to a new partition. The **none** chassis partition is a special hidden partition that is used to temporarily remove slots from an existing chassis partition. In the webUI the none partition is hidden and this process is handled automatically. In the CLI and API you must first move slots to the none partition, before they are eligible to be added to a new partition. .. code-block:: bash @@ -746,8 +769,7 @@ In this case we will mimic the flow in the webUI section where there are 3 blade syscon-2-active(config-slot-3)# exit syscon-2-active(config)# commit -Now these slots are available to be assigned to a new partition. Enter config mode and add the partition by defining a name, adding a management IP address, prefix, and gateway. Be sure to commit the change. -Next you'll set the version for the partition to run, and then enable it and commit. Note there are still no slots assigned to the chassis partition. +Now these slots are available to be assigned to a new partition. Enter config mode and add the partition by defining a name, adding a management IP address, prefix, and gateway. Be sure to commit the change. Next, you'll set the version for the partition to run, and then enable it and commit. Note there are still no slots assigned to the chassis partition. .. code-block:: bash @@ -1072,7 +1094,7 @@ Before creating any new chassis partitions you should ensure you have the proper } } -Next import the desired image into the system controller floating IP address using the path **images/staging**. You will need to import from a remote HTTPS, SFTP, or SCP server if using the API. There are other options avialble in the GUI where images can be imported or uploaded from a client machine. There is an insecure option if you don’t want to use certificate-based authentication to the remote HTTPS server. +Next import the desired image into the system controller floating IP address using the path **images/staging**. You will need to import from a remote HTTPS, SFTP, or SCP server if using the API. There are other options avialable in the GUI where images can be imported or uploaded from a client machine. There is an insecure option if you don’t want to use certificate-based authentication to the remote HTTPS server. .. code-block:: bash @@ -1093,7 +1115,7 @@ Next import the desired image into the system controller floating IP address usi ] } -You should see confirmation similar to below: +You should see confirmation like the output below: .. code-block:: json @@ -1110,7 +1132,7 @@ You may also check the transfer status via the API: POST https://{{velos_chassis1_system_controller_ip}}:8888/restconf/data/f5-utils-file-transfer:file/transfer-status -You will see a response similar to the one below showing status: +You will see a response similar like the output below showing status: .. code-block:: json @@ -1121,7 +1143,7 @@ You will see a response similar to the one below showing status: } -Before creating a chassis partition via the API, you must first ensure that there are available slots in order to create a partition. You can issue API commands to see what slots are available. If all the slots are assigned to a partition like **default**, then you’ll need to move some of the slots to the partition **none** before they can be added to a new partition. The **none** chassis partition is a special hidden partition that is used to temporarily remove slots from an existing chassis partition. In the webUI the none partition is hidden and this process is handled automatically. In the CLI and API you must first move slots to the none partition, before they are eligible to be added to a new partition. +Before creating a chassis partition via the API, you must first ensure that there are available slots to create a partition. You can issue API commands to see what slots are available. If all the slots are assigned to a partition like **default**, then you’ll need to move some of the slots to the partition **none** before they can be added to a new partition. The **none** chassis partition is a special hidden partition that is used to temporarily remove slots from an existing chassis partition. In the webUI the none partition is hidden and this process is handled automatically. In the CLI and API you must first move slots to the none partition, before they are eligible to be added to a new partition. The system ships with all slots configured in the default chassis partition. Before you can create a new chassis partition, you must remove any slots from the default partition so that they are eligible to be added to a new chassis partition. To view the current assignment of slots to partitions use the following API command: @@ -1275,7 +1297,7 @@ Finally, the chassis partition **Production** containing slots 1 & 2 will be ena "enabled": true } -Next, a chassis partition called **Devlopment** will be created. It will be assigned an out-of-band management IP address, mask and gateway, along with an F5OS ISO version that must be loaded before the partition can be created. +Next, a chassis partition called **Development** will be created. It will be assigned an out-of-band management IP address, mask and gateway, along with an F5OS ISO version that must be loaded before the partition can be created. .. code-block:: bash @@ -1354,7 +1376,7 @@ The chassis partitions will have a default username/password of admin/admin. Whe ] } -Repeat the same proicess for the chassis partition Development: +Repeat the same process for the chassis partition Development: .. code-block:: bash @@ -1538,7 +1560,7 @@ Once the chassis partitions have been created you can query their status using t System Controller Configuration Options ======================================= -Once the minimum parameters have been setup, you can go back and review or edit various settings for the system controllers. Below is a review of the current webUI options that are available with the system controller F5OS configuration. Other options are available within the F5OS chassis partition interfaces, and are covered in a later section. +Once the minimum parameters have been setup, you can go back and review or edit various settings for the system controllers. Below is a review of the current webUI options that are available with the system controller F5OS configuration. Other options are available within the F5OS chassis partition interfaces and are covered in a later section. ----------------------------------------- Network Settings -> Management Interfaces @@ -1576,7 +1598,7 @@ Each chassis partition will require an F5OS-C software release to be specified w Software Management -> Controller Images ---------------------------------------- -System controllers also run a unique F5OS-C software version and have a seprate ISO from the chassis partitions. Both system controllers will need to run the same SW version. You can upload or import new F5OS-C controller images via the **Software Management > Controller Images** webUI screen. +System controllers also run a unique F5OS-C software version and have a separate ISO from the chassis partitions. Both system controllers will need to run the same SW version. You can upload or import new F5OS-C controller images via the **Software Management > Controller Images** webUI screen. .. image:: images/initial_setup_of_velos_system_controllers/image25.png :align: center @@ -1586,7 +1608,7 @@ System controllers also run a unique F5OS-C software version and have a seprate System Settings -> Alarms & Events ---------------------------------- -Alarms and Events can be viewed via the **System Settings > Alarms & Events** webUI page. You may optionally choose different severity levels to see more or less events. The **Alarms** section displays events that are currently active, while the Events section displays historical events including thoseß that have cleared. Each event should have different assertions to both **Raise** and **Clear** alarms. +Alarms and Events can be viewed via the **System Settings > Alarms & Events** webUI page. You may optionally choose different severity levels to see more or less events. The **Alarms** section displays events that are currently active, while the Events section displays historical events including those that have cleared. Each event should have different assertions to both **Raise** and **Clear** alarms. .. image:: images/initial_setup_of_velos_system_controllers/image26.png :align: center @@ -1601,11 +1623,11 @@ You may also change timeframe to see historical events, and optionally refresh t System Settings -> Controller Management ---------------------------------------- -System controller status, HA state, and software upgrades are managed via the **System Settings > Controller Management** webUI page. The **High Availability Status** refers to the Kubernetes control plane status which operates in an Active / Standby manner. Only one controller will be active from a kubernetes control plane perspective. This does not reflect the status of the layer2 switch fabric on the controllers which operates in an active/active mode. +System controller status, HA state, and software upgrades are managed via the **System Settings > Controller Management** webUI page. The **High Availability Status** refers to the Kubernetes control plane status which operates in an Active / Standby manner. Only one controller will be active from a Kubernetes control plane perspective. This does not reflect the status of the layer2 switch fabric on the controllers which operates in an active/active mode. -An administrator can failover from one system controller to the other, and also perform software upgrades to the controllers as needed. You may perform a bundled upgrade which combines both the OS and F5 service components, or they can be upgraded independently. An upgrade which includes the **OS**, will be more disruptive timewise vs. an upgrade that only updates the F5 **services**. F5 support would recommend which type of upgrade may be needed for a particular fix, or feature. Ideally F5 expects to have to update the OS less frequently in the long term than the F5 Services. Currently, F5 is recommending upgrades using the full ISO vs. separate OS and service. +An administrator can failover from one system controller to the other, and also perform software upgrades to the controllers as needed. You may perform a bundled upgrade which combines both the OS and F5 service components, or they can be upgraded independently. An upgrade which includes the **OS**, will be more disruptive timewise vs. an upgrade that only updates the F5 **services**. F5 support would recommend which type of upgrade may be needed for a particular fix, or feature. Ideally F5 expects to have to update the OS less frequently in the long term than the F5 Services. Currently, F5 is recommending upgrades using the full ISO vs. separate OS and service upgrades. -**NOTE: The intial v1.1.x F5OS-C versions did not support rolling upgrades for the system controllers. Any upgrade that is initiated will update both controllers in parallel which will result in an outage for the entire chassis. A proper outage window should be planned for any upgrades, and updating the standby chassis first is recommended if possible. Rolling upgrade support for the system controllers was added to the 1.2.x release of F5OS-C. Once the system controllers are starting from a 1.2.x release, rolling upgrade is supported.** +**NOTE: The initial v1.1.x F5OS-C versions did not support rolling upgrades for the system controllers. Any upgrade that is initiated will update both controllers in parallel which will result in an outage for the entire chassis. A proper outage window should be planned for any upgrades, and updating the standby chassis first is recommended if possible. Rolling upgrade support for the system controllers was added to the 1.2.x release of F5OS-C. Once the system controllers are starting from a 1.2.x release, rolling upgrades are supported.** .. image:: images/initial_setup_of_velos_system_controllers/image28.png :align: center @@ -1635,7 +1657,7 @@ Under **System Settings > Log Settings** you may add remote log servers for the System Settings -> File Utilities --------------------------------- -The **System Settings > File Utilities** page allows for importing or exporting specific types of files to and from the system controllers. Logs from the various log directories log can be exported, cores and qkviews can be imported/exported from diags/shared and system controller and chassis partition software images can be imported into import/staging. +The **System Settings > File Utilities** page allows for importing or exporting specific types of files to and from the system controllers. Logs from the various log directories log can be exported, cores and qkviews can be exported from diags/shared and system controller and chassis partition software images can be imported into import/staging. .. image:: images/initial_setup_of_velos_system_controllers/image31.png :align: center @@ -1658,20 +1680,30 @@ Under the **System Settings > Time Settings** page Network Time Protocol servers :scale: 70% ------------------------------------- -System Settings -> Device Certificate +System Settings -> System Security ------------------------------------- -Device certificates and keys can be uploaded via the **Systems Settings > Device Certificates** page. +Access to specific services running on the system controller F5OS layer can be restricted to certain IP addresses and/or subnets via the **Systems Settings > Security** page. The **Allowed IP Addresses** section will allow the admin to restrict who can access the F5OS layer. Here an administrator can also but F5OS into appliance mode, which will disable bash/root access on the controllers. Other security related parameters such as httpd and sshd cipher suites as well as the CLI idle timeout can be set on this page. .. image:: images/initial_setup_of_velos_system_controllers/image34.png :align: center :scale: 70% +------------------------------------- +System Settings -> SNMP Configuration +------------------------------------- + +SNMP **Communities**, **Users**, and **Targets** can be setup on the **System Settings -> SNMP Configuration** page. Here, an admin can enable access for SNMP monitoring of the system through either communities for SNMPv1/v2c, or through users for SNMPv3. In addition, remote SNMP Trap receiver locations can be enabled for alerting. + +.. image:: images/initial_setup_of_velos_system_controllers/snmp.png + :align: center + :scale: 70% + --------------------------------- -System Settings -> System Reports +Diagnostics -> System Reports --------------------------------- -The **System Settings > System Reports** page allows an admin to generate QKViews and optionally upload them to iHealth. +The **Diagnostics > System Reports** page allows an admin to generate QKViews and optionally upload them to iHealth. .. image:: images/initial_setup_of_velos_system_controllers/image35.png :align: center @@ -1723,7 +1755,7 @@ You may backup the confd configuration databases for the system controller via t System Settings -> Licensing ---------------------------- -Licensing for the VELOS system is handled at the chassis level. This is similar to how VIPRION licensing is handled, where the system is licensed once, and all subsystems inherit their licensing from the chassis. With VELOS licensing is applied at the system controller and all chassis partitions and tenants will inherit their licenses from the base system. There is no need to add-on license for MAX SSL/Compression or for tenancy. This is different than VIPRION where there was an extra charge for virtualization/vCMP and in some cases for MAX SSL/Compression. For VELOS these are included in the base license at no extra cost. +Licensing for the VELOS system is handled at the chassis level. This is like how VIPRION licensing is handled, where the system is licensed once, and all subsystems inherit their licensing from the chassis. With VELOS licensing is applied at the system controller and all chassis partitions and tenants will inherit their licenses from the base system. There is no need to add-on license for MAX SSL/Compression or for tenancy. This is different than VIPRION where there was an extra charge for virtualization/vCMP and in some cases for MAX SSL/Compression. For VELOS these are included in the base license at no extra cost. Licenses can be applied via CLI, webUI, or API. A base registration key and optional add-on keys are needed, and it follows the same manual or automatic licensing capabilities of other BIG-IP systems. diff --git a/docs/introduction.rst b/docs/introduction.rst index 57a9f1d..464247a 100644 --- a/docs/introduction.rst +++ b/docs/introduction.rst @@ -12,7 +12,7 @@ VELOS Overview Kubernetes Based Platform Layer ------------------------------- -The major difference between VELOS and VIPRION is the introduction of a new Kubernetes-based platform layer called F5OS that will allow for exciting new capabilities. Luckily customers won’t need to learn Kubernetes in order to manage the new chassis, it will be abstracted from the administrator. who will be able to manage the new platform layer via familiar F5 CLI, webUI, or API interfaces. +The major difference between VELOS and VIPRION is the introduction of a new Kubernetes-based platform layer called F5OS that will allow for exciting new capabilities. Luckily customers won’t need to learn Kubernetes to manage the new chassis, it will be abstracted from the administrator. who will be able to manage the new platform layer via familiar F5 CLI, webUI, or API interfaces. VELOS will continue to provide hardware acceleration and offload capabilities in a similar way that VIPRION did, however more modern FPGA, CPU, and crypto offload capabilities have been introduced. The new F5OS platform layer allows VELOS to run different types of tenants within the same chassis. As an example, VELOS will be able to run: @@ -27,9 +27,9 @@ VELOS will continue to provide hardware acceleration and offload capabilities in -Customers will be able to migrate existing BIG-IP devices, or vCMP guests into a **tenant** running on VELOS. A tenant is conceptually similar to a vCMP guest on the VIPRION platform. Once inside the tenant, the management experience will be similar to the experience on existing BIG-IP platforms. The BIG-IP tenant will be managed just as a vCMP guest is managed today on VIPRION. The administrator can connect directly to the tenant’s webUI, CLI, or API and have the same experience as they have with their existing BIG-IP platforms. +Customers will be able to migrate existing BIG-IP devices, or vCMP guests into a **tenant** running on VELOS. A tenant is conceptually like a vCMP guest on the VIPRION platform. Once inside the tenant, the management experience will be like the experience on existing BIG-IP platforms. The BIG-IP tenant will be managed just as a vCMP guest is managed today on VIPRION. The administrator can connect directly to the tenant’s webUI, CLI, or API and have the same experience as they have with their existing BIG-IP platforms. -BIG-IP Next tenants will be able to be provisioned within the same chassis, which will allow customers to leverage the next generation of BIG-IP software side-by-side with the existing BIG-IP software. The F5OS platfrom layer provides a bridge between the current generation of BIG-IP and the newer generation BIG-IP Next. What will differ from an administrator’s perspective, is the initial setup of the F5OS platform layer. The new F5OS platfrom layer is leverged in both VELOS chassis (F5OS-C), and rSeries appliances (F5OS-A). We’ll look at some additional architecture differences between VELOS and VIPRION, before exploring how to manage, and monitor the new F5OS platform layer. +BIG-IP Next tenants will be able to be provisioned within the same chassis, which will allow customers to leverage the next generation of BIG-IP software side-by-side with the existing BIG-IP software. The F5OS platform layer provides a bridge between the current generation of BIG-IP and the newer generation BIG-IP Next. What will differ from an administrator’s perspective, is the initial setup of the F5OS platform layer. The new F5OS platform layer is leveraged in both VELOS chassis (F5OS-C), and rSeries appliances (F5OS-A). We’ll look at some additional architecture differences between VELOS and VIPRION, before exploring how to manage, and monitor the new F5OS platform layer. --------------------------------- Smaller Footprint, Higher Density @@ -57,7 +57,7 @@ A VIPRION chassis in comparison does not have a centralized switch fabric, and a :align: center :scale: 70% -The system controllers in VELOS are also the central point of management for the entire chassis. VIPRION required a dedicated out-of-band Ethernet management port and console connection for each blade inserted in the chassis. This meant more cabling, layer2 switch ports, and external terminal servers in order to fully manage the VIPRION chassis as seen below: +The system controllers in VELOS are also the central point of management for the entire chassis. VIPRION required a dedicated out-of-band Ethernet management port and console connection for each blade inserted in the chassis. This meant more cabling, layer2 switch ports, and external terminal servers to fully manage the VIPRION chassis as seen below: .. image:: images/velos_introduction/image5.png :align: center @@ -76,7 +76,7 @@ Additionally, the out-of-band Ethernet ports on the system controllers can be bu The Kubernetes Control Plane ---------------------------- -In addition to being the centralized layer2 switch fabric for the entire chassis, the system controllers also host the Kubernetes control plane, that is responsible for provisioning resources/workloads within the chassis. VELOS utilizes an open source distribution of Kubernetes called OpenShift, and specifically uses the OKD project/distribution. This is largely abstracted away from the administrator, as they won’t be configuring or monitoring containers or Kubernetes components. In the future some Kubernetes-like features will start to be exposed, but it will likely be done through the new VELOS F5OS-C CLI, webUI, or API’s. +In addition to being the centralized layer2 switch fabric for the entire chassis, the system controllers also host the Kubernetes control plane, that is responsible for provisioning resources/workloads within the chassis. VELOS utilizes an open-source distribution of Kubernetes called OpenShift, and specifically uses the OKD project/distribution. This is largely abstracted away from the administrator, as they won’t be configuring or monitoring containers or Kubernetes components. In the future some Kubernetes-like features will start to be exposed, but it will likely be done through the new VELOS F5OS-C CLI, webUI, or API’s. A combination of Docker Compose and Kubernetes is used within the F5OS layer. Docker Compose is used to bring up the system controller and chassis partition software stacks, as they need to be fully functional early in the startup process. Then, Kubernetes takes over and is responsible for deploying workloads to the blades. One of the system controllers will be chosen to serve as primary, and the other secondary from a Kubernetes control plane perspective. The central VELOS chassis F5OS API, CLI, and webUI are served up from the primary system controller. The floating IP address will always follow the primary controller so CLI, webUI, and API access should not be prevented due to a controller failure. @@ -84,7 +84,7 @@ A combination of Docker Compose and Kubernetes is used within the F5OS layer. Do :align: center :scale: 40% -The diagram above is somewhat simplified, as it shows a single software stack for the Kubernetes control plane. In reality there are multiple instances that run on the system controllers. There is a software stack for the system controllers themselves which provides F5OS-C CLI, webUI, and API management for the controllers, as well as chassis partition (a grouping of blades) lifecycle management. There is also a unique stack for every chassis partition in the system. This software stack resides on the system controllers, and can fail over from one controller to the other for added redundancy. It provides the F5OS CLI, webUI, and API functions for the chassis partition, as well as support for the networking services such as; stpd, lldpd, lacpd, that get deployed as workloads on the blades. +The diagram above is somewhat simplified, as it shows a single software stack for the Kubernetes control plane. There are multiple instances that run on the system controllers. There is a software stack for the system controllers themselves which provides F5OS-C CLI, webUI, and API management for the controllers, as well as chassis partition (a grouping of blades) lifecycle management. There is also a unique stack for every chassis partition in the system. This software stack resides on the system controllers and can fail over from one controller to the other for added redundancy. It provides the F5OS CLI, webUI, and API functions for the chassis partition, as well as support for the networking services such as stpd, lldpd, lacpd, that get deployed as workloads on the blades. The Kubernetes control plane is responsible for deploying workloads to the blades. This would happen when tenants or **chassis partitions** (see next section) are configured. We won’t get too deep into the Kubernetes architecture, as its not required to manage the VELOS chassis. Know that the Kubernetes platform layer will allow F5 to introduce exciting new features in the future, but F5 will continue to provide abstracted interfaces for ease of management. By leveraging microservices and containers, F5 may be able to introduce new options such as shared multitenancy and dynamic scaling in the future. These are features that were not supported on VIPRION. @@ -92,11 +92,11 @@ The Kubernetes control plane is responsible for deploying workloads to the blade Chassis Partitions ------------------ -Another exciting new feature is the notion of grouping multiple VELOS blades together to form “mini VIPRIONS” within the same VELOS chassis. This will allow for another layer of isolation, in addition to tenancy (similar to vCMP guests) that VIPRION didn’t support. This could be used to separate production from development/test environments, or to provide different security zones for different classes of applications. Within a VELOS chassis, an administrator can group together one or more blades to form a chassis partition. A chassis may contain multiple chassis partitions, and a blade may belong to only one chassis partition at a time. The minimum unit for a chassis partition is one blade, and the maximum is 8 blades within the CX410 chassis. +Another exciting new feature is the notion of grouping multiple VELOS blades together to form “mini VIPRIONS” within the same VELOS chassis. This will allow for another layer of isolation, in addition to tenancy (like vCMP guests) that VIPRION didn’t support. This could be used to separate production from development/test environments, or to provide different security zones for different classes of applications. Within a VELOS chassis, an administrator can group together one or more blades to form a chassis partition. A chassis may contain multiple chassis partitions, and a blade may belong to only one chassis partition at a time. The minimum unit for a chassis partition is one blade, and the maximum is 8 blades within the CX410 chassis. **Note: Chassis partitions are not related to TMOS admin partitions, which are typically used to provide admin separation within a TMOS instance.** -A chassis partition runs its own unique F5OS-C software image, has a unique set of users/authentication, and is accessed via its own webUI, CLI, and API. The chassis partition can be further divided to support multiple BIG-IP tenants. A tenant operates in a similar manner to how vCMP guests operate within the VIPRION chassis. It is assigned dedicated vCPU and memory resources, and is restricted to specific VLANs by the administrator for network connectivity. +A chassis partition runs its own unique F5OS-C software image, has a unique set of users/authentication, and is accessed via its own webUI, CLI, and API. The chassis partition can be further divided to support multiple BIG-IP tenants. A tenant operates in a similar manner to how vCMP guests operate within the VIPRION chassis. It is assigned dedicated vCPU and memory resources and is restricted to specific VLANs by the administrator for network connectivity. Below is an example of a VELOS CX410 chassis; divided into 3 chassis partitions (red, green, and blue). These chassis partitions are completely isolated from each other, and the system controllers ensure no traffic can bleed from one chassis partition to another. Once a chassis partition is created, individual tenants can be deployed, and they will be restricted to only the resources within that chassis partition. @@ -108,7 +108,7 @@ Below is an example of a VELOS CX410 chassis; divided into 3 chassis partitions Tenants ------- -Tenancy is required to deploy any BIG-IP resources. VELOS is a multitenant chassis by default, there is no bare-metal mode, although it can be configured to emulate this mode with a single large tenant. You can configure one big chassis partition and assign all blades in the system to this resource. In fact, there is a “Default” partition that all blades are part of when inserted. You may change the slots assigned to the chassis partition by removing it from default, and assigning to a new or existing chassis partition. A tenant could then be assigned to utilize all CPU and memory across that chassis partition. This would emulate a VIPRION system running “bare metal” where vCMP is not provisioned. +Tenancy is required to deploy any BIG-IP resources. VELOS is a multitenant chassis by default, there is no bare-metal mode, although it can be configured to emulate this mode with a single large tenant. You can configure one big chassis partition and assign all blades in the system to this resource. In fact, there is a “Default” partition that all blades are part of when inserted. You may change the slots assigned to the chassis partition by removing it from default and assigning to a new or existing chassis partition. A tenant could then be assigned to utilize all CPU and memory across that chassis partition. This would emulate a VIPRION system running “bare metal” where vCMP is not provisioned. When configuring HA between two VELOS chassis, there is no HA relationship across chassis at the F5OS-C layer, where the system controllers or chassis partitions are configured. All HA is configured at the tenant level using Device Service Clustering, similar to how HA is configured between vCMP guests in separate VIPRION chassis. diff --git a/docs/monitoring_velos.rst b/docs/monitoring_velos.rst index 3e8987f..e6f9620 100644 --- a/docs/monitoring_velos.rst +++ b/docs/monitoring_velos.rst @@ -9,7 +9,7 @@ Some admins may want CLI commands to monitor, or API calls to query the system, Accessing the F5OS API ====================== -The VELOS platform API’s for the system controllers and chassis partitions can be reached on port 8888. In this document we will use the Postman tool to access VELOS platform layer API’s. You can download the Postman tool at: +The VELOS platform APIs for the system controllers and chassis partitions can be reached on port 8888. In this document we will use the Postman tool to access VELOS platform layer APIs. You can download the Postman tool at: https://www.postman.com/downloads/ @@ -422,7 +422,7 @@ Or: The output of the API call above will be broken out into the following detail: -The beginning of the output highlights any equipment failures or mismatches and whether or not the chassis is NEBS enabled. Next is the current status of the platform memory for this system controller showing available, used, and used-precent. Next are the thermal readings for temperature showing **current**, **average**, **minimum**, & **maximum** readings. +The beginning of the output highlights any equipment failures or mismatches and whether the chassis is NEBS enabled. Next is the current status of the platform memory for this system controller showing available, used, and used-precent. Next are the thermal readings for temperature showing **current**, **average**, **minimum**, & **maximum** readings. .. code-block:: json @@ -787,7 +787,7 @@ Recent system level alerts can be accessed via the API. System Controller Monitoring via CLI ------------------------------------ -To see if the openshift cluster is up and running use the **show cluster** command. You should see status for each installed blade and controller in the **Ready** state. Each section under **Stage Name** should show a **Status** of **Done**. During the bootup process you can monitor the status of the individual stages. The most recent openshift logs are displayed, and you can determine if the chassis is healthy or having issues. +To see if the Openshift cluster is up and running use the **show cluster** command. You should see status for each installed blade and controller in the **Ready** state. Each section under **Stage Name** should show a **Status** of **Done**. During the bootup process you can monitor the status of the individual stages. The most recent Openshift logs are displayed, and you can determine if the chassis is healthy or having issues. .. code-block:: bash @@ -863,7 +863,7 @@ Monitoring the Layer2 Switch Fabric on the System Controllers This section will outline what status should and can be monitored for the Layer2 switch fabric function on the system controllers. Administrators will want to monitor the internal and external interfaces and LAGs for both status and to view stats to understand current utilization. They will be looking to understand what the utilization of each port is and how is traffic balanced between the two switch fabrics on the system controllers. This section will detail what sort of monitoring is currently supported via CLI, webUI, API, and SNMP, and will also detail any altering, logging, or SNMP traps that are available. -Before getting into what monitoring is supported, it is important to understand how things connect together and their labeling. The diagram below provides the internal interface numbering on the system controllers so that an admin can monitor the status and statistics of each interface. This will give them visibility into the traffic distribution across the backplane and dual switch fabrics. Link Aggregation is configured on the blade side of the connection, but not on the system controller side. Note that the blade in slot 1 will have two connections, one to system controller 1 interface **1/3.1** and one to system controller 2 interface **2/3.1**, the numbering follows the same logic for other slots: +Before getting into what monitoring is supported, it is important to understand how things connect and their labeling. The diagram below provides the internal interface numbering on the system controllers so that an admin can monitor the status and statistics of each interface. This will give them visibility into the traffic distribution across the backplane and dual switch fabrics. Link Aggregation is configured on the blade side of the connection, but not on the system controller side. Note that the blade in slot 1 will have two connections, one to system controller 1 interface **1/3.1** and one to system controller 2 interface **2/3.1**, the numbering follows the same logic for other slots: .. image:: images/monitoring_velos/image6.png :align: center @@ -932,7 +932,7 @@ There is a CLI command to monitor all the internal and external ports and LAGs o ethernet state counters out-8021q-frames 0 -The **show lacp** CLI command will show both external LAG interfaces if the management ports are bonded together, and internal LAG’s to each slot. In the output below there are 3 blades installed in slots 1-3. They will be labeled **cplagg_1.**. The **mgmt_aggr** is a name provided by the admin when the LAG for the external management piorts were configured. This name will be different depending on what the admin chooses for a name. +The **show lacp** CLI command will show both external LAG interfaces if the management ports are bonded together, and internal LAG’s to each slot. In the output below there are 3 blades installed in slots 1-3. They will be labeled **cplagg_1.**. The **mgmt_aggr** is a name provided by the admin when the LAG for the external management ports were configured. This name will be different depending on what the admin chooses for a name. .. code-block:: bash @@ -4244,7 +4244,7 @@ The following API command will show all system controller Ethernet interfaces an Link Aggregation Status of System Controllers ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The following API call will list the current status of all backplane LACP interfaces, as well as front panel management port lacp interfaces: +The following API call will list the status of all backplane LACP interfaces, as well as front panel management port lacp interfaces: .. code-block:: bash diff --git a/docs/monitoring_velos_health_status.rst b/docs/monitoring_velos_health_status.rst index 2b6b60b..4b7d7d8 100644 --- a/docs/monitoring_velos_health_status.rst +++ b/docs/monitoring_velos_health_status.rst @@ -7,7 +7,7 @@ VELOS has many components and subsystems which can be monitored via CLI, webUI, Active Alerts ============= -The VELOS system has an alerting system where certain known issues will raise alerts when the occur, and will be be cleared when they are addressed or self-heal. +The VELOS system has an alerting system where certain known issues will raise alerts when they occur and will be cleared when they are addressed or self-heal. ------------------------------ Checking Active Alerts via CLI @@ -67,7 +67,10 @@ To see past events use the command **show system events**. Checking Active Alerts via webUI ------------------------------ -In the system controller webUI you can go to the **System Events > Alarms & Events** page to see if there are any known alerts for the system. The alerting page is focused on **Active** alerts, and not issues that have cleared. If for example the temperature rises beyond an acceptable threshold then a temperature alert will be raised. It will be seen in this page. If the temperature then falls back into a safe range then the alert will be removed. Each of these alerts will also generate a corresponding SNMP Trap. Please see the the VELOS F5OS SNMP Monitoring and Alerting section. +In the system controller webUI you can go to the **System Events > Alarms & Events** page to see if there are any known alerts for the system. The alerting page is focused on **Active** alerts, and not issues that have cleared. If for example the temperature rises beyond an acceptable threshold, then a temperature alert will be raised. It will be seen in this page. If the temperature reading then falls back into a safe range then the alert will be removed. Each of these alerts will also generate a corresponding SNMP Trap. Please see the VELOS F5OS SNMP Monitoring and Alerting section. + +`VELOS F5OS SNMP Monitoring and Alerting `_ + .. image:: images/monitoring_velos_health_status/image1.png :align: center @@ -77,14 +80,14 @@ In the system controller webUI you can go to the **System Events > Alarms & Even Checking Active Alerts via API ------------------------------ -Recent system level alerts can be accessed via the API. Below is an API call to see all the system events that have occured for a particular chassis: +Recent system level alerts can be accessed via the API. Below is an API call to see all the system events that have occurred for a particular chassis: .. code-block:: bash GET https://{{System-Controller-IP}}:8888/restconf/data/openconfig-system:system/f5-event-log:events -Below are examples of events from a chassis in a test lab.Note there are ASSERT and Deasserted messages that clear them. +Below are examples of events from a chassis in a test lab. Note, there are ASSERT and Deasserted messages that clear them. .. code-block:: json @@ -389,16 +392,16 @@ Below are examples of events from a chassis in a test lab.Note there are ASSERT System Health ============= -VELOS also has a very robust **system health** utility where all the various hardware and software subsystems will first provide a high level health status, but then deeper detail can be gained on what is monitored via each subsystem. +VELOS also has a very robust **system health** utility where all the various hardware and software subsystems will first provide a high-level health status, but then deeper detail can be gained on what is monitored via each subsystem. ------------------------------ Checking System Health via CLI ------------------------------ -Below is the full output from a chassis with 3 blades installed. There is a lot of info in the output when unfiltered, but everything is broken into sections, within each section you'll get a high level status of that subsection with the **state name** and **state health**. You can then see all the subcomponenets that bubble up into the higher level health status for that section. As an example you don't need to know what threasholds will trigger an event, the system health will monitor that for you. If any component is out of tolerance, it will change status so that is not OK, and then it bubble up to the higher level status. +Below is the full output from a chassis with 3 blades installed. There is a lot of info in the output when unfiltered, but everything is broken into sections, within each section you'll get a high-level status of that subsection with the **state name** and **state health**. You can then see all the sub-componenets that bubble up into the higher-level health status for that section. As an example, you don't need to know what thresholds will trigger an event, the system health will monitor that for you. If any component is out of tolerance, it will change status so that is not OK, and then it bubbles that status up to the higher level status. -After the full output below some CLI examples of how to filter all this information down into a high level status will be provided. +After the full output below some CLI examples of how to filter all this information down into a high-level status will be provided. .. code-block:: bash @@ -9677,7 +9680,7 @@ After the full output below some CLI examples of how to filter all this informat syscon-1-active# -You can use the built-in filtering in the CLI to just get the top level state health of each susbsystem. This doesn't list the subsystems, but shows all the top level system health status outputs. If one subsystem shows a status other than **ok** you can then adjust the filtering to only display that subsystem. +You can use the built-in filtering in the CLI to just get the top-level state health of each subsystem. This doesn't list the subsystems but shows all the top level system health status outputs. If one subsystem shows a status other than **ok** you can then adjust the filtering to only display that subsystem. use the output modifier after the **show system health** command to **include "state health"**. @@ -10177,7 +10180,7 @@ use the output modifier after the **show system health** command to **include "s state health ok syscon-1-active# -You can also filter using this CLI command for better summaries. You can see that all service are in an **ok** state meaning there are no faults: +You can also filter using this CLI command for better summaries. You can see that all services are in an **ok** state meaning there are no faults: .. code-block:: bash @@ -10242,7 +10245,7 @@ You can also filter using this CLI command for better summaries. You can see tha psu:state:input-unit-off-for-low-input-voltage PSU Input Unit Off For Low Input Voltage ok info 0 2021-09-16T00:16:47Z syscon-1-active# -You can further filter to just get a high level blade status, again note all 3 blades have an **ok** status: +You can further filter to just get a high-level blade status, again note all 3 blades have an **ok** status: .. code-block:: bash @@ -10266,7 +10269,7 @@ Just like the CLI output you can dump the full system health and all its subsyst GET https://{{velos_chassis1_system_controller_ip}}:8888/restconf/data/openconfig-system:system/f5-system-health:health -This provides a very large output of all the subsystems and their subcomponents. Below the large output are examples of how to futher filter/reduce the output via API calls. +This provides a very large output of all the subsystems and their subcomponents. Below the large output are examples of how to further filter/reduce the output via API calls. .. code-block:: json @@ -63554,7 +63557,7 @@ This provides a very large output of all the subsystems and their subcomponents. } } -You can filter the above output in many ways. Below is an example of how to only get the system health data for a specifc blades. In this case blade-1, blade-2, blade-3. This will still result in a lot of data outputted in the response: +You can filter the above output in many ways. Below is an example of how to only get the system health data for a specific blades. In this case blade-1, blade-2, blade-3. This will still result in a lot of data outputted in the response: .. code-block:: bash diff --git a/docs/velos_components.rst b/docs/velos_components.rst index dba8a87..93e08ae 100644 --- a/docs/velos_components.rst +++ b/docs/velos_components.rst @@ -15,7 +15,7 @@ The CX410 is a 4 Rack Unit (RU) chassis, that has eight ¼ width slots that can Rack Mounting the Chassis ========================= -An ideal environment for VELOS, is to use a minimum of a 30” rack. The rackmount kits that ship with the system will support the minimum depth of 30”, and can telescope to longer depths if needed. If the rack depth is less than 30”, then custom rack mount kits will need to be ordered. +An ideal environment for VELOS, is to use a minimum of a 30” rack. The rackmount kits that ship with the system will support the minimum depth of 30” and can telescope to longer depths if needed. If the rack depth is less than 30”, then custom rack mount kits will need to be ordered. .. image:: images/velos_components/image2.png :align: center @@ -50,7 +50,7 @@ The PSU controllers are accessible in the upper left-hand corner of the back of :width: 45% -A DC power version of the CX410 chassis is also avilable, and runs the same system controllers and line cards. +A DC power version of the CX410 chassis is also available and runs the same system controllers and line cards. .. image:: images/velos_components/image8.png :align: center @@ -59,7 +59,7 @@ A DC power version of the CX410 chassis is also avilable, and runs the same syst Fan Tray & Cooling =================== -The VELOS chassis implements front-to-back cooling/airflow, and it is recommended that customers install such that VELOS is not intaking hot airflow from other devices. The fan tray is removable if it needs to be replaced, but should not be removed for long periods of time, as overheating may occur. +The VELOS chassis implements front-to-back cooling/airflow, and it is recommended that customers install such that VELOS is not intaking hot airflow from other devices. The fan tray is removable if it needs to be replaced but should not be removed for long periods of time, as overheating may occur. .. image:: images/velos_components/image9.png :align: center @@ -87,7 +87,7 @@ In the initial release of VELOS most of the LCD functionality has not been enabl SX410 System Controller ======================== -Each CX410 chassis ships with two SX410 system controllers already installed. They are not optional, and are not ordered separately. The system controllers perform two main functions: +Each CX410 chassis ships with two SX410 system controllers already installed. They are not optional and are not ordered separately. The system controllers perform two main functions: * They provide the active backplane connectivity, and layer2 switching to all line cards (BX110) @@ -97,7 +97,7 @@ Each CX410 chassis ships with two SX410 system controllers already installed. Th * They operate in an active/standby manner for these functions -It is recommended that a system always operate with two system controllers for redundancy. If one should fail, the remaining system controller can take over, however backplane capacity will drop from 1.6Tbps to 800Gbps. The Kubernetes control plane will run on the active system controller, and will fail over to the standby if the active should fail. +It is recommended that a system always operate with two system controllers for redundancy. If one should fail, the remaining system controller can take over, however backplane capacity will drop from 1.6Tbps to 800Gbps. The Kubernetes control plane will run on the active system controller and will fail over to the standby if the active should fail. .. image:: images/velos_components/image14.png :align: center @@ -125,7 +125,7 @@ The BX110 blade is a next generation data plane/line card. It has 2 high speed ( :align: center :scale: 60% -The BX110 has 14 physical cores, which are hyperthreaded into 28 vCPUs. Six of the vCPUs are reserved for the F5OS-C platform layer, leaving 22 vCPUs available for multitenancy. Each blade comes with a 1TB SSD drive, and is populated with 128GB of RAM (double the current generation VIPRION B2250). Each BX110 has two Field Programmable Gate Arrays (FPGA's), which provide hardware offload for certain functions and workloads. The Application Traffic Service Engine (ATSE) is the “front panel FPGA”, which does initial classifications and offload, while the VELOS Queuing FPGA (VQF), is the “back panel FPGA” that implements queuing and CoS through the chassis backplane. The CPU complex provides hardware offload for SSL/TLS and compression, similar to how previous generations of BIG-IP (such as iSeries and VIPRION B4450) performed these operations, but with a newer generation of processor. +The BX110 has 14 physical cores, which are hyperthreaded into 28 vCPUs. Six of the vCPUs are reserved for the F5OS-C platform layer, leaving 22 vCPUs available for multitenancy. Each blade comes with a 1TB SSD drive and is populated with 128GB of RAM (double the current generation VIPRION B2250). Each BX110 has two Field Programmable Gate Arrays (FPGA's), which provide hardware offload for certain functions and workloads. The Application Traffic Service Engine (ATSE) is the “front panel FPGA”, which does initial classifications and offload, while the VELOS Queuing FPGA (VQF), is the “back panel FPGA” that implements queuing and CoS through the chassis backplane. The CPU complex provides hardware offload for SSL/TLS and compression, like previous generations of BIG-IP (such as iSeries and VIPRION B4450) performed these operations, but with a newer generation of processor. .. image:: images/velos_components/image18.png :align: center diff --git a/docs/velos_deploying_a_tenant.rst b/docs/velos_deploying_a_tenant.rst index 937def8..5eba304 100644 --- a/docs/velos_deploying_a_tenant.rst +++ b/docs/velos_deploying_a_tenant.rst @@ -7,7 +7,7 @@ Deploying a Tenant Tenant Image Types ------------------ -Tenant images for F5OS are available on downloads.f5.com. VELOS allows different packaging options for tenant images. It will be up to administrators to choose the image that is best suited for their environment. The main differences between the image types will be how much space they can consume on disk, and whether or not they allow in place upgrades. VELOS only supports specific TMOS releases (currently 14.1.4 and later, and 15.1.4 and later). There is no plan to support v16.0, 16.1, or 17.0 tenants, and the next targeted tenant release will be v17.1. Tenant images for VELOS can be found on downloads.f5.com. +Tenant images for F5OS are available on downloads.f5.com. VELOS allows different packaging options for tenant images. It will be up to administrators to choose the image that is best suited for their environment. The main differences between the image types will be how much space they can consume on disk, and whether they allow in place upgrades. VELOS only supports specific TMOS releases (currently 14.1.4 and later, and 15.1.4 and later). There is no plan to support v16.0, 16.1, or 17.0 tenants, and the next targeted tenant release will be v17.1. Tenant images for VELOS can be found on downloads.f5.com. .. image:: images/velos_deploying_a_tenant/image1.png :align: center @@ -33,9 +33,9 @@ The **T1-F5OS** image type should be used with extreme caution. It is the smalle The remaining images (T2, ALL, T4) all support in place upgrades; however, they may limit the amount of disk space that can be used by the tenant. If more disk space is needed by the tenant in the future, the tenant can be moved to provisioned state and the disk can be expanded. There is no ability to decrease the disk space, so starting smaller and increasing will ensure there is adequate disk space for many tenants. -The **T2-F5OS** image is intended for a tenant that will run LTM and / or DNS only; it is not suitable for tenants needing other modules provisioned (AVR may be an exception). This type of image is best suited in a high density tenant environment where the number of tenants is going to be high per blade and using minimum CPU resources (1 or 2 vCPUs per tenant). You may want to limit the amount of disk space each tenant can use as a means of ensuring the file system on the blade does not become full. As an example, there is 1TB of disk per blade, and 22 tenants each using the 142GB T4 image would lead to an over provisioning situation. Because tenants are deployed in sparse mode which allows over provisioning, this may not be an issue initially, but could become a problem later in the tenant’s lifespan as it writes more data to disk. To keep the tenants in check, you can deploy smaller T2 images, which can consume 45GB each. LTM/DNS deployments use much less disk than other BIG-IP modules, which do extensive local logging and utilize databases on disk. +The **T2-F5OS** image is intended for a tenant that will run LTM and / or DNS only; it is not suitable for tenants needing other modules provisioned (AVR may be an exception). This type of image is best suited in a high-density tenant environment where the number of tenants is going to be high per blade and using minimum CPU resources (1 or 2 vCPUs per tenant). You may want to limit the amount of disk space each tenant can use as a means of ensuring the file system on the blade does not become full. As an example, there is 1TB of disk per blade, and 22 tenants each using the 142GB T4 image would lead to an over provisioning situation. Because tenants are deployed in sparse mode which allows over provisioning, this may not be an issue initially, but could become a problem later in the tenant’s lifespan as it writes more data to disk. To keep the tenants in check, you can deploy smaller T2 images, which can consume 45GB each. LTM/DNS deployments use much less disk than other BIG-IP modules, which do extensive local logging and utilize databases on disk. -The **All-F5OS** image is suitable for any module configuration and supports a maximum of 76GB for the tenant. It is expected that the number of tenants per blade would be much less, as the module combinations that drive the need for more disk space typically require more CPU/memory, which will artificially reduce the tenant count per blade. Having a handful of 76GB or 156GB images per blade should not lead to an out of space condition. There are some environments where some tenants may need more disk space and the T4 image can provide for that. It may be best to default to using the T4 image as that is essentially the default size for vCMP deployments today. +The **All-F5OS** image is suitable for any module configuration and supports a maximum of 76GB for the tenant. It is expected that the number of tenants per blade would be much less, as the module combinations that drive the need for more disk space typically require more CPU/memory, which will artificially reduce the tenant count per blade. Having a handful of 76GB or 156GB images per blade should not lead to an out of space condition. There are some environments where some tenants may need more disk space and the T4 image can provide for that. It may be best to default using the T4 image as that is essentially the default size for vCMP deployments today. The **T4-VELOS** image also supports any module combination but has additional disk capacity. If you intend to have a lot of software images, databases for modules, run modules like SWG which utilize a lot of disk, and local logging, then the added capacity is recommended. More detail on the image types can be found in the following solution article. @@ -47,19 +47,19 @@ Note that the image sizes in the chart are the maximum amount of space a tenant :align: center :scale: 70% -This means the disk consumption on the chassis partition disk is actually much smaller than what appears inside the tenant. In the example below the tenant believes it has 77GB of disk allocated: +This means the disk consumption on the chassis partition disk is much smaller than what appears inside the tenant. In the example below the tenant believes it has 77GB of disk allocated: .. image:: images/velos_deploying_a_tenant/image6.png :align: center :scale: 70% -However, the 76GB image is allocated in a sparse manner meaning the tenant is only utilizing what it needs, and on the file system of the blade it is actually consuming only 11GB on the disk: +However, the 76GB image is allocated in a sparse manner meaning the tenant is only utilizing what it needs, and on the file system of the blade it is consuming only 11GB on the disk: .. image:: images/velos_deploying_a_tenant/image7.png :align: center :scale: 70% -This is analogous to thin provisioning in a hypervisor, where you can over-allocate resources. vCMP as an example today uses an image similar in size to the T4 image. There may be rare instances where a tenant running in production for a long time can end up with lots of extra space consumed on disk. This could be due to many in-place software upgrades, local logging, core files, database use etc…There is no utility available to reclaim that space that may have been used at one point but is no longer used. If the disk utilization becomes over-utilized, you could back up the tenant configuration, create a new fresh tenant, and restore the configuration from the old tenant, and then delete the old tenant. This would free up all the unused space again. +This is analogous to thin provisioning in a hypervisor, where you can over-allocate resources. vCMP as an example today uses an image similar in size to the T4 image. There may be rare instances where a tenant running in production for a long time can end up with lots of extra space consumed on disk. This could be due to many in-place software upgrades, local logging, core files, database use and other factors. There is no utility available to reclaim that space that may have been used at one point but is no longer used. If the disk utilization becomes over-utilized, you could back up the tenant configuration, create a new fresh tenant, and restore the configuration from the old tenant, and then delete the old tenant. This would free up all the unused space again. Tenant Deployment via CLI ------------------------- @@ -234,7 +234,7 @@ You can deploy a tenant from the webUI using the **Add** button in the **Tenant :align: center :scale: 70% -The tenant deployment options are almost identical to deploying a vCMP guest, with a few minor differences. You’ll supply the tenant a name, and choose the image for it to run. Next, you will pick what slots (blades) within the chassis partition you want the tenant to run on, and assign an out-of-band management address, prefix, and gateway. There are **Recommended** and **Advanced** options for resource provisioning, Choosing Recommended will automatically adjust memory based on the vCPUs allocated to the tenant. Choosing Advanced will allow you to over-allocate memory which is something VIPRION did not support. You can choose different states (Configured, Provisioned, Deployed) just like vCMP, and there is an option to enable/disable hardware crypto acceleration (Enable is recommended). And finally, there is an option to enable Appliance mode which will disable root/bash access to the tenant. +The tenant deployment options are almost identical to deploying a vCMP guest, with a few minor differences. You’ll supply the tenant a name and choose the image for it to run. Next, you will pick what slots (blades) within the chassis partition you want the tenant to run on and assign an out-of-band management address, prefix, and gateway. There are **Recommended** and **Advanced** options for resource provisioning, Choosing Recommended will automatically adjust memory based on the vCPUs allocated to the tenant. Choosing Advanced will allow you to over-allocate memory which is something VIPRION did not support. You can choose different states (Configured, Provisioned, Deployed) just like vCMP, and there is an option to enable/disable hardware crypto acceleration (Enable is recommended). And finally, there is an option to enable Appliance mode which will disable root/bash access to the tenant. .. image:: images/velos_deploying_a_tenant/image11.png :align: center @@ -270,7 +270,7 @@ The upload utility requires a remote HTTPS, SCP, or SFTP server that is hosting ] } -To list the current tenant images available on the chassis partition use the following API Call: +To list the current tenant images available on the chassis partition, use the following API Call: .. code-block:: bash @@ -566,7 +566,7 @@ Below is an example output from a VELOS system: Resizing a Tenant ----------------- -VELOS tenants have static CPU and memory allocations. These can be changed after a tenant has been deployed, but the tenant will have to be temporarily suspended (put in the **provisioned** state), then the change to CPU and/or memory allocation can be made. A tenant can be expanded within a single blade or it can be configured to extend across blades assuming adequate resources are available. Once the changes are completed the tenant can be put into the **deployed** state and returned to service. +VELOS tenants have static CPU and memory allocations. These can be changed after a tenant has been deployed, but the tenant will have to be temporarily suspended (put in the **provisioned** state), then the change to CPU and/or memory allocation can be made. A tenant can be expanded within a single blade, or it can be configured to extend across blades assuming adequate resources are available. Once the changes are completed the tenant can be put into the **deployed** state and returned to service. Expanding a Tenant within the Same Blade via webUI ------------------------------------------------ @@ -587,7 +587,7 @@ Click **OK**. This will move the tenant from **deployed** to **provisioned** sta :align: center :scale: 70% -Next click on the hyperlink for tenant1. This will bring you into the configuration page for that tenant. Change the **vCPUs per slot** to **4**, and the **Memory per Slot** to **14848**, and set the state back to **deployed**. When finished click Save and the tenant will start up again with the new configuration. +Next click on the hyperlink for tenant1. This will bring you into the configuration page for that tenant. Change the **vCPUs per slot** to **4**, and the **Memory per Slot** to **14848** and set the state back to **deployed**. When finished click Save and the tenant will start up again with the new configuration. .. image:: images/velos_deploying_a_tenant/image15.png :align: center @@ -791,9 +791,9 @@ Expanding a Tenant Across Blades via webUI VELOS tenants can be configured to expand across multiple blades. You can pre-configure a tenant to span more than one blade, and as blades are added to a chassis partition the tenant should automatically expand and start using additional resources it has been configured for. Spanning tenants across two or more blades have advantages and disadvantages that need to be considered. -For tenants where the control plane is heavily utilized, spanning the tenant across blades can make the control plane performance worse, as it now needs to replicate its state between blades and this adds addtional overhead. Spanning tenants across blades also requires more IP addresses inside the tenants (one for each blade the tenant resides on) to ensure all failure cases are handled properly. A tenant can be configured to survive a blade failure and not failover to its peer, provided it has enough resources to run on a single blade. This is handled through HA group configuration within the tenant itself. It may be better in some cases to just failover to the tenant's peer in another chassis if a blade failure occurs. Expanding a tenant across blades can provide much higher data plane performance for a single tenant, so all these considerations need to be examined to determine the best configuration. +For tenants where the control plane is heavily utilized, spanning the tenant across blades can make the control plane performance worse, as it now needs to replicate its state between blades, and this adds additional overhead. Spanning tenants across blades also requires more IP addresses inside the tenants (one for each blade the tenant resides on) to ensure all failure cases are handled properly. A tenant can be configured to survive a blade failure and not failover to its peer, provided it has enough resources to run on a single blade. This is handled through HA group configuration within the tenant itself. It may be better in some cases to just failover to the tenant's peer in another chassis if a blade failure occurs. Expanding a tenant across blades can provide much higher data plane performance for a single tenant, so all these considerations need to be examined to determine the best configuration. -One consideration when expanding a tenant across more than one blade is that you will need to configure additional out-of-band IP addresses for each blade that the tenant will reside on. This is required for proper HA communication and failover to cover specific cases around blade failures. Below is a webUI screenshot inside a VELOS tenant that shows the out-of-band management IP address along with the **Cluster Member IP Addresses**. You should configure a Cluster Member IP Address for each slot that a tenant will span. The **Alternate Management** and **Alternate Cluster Member IP addresses** are for dual stack IPv4/IPv6 support and you would configure IPv6 addresses here, if the primary addresses were IPv4. +One consideration when expanding a tenant across more than one blade is that you will need to configure additional out-of-band IP addresses for each blade that the tenant will reside on. This is required for proper HA communication and failover to cover specific cases around blade failures. Below is a webUI screenshot inside a VELOS tenant that shows the out-of-band management IP address along with the **Cluster Member IP Addresses**. You should configure a Cluster Member IP Address for each slot that a tenant will span. The **Alternate Management** and **Alternate Cluster Member IP addresses** are for dual stack IPv4/IPv6 support and you would configure IPv6 addresses here if the primary addresses were IPv4. .. image:: images/velos_deploying_a_tenant/image17.png :align: center @@ -895,7 +895,7 @@ Next alter the nodes configuration to [ 1 2 ] so that the tenant will deploy ont Production-1(config-tenant-tenant2)# commit Commit complete. -You can verify the tenant status using the show tenants command. Note that Node 1 and Node 2 have an instance of tenant2 running. +You can verify the tenant status using the **show tenants** command. Note that Node 1 and Node 2 have an instance of tenant2 running. .. code-block:: bash @@ -930,7 +930,7 @@ You can verify the tenant status using the show tenants command. Note that Node Expanding a Tenant Across Blades via API ---------------------------------------- -If the tenant is already deployed, then you must first change the tenant to a provisioned state before changes can be made. This will cause the tenant to shutdown. The following API call will move the tenant to a provisioned state. +If the tenant is already deployed, then you must first change the tenant to a provisioned state before changes can be made. This will cause the tenant to shut down. The following API call will move the tenant to a provisioned state. .. code-block:: bash diff --git a/docs/velos_diagnostics.rst b/docs/velos_diagnostics.rst index e0b7e11..c375e19 100644 --- a/docs/velos_diagnostics.rst +++ b/docs/velos_diagnostics.rst @@ -5,7 +5,7 @@ VELOS Diagnostics Qkviews ======= -VELOS supports the ability to generate qkview reporte to collect and bundle configuration and diagnostic data that can be sent to support or uploaded to iHealth. It is important to understand the VELOS architecture when generating qkview reports. Generating a qkview report from the system controller will capture OS data and container information related to the system controller software, while generating a qkview report inside a chassis partition will capture data and container information related to the partition layer. To capture tenant level information, you’ll need to run a qkview report inside the TMOS layer of the tenant. +VELOS supports the ability to generate qkview reports to collect and bundle configuration and diagnostic data that can be sent to support or uploaded to iHealth. It is important to understand the VELOS architecture when generating qkview reports. Generating a qkview report from the system controller will capture OS data and container information related to the system controller software, while generating a qkview report inside a chassis partition will capture data and container information related to the partition layer. To capture tenant level information, you’ll need to run a qkview report inside the TMOS layer of the tenant. K02521182: Generating diagnostic data for the VELOS system using the qkview utility @@ -44,7 +44,7 @@ In both the system controller and the chassis partition the qkview can be genera :align: center :scale: 70% -To generate a qkview report, click the button in the upper right-hand corner. It will take some time for the qkview to be generated. Once the qkview is generated, you can click the checkbox next to it, and then select **Upload to iHealth**. Your iHealth credentials will automatically fill in if you entered them previously, and can be cleared if you want to use another account. You can optionally add an **F5 Support Case Number** and **Description** when uploading to iHealth. +To generate a qkview report, click the button in the upper right-hand corner. It will take some time for the qkview to be generated. Once the qkview is generated, you can click the checkbox next to it, and then select **Upload to iHealth**. Your iHealth credentials will automatically fill in if you entered them previously and can be cleared if you want to use another account. You can optionally add an **F5 Support Case Number** and **Description** when uploading to iHealth. .. image:: images/velos_diagnostics/image2.png @@ -142,13 +142,13 @@ The output of the command will show the percentage complete of the qkview. } } -If you'd like to copy the qkview directly to iHealth once it is completed use the following API command referencing the previously completed qkview file. +If you'd like to copy the qkview directly to iHealth once it is completed, use the following API command referencing the previously completed qkview file. .. code-block:: bash POST https://{{velos_velos_chassis1_system_controller_ip}}:8888/restconf/data/openconfig-system:system/f5-system-diagnostics-qkview:diagnostics/f5-system-diagnostics-ihealth:ihealth/f5-system-diagnostics-ihealth:upload -In the body of the API call add details with the filename, optional description and SR number. The call below assumes you have previously stored iHealth credentials, otherwise you can add them inside the API call. +In the body of the API call add details with the filename, optional description, and SR number. The call below assumes you have previously stored iHealth credentials, otherwise you can add them inside the API call. .. code-block:: json @@ -696,7 +696,7 @@ Logging Software Component Descriptions Below is a brief description of what each sw-component is responsible for, and some example logging messages when DEBUG is enabled. Note that when DEBUG level is set these messages are not intended for customers, they are for F5 developers and support personnel. -**alert-service** - The Alert Service runs on the both System Controllers and also each blade. "Alarm" is the user-facing term for alerts. Applications can send an AlertNotification or ThresholdNotification message over ZeroMQ to their local alert service. The blades and the standby controller forward all of their alert messages to the alert service running on the active controller. It aggregates all alerts and publishes them to ConfD. +**alert-service** - The Alert Service runs on the both System Controllers and also each blade. "Alarm" is the user-facing term for alerts. Applications can send an AlertNotification or ThresholdNotification message over ZeroMQ to their local alert service. The blades and the standby controller forward all alert messages to the alert service running on the active controller. It aggregates all alerts and publishes them to ConfD. https://docs.f5net.com/display/PDDESIGN/Vanquish+Alert+Service+Spec @@ -1287,7 +1287,7 @@ You can use the **tcpdump** utility on the VELOS system to capture traffic in ch When you use the tcpdump utility to capture traffic on a VELOS system, traffic is captured based on the chassis partition in which the command was run. Only the traffic that occurs on that chassis partition is captured. This includes traffic traversing the front panel ports on the chassis blades in the chassis partition as well as backplane traffic for the chassis partition. -When you run tcpdump in a chassis partition, a secondary tcpdump operation runs on each member blade in the chassis partition. The packets captured by the secondary tcpdumps are collected together in the command output. +When you run tcpdump in a chassis partition, a secondary tcpdump operation runs on each member blade in the chassis partition. The packets captured by the secondary tcpdumps are collected in the command output. In addition to the normal tcpdump output, the following fields have been added that are specific to the VELOS system: @@ -1306,7 +1306,7 @@ https://support.f5.com/csp/article/K12313135 You can capture traffic for a specific interface on a blade using the interface keyword in the tcpdump command. The interface is specified as /.. If the interface keyword is not supplied, or if 0/0.0 is specified for the interface, no interface filtering occurs and the command captures all interfaces in the partition. -Important: The interfaces on the VELOS system are capable of very high traffic rates. To prevent dropped packets during traffic capture, you should specify appropriate filters in order to capture only the intended traffic and reduce the total amount of captured traffic. +Important: The interfaces on the VELOS system are capable of very high traffic rates. To prevent dropped packets during traffic capture, you should specify appropriate filters to capture only the intended traffic and reduce the total amount of captured traffic. For example, the following command captures traffic on interface 1.0 on blade number 2: @@ -1337,7 +1337,7 @@ system diagnostics tcpdump bpf "src host 10.10.1.1 and dst port 443" Specify an output file ---------------------- -To send the captured traffic to a file, specify the filename using the outfile keyword. The resulting file is placed in the /var/F5// directory by default, or you can specify the directory in which to save the file. +To send the captured traffic to a file, specify the filename using the outfile keyword. The resulting file is placed in the /var/F5// directory by default, or you can specify the directory in which to save the file. For example, the following command sends the output of the tcpdump command to the /var/F5/partition/shared/example_capture.pcap file: @@ -1362,11 +1362,11 @@ You may have a need to access the console of a VELOS BX110 blade, one of the sys • System controller ports 7100 and 7200 map to system controllers 1 & 2 • Chassis partition ports 700x map to tenant ID’s (requires tenant name as username) -You can connect to any blade by SSH’ing to the floating IP address of the system controller and specifying the proper port for the blade you want to connect with. Port 7001 maps to blade-1, 7002 to blade-2 etc. Once connected to the terminal server, you will need to log in as root to the blade. The blade will have the default root password and will need to be changed on first reboot. The example below shows connecting to blade 2 ( port 7002) through a terminal server. +You can connect to any blade by SSH’ing to the floating IP address of the system controller and specifying the proper port for the blade you want to connect with. Port 7001 maps to blade-1, 7002 to blade-2 etc. Once connected to the terminal server, you will need to log in as root to the blade. The blade will have the default root password and will need to be changed on first reboot. The example below shows connecting to blade 2 (port 7002) through a terminal server. .. code-block:: bash - FLD-ML-00054045:~ jmccarron$ ssh -l admin 10.255.0.147 -p 7002 + $ ssh -l admin 10.255.0.147 -p 7002 admin@10.255.0.147's password: Terminal session established @@ -1384,7 +1384,7 @@ Connecting to a system controller follows the same general process but uses port .. code-block:: bash - FLD-ML-00054045:~ jmccarron$ ssh -l admin 10.255.0.147 -p 7100 + $ ssh -l admin 10.255.0.147 -p 7100 admin@10.255.0.147's password: Terminal session established diff --git a/docs/velos_f5os_configuration_backup_and_restore.rst b/docs/velos_f5os_configuration_backup_and_restore.rst index 6ec9911..5473b7b 100644 --- a/docs/velos_f5os_configuration_backup_and_restore.rst +++ b/docs/velos_f5os_configuration_backup_and_restore.rst @@ -128,7 +128,7 @@ To transfer a file using the CLI use the **file list** command to see the conten } -To transfer the file from the CLI you can use the **file export** command. The option below is exporting to a remote HTTPS server. there are options to trasnfer using SFTP, and SCP as well. +To transfer the file from the CLI you can use the **file export** command. The option below is exporting to a remote HTTPS server. there are options to transfer using SFTP, and SCP as well. .. code-block:: bash @@ -148,7 +148,7 @@ To check on status of the export use the **file transfer-status** command: 2 |Export file|HTTPS |/mnt/var/confd/configs/chassis1-sys-controller-backup-2-26-21|10.255.0.142 |chassis1-sys-controller-backup-2-26-21 |Failed to open/read local data from file/application 3 |Export file|HTTPS |/mnt/var/confd/configs/chassis1-sys-controller-backup-2-26-21|10.255.0.142 |/backup |Failed to open/read local data from file/application -If you don’t have an external HTTPS server that allows uploads, then you can log into the system controllers floating IP address with root access and scp the file from the shell. Go to the **/var/confd/configs** directory and scp the file to an external location. Note in the CLI and webUI the path is simplified to configs, but in the underlying file system it is actually stored in the **/var/confd/configs** directory. +If you don’t have an external HTTPS server that allows uploads, then you can log into the system controllers floating IP address with root access and scp the file from the shell. Go to the **/var/confd/configs** directory and scp the file to an external location. Note in the CLI and webUI the path is simplified to configs, but in the underlying file system it is stored in the **/var/confd/configs** directory. .. code-block:: bash @@ -189,7 +189,7 @@ To copy a ConfD configuration backup file from the system controller to a remote Backing Up Chassis Partition Databases ====================================== -In addition to backing up the system controller database, you should backup the configuration database on each chassis partition within the VELOS system. In the example below there are two chassis partitions currently in use; **Production** and **Development**. Both must be backed up and archived off of the VELOS system. +In addition to backing up the system controller database, you should backup the configuration database on each chassis partition within the VELOS system. In the example below there are two chassis partitions currently in use: **Production** and **Development**. Both must be backed up and archived off of the VELOS system. Backing Up Chassis Partition Databases via CLI ---------------------------------------------- @@ -318,7 +318,7 @@ You can use the CLI command **file transfer-status** to see if the file was copi Production-1# -If you do not have a remote HTTPS, SCP, or SFTP server with the proper access to POST files, then you can copy the chassis partition backups from the system controller shell (Note, there is no shelll access via the chassis partition IP). You’ll need to login to the system controllers shell using the root account. Once logged in list the contents of the **/var/F5** directory. You’ll notice **partition** directories, where equals the ID assigned to each partition. +If you do not have a remote HTTPS, SCP, or SFTP server with the proper access to POST files, then you can copy the chassis partition backups from the system controller shell (Note, there is no shell access via the chassis partition IP). You’ll need to login to the system controllers shell using the root account. Once logged in list the contents of the **/var/F5** directory. You’ll notice **partition** directories, where equals the ID assigned to each partition. .. code-block:: bash @@ -361,7 +361,7 @@ Now repeat the same steps for each chassis partition in the system. Export Backup From the Chassis Partition API -------------------------------------------- -Each chassis partition in the system needs to be backed up independently. Below is an API example of exportinh the backuo up the chassis partition **Development**. Note the API call is sent to the chassis partition IP address. Currently a remote HTTPS, SCP, or SFTP server is required to export the copy of the configuration backup. +Each chassis partition in the system needs to be backed up independently. Below is an API example exporting the backup of the chassis partition **Development**. Note the API call is sent to the chassis partition IP address. Currently a remote HTTPS, SCP, or SFTP server is required to export the copy of the configuration backup. .. code-block:: bash @@ -480,7 +480,7 @@ For the final step, reset the system controllers ConfD database. This will essen syscon-2-active(config)# system database config reset-default-config true syscon-2-active(config)# commit -Once this has been committed, both controllers need to be rebooted manually and in quick succession of each other. Login to the active controller and enter **config** mode, and then issue the **system reboot controllers controller standby** command, this will reboot the standby controller first. Run the same command again but this time reboot the **active** controller immediately after resetting the primary controller. You don't want any sort of long pause (minutes) between the resets. Ideally these commands should be run back to back. +Once this has been committed, both controllers need to be rebooted manually and in quick succession of each other. Login to the active controller and enter **config** mode, and then issue the **system reboot controllers controller standby** command, this will reboot the standby controller first. Run the same command again but this time reboot the **active** controller immediately after resetting the primary controller. You don't want any sort of long pause (minutes) between the resets. Ideally these commands should be run back-to-back. .. code-block:: bash @@ -575,7 +575,7 @@ The last step in the reset procedure is to set the system controllers ConfD data "f5-database:reset-default-config": "true" } -Once this has been committed, both controllers need to be rebooted manually and in quick succession of each other. Login to the active controller and enter **config** mode, and then issue the **system reboot controllers controller standby** command, this will reboot the standby controller first. Run the same command again but this time reboot the **active** controller immediately after resetting the primary controller. You don't want any sort of long pause (minutes) between the resets. Ideally these commands should be run back to back. +Once this has been committed, both controllers need to be rebooted manually and in quick succession of each other. Login to the active controller and enter **config** mode, and then issue the **system reboot controllers controller standby** command, this will reboot the standby controller first. Run the same command again but this time reboot the **active** controller immediately after resetting the primary controller. You don't want any sort of long pause (minutes) between the resets. Ideally these commands should be run back-to-back. .. code-block:: bash @@ -673,7 +673,7 @@ To transfer files into the system controller you’ll have to manually configure Importing System Controller Backups =================================== -Once the system is configured and out-of-band connectivity is restored, you can now copy the ConfD database archives back into the system controllers. If you are in the bash shell you can simply SCP the file into the **/var/confd/configs** directory. If it doesn’t exist, you can create it by creating a dummy backup of the system controllers configuration as outlined earlier. +Once the system is configured and out-of-band connectivity is restored, you can now copy the ConfD database archives back into the system controllers. If you are in the bash shell you can simply SCP the file into the **/var/confd/configs** directory. If it doesn’t exist, you can create it by creating a dummy backup of the system controller's configuration as outlined earlier. Next SCP the file from a remote server: @@ -1135,7 +1135,7 @@ You can check on the file transfer status by issubg the following API call: POST https://{{velos_chassis1_chassis_partition1_ip}}:8888/api/data/f5-utils-file-transfer:file/transfer-status -A status similar to the one below will show a status of completed if successful: +A status like the one below will show a status of completed if successful: .. code-block:: json diff --git a/docs/velos_high_availability.rst b/docs/velos_high_availability.rst index f40f18b..f8ec3ac 100644 --- a/docs/velos_high_availability.rst +++ b/docs/velos_high_availability.rst @@ -7,7 +7,7 @@ System Controller HA The system controllers perform two main functions for the VELOS chassis. They are the centralized layer2 switch fabric connecting all blades within the system, and they also host the Kubernetes control plane which manages the F5OS layer. The CX410 system ships with redundant system controllers. -The layer2 switch fabric function performed by the system controllers runs in an active-active manner. Both switch fabrics are active, and each BX110 blade is dual-homed with a 100Gb backplane connection to each system controller (200Gb total). On the BX110 blade, the two 100Gb ports are bonded together in a static Link Aggregation Group (LAG). Traffic destined for other blades in the system will hash over the two links (assuming both are active), then traverse the switch fabrics on the system controllers on its way to the destination blade. +The layer2 switch fabric function performed by the system controllers runs in an active-active manner. Both switch fabrics are active, and each BX110 blade is dual homed with a 100Gb backplane connection to each system controller (200Gb total). On the BX110 blade, the two 100Gb ports are bonded together in a static Link Aggregation Group (LAG). Traffic destined for other blades in the system will hash over the two links (assuming both are active), then traverse the switch fabrics on the system controllers on its way to the destination blade. .. image:: images/velos_high_availability/image1.png @@ -16,18 +16,30 @@ The layer2 switch fabric function performed by the system controllers runs in an While both switch fabrics are active, there is 1.6Tbs of switching capacity between all the blades in the system. If one of the switch fabrics should fail, then the total bandwidth will be cut in half to 800Gbs on the backplane, and each blade will be limited to 100Gbs of backplane capacity. Note that the current throughput rating on the BX110 blades (95Gb) will not push the backplane to full capacity, even if there is a single system controller failure. -The second function the system controllers perform is the management of the control plane for the new Kubernetes platform layer (F5OS) . At this layer the system controllers run in an active/standby fashion. One system controller will be designated primary/active, and the other will be designated secondary/standby. All Kubernetes services will be active on the primary including the API server, scheduler, controller manager, etcd, webUI services, etc. The active system controller can always be reached via the floating IP address that is assigned to the system controllers. The floating address will move in the case of a controller failure. The secondary controller operates in a read-only manner and any changes to configuration must be made on the primary. All changes are replicated from primary to secondary controller, so they should always be in sync, there is no need to manually sync them. The Kubernetes control plane is responsible for deploying workloads on the BX110 blades. +The second function the system controllers perform is the management of the control plane for the new Kubernetes platform layer (F5OS). At this layer the system controllers run in an active/standby fashion. One system controller will be designated primary/active, and the other will be designated secondary/standby. All Kubernetes services will be active on the primary including the API server, scheduler, controller manager, etcd, webUI services, etc. The active system controller can always be reached via the floating IP address that is assigned to the system controllers. The floating address will move in the case of a controller failure. The secondary controller operates in a read-only manner and any changes to configuration must be made on the primary. All changes are replicated from primary to secondary controller, so they should always be in sync, there is no need to manually sync them. The Kubernetes control plane is responsible for deploying workloads on the BX110 blades. .. image:: images/velos_high_availability/image2.png :align: center :scale: 70% -You may view the current high availability status in the dashboard of the system controller webUI. It will show which controller is primary and standby, as well as other system infromation related to the system controllers. +You may view the current high availability status in the dashboard of the system controller webUI. It will show which controller is primary and standby, as well as other system information related to the system controllers. .. image:: images/velos_high_availability/image3.png :align: center :scale: 70% +The **Controller Overview** tab will show CPU utilization, controller role and status, system memory utilization, system storage utilization and the Base OS and Service versions running on the controllers. + +.. image:: images/velos_high_availability/image3a.png + :align: center + :scale: 70% + +The **Active Alarms** tab will display any active alarms on the system controllers. + +.. image:: images/velos_high_availability/image3b.png + :align: center + :scale: 70% + You can view and configure the High Availability options for the system controllers in the webUI **Systems Settings > Controller Management** screen. You can force a failover, configure auto failback, as well as set the preferred node: .. image:: images/velos_high_availability/image4.png @@ -47,15 +59,15 @@ This will fail the Kubernetes control plane services from the active system cont A physical reboot of the active system controller will cause all backplane traffic to temporarily use only the controller previously in the standby status, until the system controller that was rebooted returns. At that point, traffic will resume forwarding across both system controllers in an active/active manner. It is possible a slight disruption may occur to any tenant traffic that was in-flight across the backplane when the reboot occurs. -If a software upgrade is being performed, then both system controllers are upgraded in parallel resulting in an outage for the entire chassis for the initial 1.1.x versions of F5OS. Rolling upgrade support for the system controllers was added in the F5OS 1.2.x release. This allows for a serial rolling upgrade to take place for the system controllers and minimal downtime to the tenants when the F5OS-C image on the controllers is upgraded. +If a software upgrade is being performed on the system controllers, then a rolling upgrade will be performed. Support for rolling upgrades of the system controllers was added in the F5OS-C 1.2.x release. This allows for a serial rolling upgrade to take place for the system controllers and minimal downtime to the tenants when the F5OS-C image on the controllers is upgraded. Blade Level HA ============== -From a design standpoint it is best to spread in-band network connections across multiple blades if possible. Assuming there is more than one blade within a chassis partition spreading network connections via LAGs will give the best resiliency and performance. Consider the diagram below where a LAG is split across two different BX110 blades. Incoming traffic should hash across both links and statistically should provide fairly even distribution given enough connections. Performance can still be lumpy, as the upstream switch is likely performing a hash-based distribution, and not distributing based on load. +From a design standpoint it is best to spread in-band network connections across multiple blades if possible. Assuming there is more than one blade within a chassis partition, spreading network connections via LAGs will give the best resiliency and performance. Consider the diagram below where a LAG is split across two different BX110 blades. Incoming traffic should hash across both links and statistically should provide fairly even distribution given enough connections. Performance can still be lumpy, as the upstream switch is likely performing a hash-based distribution, and not distributing based on load. -Incoming traffic will go through a disaggregation (DAG) process where connections are spread across all processors within the VELOS tenant. Again, a hash-based distribution will decide which blade/processor should handle the incoming connection. Given enough connections the expectation is that half are processed locally on the incoming blade, and the other half will be sent across the backplane to another blade assuming a tenant is configured to utilize more than one blade. If the return traffic going outbound can egress on the same blade that processed the connection, then a backplane traversal is saved, it doesn’t have to go back to the incoming blade. If a blade fails, or one of the links in the LAG should fail, then all traffic will ingress and egress on the remaining blade. There are more granular configuration options within the tenant to determine how failover cases should be handled if a blade should fail. Of course, additional blades/links can be added to a chassis partition, but they follow this forwarding behavior: +Incoming traffic will go through a disaggregation (DAG) process where connections are spread across all processors within the VELOS tenant. Again, a hash-based distribution will decide which blade/processor should handle the incoming connection. Given enough connections the expectation is that half are processed locally on the incoming blade, and the other half will be sent across the backplane to another blade, assuming a tenant is configured to utilize more than one blade. If the return traffic going outbound can egress on the same blade that processed the connection, then a backplane traversal is saved, it doesn’t have to go back to the incoming blade. If a blade fails, or one of the links in the LAG should fail, then all traffic will ingress and egress on the remaining blade. There are more granular configuration options within the tenant to determine how failover cases should be handled if a blade should fail. Of course, additional blades/links can be added to a chassis partition, but they follow this forwarding behavior: .. image:: images/velos_high_availability/image6.png :align: center @@ -85,7 +97,7 @@ Tenants on different chassis should have the same number of vCPUs and be configu Tenant Level HA within the Chassis ================================== -VELOS does not support configuring HA relationships between tenants within the same chassis. You can configure tenants to span multiple blades, and then depending on what failover behavior you want, you can have the tenant run with less capacity within the same chassis if a blade fails, or fail over to the tenant in the other chassis. This is controlled within the tenant itself, just like vCMP guests HA failover was configured. HA groups allow an administrator to fail over based on pool, trunk, or blade availability. +VELOS does not support configuring HA relationships between tenants within the same chassis. You can configure tenants to span multiple blades, and then depending on what failover behavior you want, you can have the tenant run with less capacity within the same chassis if a blade fails or fail over to the tenant in the other chassis. This is controlled within the tenant itself, just like vCMP guests HA failover was configured. HA groups allow an administrator to fail over based on pool, trunk, or blade availability. **Note: The HA Groups failover option based on trunks is not supported in VELOS tenants running 14.1.4.x and F5OS v1.1.x. The tenants do not have visibility into the F5OS layer Link Aggregation Groups. F5OS 1.2.x and later along with VELOS tenant running 15.1.4 or later support trunks within an HA Group.** @@ -104,7 +116,7 @@ Inside the tenant, one **Cluster Member IP Address** will need to be configured :align: center :scale: 90% -For planning purposes a single large tenant “SuperVIP” spanning 8 total blades would require 13 out-of-band management IP addresses for each chassis. In-band Self-IP and Virtual addresses are not included in this calculation. +For planning purposes, a single large tenant “SuperVIP” spanning 8 total blades would require 13 out-of-band management IP addresses for each chassis. In-band Self-IP and Virtual addresses are not included in this calculation. +------------------------------+----------------------------------+--------------------+ | **IP Addresses Required** | **Single Chassis** | **HA Environment** | @@ -139,7 +151,7 @@ As with previous generation BIG-IP appliances and chassis, there are multiple HA VELOS BX110 Blade ----------------- -Each VELOS BX110 blade has two physical ports, that currently support the following options for connectivity: 100Gb, 40Gb, 4 x 25Gb, and 4 x 10Gb. Currently, both ports on the same blade must be configured for the same speed and mode. The number of blades installed may dictate which approach makes the most sense, as the number of ports available, and the performance required, may dictate some topology decisions. +Each VELOS BX110 blade has two physical ports, that currently support the following options for connectivity: 100Gb, 40Gb, 4 x 25Gb, and 4 x 10Gb. The number of blades installed may dictate which approach makes the most sense, as the number of ports available, and the performance required, may dictate some topology decisions. .. image:: images/velos_high_availability/image12.png :align: center @@ -148,13 +160,13 @@ Each VELOS BX110 blade has two physical ports, that currently support the follow HA Topology Options ------------------- -VELOS does not support tenant HA within the same chassis. F5 recommends configuring dual VELOS chassis, with identically configured tenants, on the same slots, and maintaining HA relationships at the tenant level as seen below. This mimics the previous HA behavior supported on VIPRION between vCMP guests. There is no HA or redundancy configuration between chassis at the F5OS platform layer. The chassis themselves are unaware of the other chassis, and there is no HA communication at this level. It’s the tenants on seprate chassis that form the HA relationship. +VELOS does not support tenant HA within the same chassis. F5 recommends configuring dual VELOS chassis, with identically configured tenants, on the same slots, and maintaining HA relationships at the tenant level as seen below. This mimics the previous HA behavior supported on VIPRION between vCMP guests. There is no HA or redundancy configuration between chassis at the F5OS platform layer. The chassis themselves are unaware of the other chassis, and there is no HA communication at this level. It’s the tenants on separate chassis that form the HA relationship. .. image:: images/velos_high_availability/image8.png :align: center :scale: 70% -Tenants on different chassis should have the same number of vCPUs, and be configured to run on the same slots. HA interconnect VLANs would be configured between chassis partitions in the two separate chassis, and then tenants would configure HA just as is the case with vCMP guest HA relationships. Below is an example of two VELOS chassis with multiple chassis partitions each with their own HA interconnects and in-band networking. +Tenants on different chassis should have the same number of vCPUs and be configured to run on the same slots. HA interconnect VLANs would be configured between chassis partitions in the two separate chassis, and then tenants would configure HA just as is the case with vCMP guest HA relationships. Below is an example of two VELOS chassis with multiple chassis partitions each with their own HA interconnects and in-band networking. .. image:: images/velos_high_availability/image13.png @@ -170,7 +182,7 @@ Most modern environments will have dual upstream layer2 switches that handle the :align: center :scale: 60% -If the environment only has a single blade in each chassis, and 100Gb or 40Gb connectivity is desired, then putting both ports on the BX110 into a LAG and dual-homing it to the two upstream switches in a vPC makes the most sense. Since there aren’t more ports to dedicate to an HA interconnect LAG, this drives the decision of which topology is best. In the example below, the HA VLAN(s) will run on the same LAG as the in-band traffic. +If the environment only has a single blade in each chassis, and 100Gb or 40Gb connectivity is desired, then putting both ports on the BX110 into a LAG and dual homing it to the two upstream switches in a vPC makes the most sense. Since there aren’t more ports to dedicate to an HA interconnect LAG, this drives the decision of which topology is best. In the example below, the HA VLAN(s) will run on the same LAG as the in-band traffic. .. image:: images/velos_high_availability/image15.png :align: center @@ -183,7 +195,7 @@ If the environment is not running 100Gb or 40Gb, then the BX110 blade can be con :align: center :scale: 60% -As more BX110 blades are added to each VELOS chassis, more options become available, as the restriction of running only one speed/mode is lifted because a second blade could be configured to run at different speeds. This could allow some ports to run at lowers speeds (4 x 25Gb or 4 x 10Gb) and break out, while other ports are running higher speed (40Gb or 100Gb). +As more BX110 blades are added to each VELOS chassis, more options become available. Initially, there was a restriction of running only one speed/mode per blade, but that restriction was removed in F5OS-C 1.5.1. This could allow some ports to run at lowers speeds (4 x 25Gb or 4 x 10Gb) and break out, while other ports are running higher speed (40Gb or 100Gb). As additional blades are added, it makes sense to spread the LAG across more blades for added redundancy. It is not a requirement to extend the LAG to every blade within a chassis partition, but this can be done to optimize traffic flows and avoid extra backplane traversals. Consider the diagram below; where a single LAG on one blade is configured but 2 blades are installed. Traffic will enter blade1 and go through a disaggregation process where some traffic may be processed locally on blade1, and other traffic will be sent to the remote blade2. Tenant slot configuration will also play into this decision. This means an extra hop across the backplane/switch fabrics for transactions to be processed, and then the response having to come back across the backplane to exit the chassis via the LAG. @@ -213,9 +225,14 @@ The two topologies below are identical, except one has a dedicated LAG for the H :align: center :scale: 60% -Consider the case where mirror traffic is intermingled over the in-band LAG with application traffic. Unless there is some sort of prioritization implemented, it’s possible that heartbeat and mirroring type traffic may be affected by saturation somewhere in the upstream switch or within the networking layer. The main disadvantage of this topology is HA VLAN disruption due to an upstream switch error. This can affect mirroring and heartbeat, whereas a dedicated HA interconnect between the VELOS chassis has no dependencies on upstream switches or networking. The biggest concern is disrupting the failover heartbeats from the SOD process (udp port 1026) between the tenants. +Consider the case where mirror traffic is intermingled over the in-band LAG with application traffic. Unless there is some sort of prioritization implemented, it’s possible that heartbeat and mirroring type traffic may be affected by saturation somewhere in the upstream switch or within the networking layer. The main disadvantage of this topology is HA VLAN disruption due to an upstream switch error. This can affect mirroring and heartbeat, whereas a dedicated HA interconnect between the VELOS chassis has no dependencies on upstream switches or networking. The biggest concern is disrupting the failover heartbeats from the SOD process (UDP port 1026) between the tenants. + +The proper way to deploy, is to configure HA heartbeats over the management interface, as well as over the HA VLAN as outlined in the article below. Although it is written for VIPRION, the same logic applies for VELOS. + +`K37361453: Configuring network failover for redundant VIPRION systems (13.x - 16.x) `_ + -The proper way to deploy, is to configure HA heartbeats over the management interface, as well as over the HA VLAN (K37361453). Unfortunately, this may be more difficult than it seems for BIG-IP tenants that span multiple slots/blades in VELOS. You must make sure that each slot has an individual management address, and you must also configure either management multicast (and make sure it works), or a mesh of unicast management addresses. Many customers overlook this step, and if it not set up properly, stability of the HA environment will rely solely on the stability of the HA VLAN. +Unfortunately, this may be more difficult than it seems for BIG-IP tenants that span multiple slots/blades in VELOS. You must make sure that each slot has an individual management address, and you must also configure either management multicast (and make sure it works), or a mesh of unicast management addresses. Many customers overlook this step, and if it not set up properly, stability of the HA environment will rely solely on the stability of the HA VLAN. The example below shows a BIG-IP tenant configured on VELOS. For a single slot tenant (a tenant that only utilizes one slot/blade), you only need to configure the single management IP address. If a VELOS tenant spans more than one blade, then you must configure a separate cluster member IP address for each slot/blade that the tenant will run on. You cannot reuse these IP addresses within other tenants; they must have their own unique cluster member IP addresses if they span more than one blade. While spanning a tenant across blades may provide some level of local redundancy, it does require additional configuration and IP addressing, and may also cause additional strain on the control plane process (MCPD), as it will need to replicate between blades. These should be considered before finalizing a design for the tenants. @@ -229,17 +246,17 @@ The diagram below shows the configuration of multiple HA heartbeat paths. One is :align: center :scale: 70% -There is an alternative to configuring the multicast option over the management network, called **Unicast Mesh**, where each blade in the tenant is added and configured as a **Failover Unicast Address**, allowing each blade to exchange heartbeat messages with all of the blades where the peer tenant is runnnig. As with the Multicast option, you must configure a separate cluster member IP address inside the tenant, for each blade that the tenant will run on. +There is an alternative to configuring the multicast option over the management network, called **Unicast Mesh**, where each blade in the tenant is added and configured as a **Failover Unicast Address**, allowing each blade to exchange heartbeat messages with all the blades where the peer tenant is running. As with the Multicast option, you must configure a separate cluster member IP address inside the tenant, for each blade that the tenant will run on. How Many Ports are Required for an HA Interconnect LAG? ------------------------------------------------------- -The number of ports required in a dedicated HA Interconnect may vary based on perfromance and deployment options. Ideally, you should have a minimum of two ports within a LAG, with dedicated tagged VLANs for each tenant HA pair. Running config-sync and mirroring over this LAG interface is preferred if it has been enabled, because it is usually directly connected, has lower latency, and doesn't compete with in-band traffic. The two links in a LAG provide redundancy if one link should fail, and you can add more interfaces to the LAG for added resiliency. These links should be spread across additional blades when possible, for added redundancy. +The number of ports required in a dedicated HA Interconnect may vary based on performance and deployment options. Ideally, you should have a minimum of two ports within a LAG, with dedicated tagged VLANs for each tenant HA pair. Running config-sync and mirroring over this LAG interface is preferred if it has been enabled, because it is usually directly connected, has lower latency, and doesn't compete with in-band traffic. The two links in a LAG provide redundancy if one link should fail, and you can add more interfaces to the LAG for added resiliency. These links should be spread across additional blades, when possible, for added redundancy. Generally, heartbeat traffic is not very bandwidth sensitive, but it can be very sensitive to latency, especially when mirroring is enabled. Mirroring will take up more bandwidth over the HA links; layer4 mirroring is less bandwidth intensive than layer7 mirroring because it is sending less traffic to keep mirror state in sync. With layer4 mirroring, there is one packet mirrored **per connection**, whereas layer7 mirroring is one mirrored connection **per packet**. -You should plan to have adequate bandwidth in the LAG if mirroring is enabled. As mentioned above layer7 mirroring will generate lots of bandwidth as every packet has to be mirrored. Not every virtual server or application requires mirroring, it is reccomened you enable only where required. +You should plan to have adequate bandwidth in the LAG if mirroring is enabled. As mentioned above, layer7 mirroring will generate lots of bandwidth as every packet has to be mirrored. Not every virtual server or application requires mirroring, it is recommended you enable only where required. -Tenant failover triggers can be configured through the **HA Group** feature. This should be properly configured within VELOS tenants to detect external failures and trigger failover. You should configure an HA Group with sufficient weight on the correct operation of the main trunk/LAG to the upstream switches or optionally when the number of memers in a cluster is too low. Without HA Groups (or less preferred VLAN Failsafe), the only requirement keeping a device or virtual server active, is to have heartbeat. +Tenant failover triggers can be configured through the **HA Group** feature. This should be properly configured within VELOS tenants to detect external failures and trigger failover. You should configure an HA Group with sufficient weight on the correct operation of the main trunk/LAG to the upstream switches or optionally when the number of members in a cluster is too low. Without HA Groups (or less preferred VLAN Failsafe), the only requirement keeping a device or virtual server active, is to have heartbeat. If you're setting up tenant to tenant connectivity, and don't have HA groups configured, you may end up with cases where failover is desired but not triggered. If a VELOS chassis partition loses its main traffic LAG, it will continue to heartbeat over the HA interface, and will continue to remain active. To avoid this, make sure the HA group setup is explicitly mentioned as a requirement if you setup peer to peer dedicated HA VLANs. diff --git a/docs/velos_inside_the_tenant.rst b/docs/velos_inside_the_tenant.rst index 71366b2..eb8e43d 100644 --- a/docs/velos_inside_the_tenant.rst +++ b/docs/velos_inside_the_tenant.rst @@ -3,7 +3,7 @@ Inside the Tenant ================= -Once a tenant is deployed you can connect/communicate directly to one of its CLI, webUI, or API interfaces. At this layer you are interacting with TMOS, i.e. the experience should be almost identical to a vCMP guest with some minor exceptions. Day to day management of the tenant will use the same CLI (tmsh), API (iControl) and webUI as TMOS instances or hardware devices running within customer envronments today. If you are using vCMP today, then many of the concepts will be familiar. If you run and appliance or VIPRION in a bare metal mode, then there will be some differences as VELOS will be configured with at least one tenant, and lower layer configuration and stats will come from the F5OS layer. +Once a tenant is deployed you can connect/communicate directly to one of its CLI, webUI, or API interfaces. At this layer you are interacting with TMOS, i.e. the experience should be almost identical to a vCMP guest with some minor exceptions. Day to day management of the tenant will use the same CLI (tmsh), API (iControl) and webUI as TMOS instances or hardware devices running within customer environments today. If you are using vCMP today, then many of the concepts will be familiar. If you run and appliance or VIPRION in a bare metal mode, then there will be some differences as VELOS will be configured with at least one tenant, and lower layer configuration and stats will come from the F5OS layer. VLAN Behavior ============= @@ -16,7 +16,7 @@ VELOS follows a similar behavior as far as tenants inheriting VLANs from the F5O :align: center :scale: 70% -These are the VLANs as they appear in the F5OS platform layer. Notice that the tenant does not see all VLAN’s, only the ones that are assigned to it by the administrator: +These are the VLANs as they appear in the F5OS platform layer. Notice that the tenant does not see all VLANs, only the ones that are assigned to it by the administrator: .. image:: images/velos_inside_the_tenant/image2.png :align: center @@ -28,7 +28,7 @@ You can delete the VLANs inside the tenant and then recreate them with a new nam Interface Behavior ================== -The number of interfaces within a tenant will be based upon the number of vCPUs assigned to the tenant and the number of slots the tenant is running on. The screenshot below shows the interfaces inside the tenant lining up with the number of physical cores per slot. In the first example there are 6 vCPUs on a single slot tenant, this will equate to 3 physical CPUs. Likewise for a dual slot tenant with 10 vCPUs per slot. You’ll see 5 physical CPU’s per slot. +The number of interfaces within a tenant will be based upon the number of vCPUs assigned to the tenant and the number of slots the tenant is running on. The screenshot below shows the interfaces inside the tenant lining up with the number of physical cores per slot. In the first example there are 6 vCPUs on a single slot tenant, this will equate to 3 physical CPUs. Likewise for a dual slot tenant with 10 vCPUs per slot. You’ll see 5 physical CPUs per slot. .. image:: images/velos_inside_the_tenant/image3.png :align: center diff --git a/docs/velos_multitenancy.rst b/docs/velos_multitenancy.rst index 9013647..5aa7e10 100644 --- a/docs/velos_multitenancy.rst +++ b/docs/velos_multitenancy.rst @@ -2,17 +2,17 @@ Multitenancy ============ -In previous generation chassis and appliances, F5 supported **vCMP** as a means of providing multitenancy and virtualization. vCMP allowed for configuration of **Guests**; which were independent virtualized instances of BIG-IP. VELOS provides a similar type of virtualization experience, however it is not based on vCMP. Instead, VELOS allows for **Tenants** to be created, which are virtualized instances of BIG-IP on top of the containerized F5OS layer. +In previous generation chassis and appliances, F5 supported **vCMP** as a means of providing multitenancy and virtualization. vCMP allowed for configuration of **Guests**; which were independent virtualized instances of BIG-IP. VELOS provides a similar type of virtualization experience; however it is not based on vCMP. Instead, VELOS allows for **Tenants** to be created, which are virtualized instances of BIG-IP on top of the containerized F5OS layer. -Unlike VIPRION, where vCMP is an option that can added to the chassis, VELOS is multitenant by default. There is no option for a “bare metal” configuration; tenancy is baked into the architecture. You may configure one large tenant to emulate a “bare-metal” BIG-IP configuration if required. For customers that run bare-metal in VIPRION today, the L4-7 configuration and inherited VLANs will be migrated into a VELOS tenant, and the lower level networking (interfaces, Link Aggregation Groups, and VLANs) will be configured in the F5OS-C platform layer. Below is a depiction of BIG-IP tenants running on top of the F5OS layer. +Unlike VIPRION, where vCMP is an option that can added to the chassis, VELOS is multitenant by default. There is no option for a “bare metal” configuration; tenancy is baked into the architecture. You may configure one large tenant to emulate a “bare-metal” BIG-IP configuration if required. For customers that run bare-metal in VIPRION today, the L4-7 configuration and inherited VLANs will be migrated into a VELOS tenant, and the lower-level networking (interfaces, Link Aggregation Groups, and VLANs) will be configured in the F5OS-C platform layer. Below is a depiction of BIG-IP tenants running on top of the F5OS layer. .. image:: images/velos_multitenancy/image1.png :align: center :scale: 80% -Each tenant will run as a Virtual Machine via a technology called Kubevirt, which allows Virtual Machines to run on top of a containerized architecture. The tenant itself will run TMOS, and it will be managed similar to how a vCMP guest is managed. Note, the tenant is not a Virtual Edition (VE), it is a highly optimized virtual machine that is fully integrated with the underlying hardware, and supports all the traditional hardware offload capabilities like SSL/TLS offload, FASTL4 forwarding, DDoS mitgation, and much more. In the future, when BIG-IP Next tenants are supported in VELOS, those tenants will run in their native containerized mode, and not run as a Virtual Machine. +Each tenant will run as a Virtual Machine via a technology called Kubevirt, which allows Virtual Machines to run on top of a containerized architecture. The tenant itself will run TMOS, and it will be managed like a vCMP guest is managed. Note, the tenant is not a Virtual Edition (VE), it is a highly optimized virtual machine that is fully integrated with the underlying hardware and supports all the traditional hardware offload capabilities like SSL/TLS offload, FASTL4 forwarding, DDoS mitigation, and much more. In the future, when BIG-IP Next tenants are supported in VELOS, those tenants will run in their native containerized mode, and not run as a Virtual Machine. -Creating a VELOS tenant is nearly identical to creating a vCMP guest on VIPRION, with a few exceptions. When creating a VELOS tenant, you’ll provide a name, a TMOS image for the tenant to run, which slots (blades) the tenant will be configured to run on, out-of-band IP addressing/mask and gateway, and which VLANs the tenant should inherit. Just like a vCMP guest, the VLANs are configured at provision time and not within the tenant itself. The tenant will inherit VLANs that have been configured at the F5OS platform layer, and have been added to the tenant configuration by the administrator during provisioning. +Creating a VELOS tenant is nearly identical to creating a vCMP guest on VIPRION, with a few exceptions. When creating a VELOS tenant, you’ll provide a name, a TMOS image for the tenant to run, which slots (blades) the tenant will be configured to run on, out-of-band IP addressing/mask and gateway, and which VLANs the tenant should inherit. Just like a vCMP guest, the VLANs are configured at provision time and not within the tenant itself. The tenant will inherit VLANs that have been configured at the F5OS platform layer and have been added to the tenant configuration by the administrator during provisioning. .. image:: images/velos_multitenancy/image2.png :align: center @@ -58,7 +58,7 @@ Each BX110 blade has 28 vCPUs, however 6 of those vCPUs are dedicated to the F5O :align: center :scale: 70% -Single vCPU (Skinny) tenants are supported, but that option is hidden under **Advanced** mode. This would allow for 22 single vCPU tenants per BX110 blade. While single vCPUs guests are supported, they are not recommended for most environments. This is due to the fact that a single vCPU tenant is running on a single hyperthread, and performance of a single thread can be influenced by other services running on the other hyperthread of a CPU. Since this can lead to unpredictable behavior, only a very lightly loaded LTM/DNS-only type tenant should be considered for this option. As always proper sizing should be done to ensure the tenant has enough resources. +Single vCPU (Skinny) tenants are supported, but that option is hidden under **Advanced** mode. This would allow for 22 single vCPU tenants per BX110 blade. While single vCPUs guests are supported, they are not recommended for most environments. This is because a single vCPU tenant is running on a single hyperthread, and performance of a single thread can be influenced by other services running on the other hyperthread of a CPU. Since this can lead to unpredictable behavior, only a very lightly loaded LTM/DNS-only type tenant should be considered for this option. As always proper sizing should be done to ensure the tenant has enough resources. A VELOS tenant supports 3 states: (**Configured**, **Provisioned**, and **Deployed**): @@ -82,6 +82,6 @@ In some VIPRION blades, there is an option to configure an **SSL Mode** for vCMP If you currently utilize the SSL Mode feature where SSL resources can be **Dedicated, Shared, or Isolated** for each vCMP guest, this configuration option is not supported on VELOS at initial release. vCMP guests operate in the default shared mode, meaning all guests get equal access to the shared SSL hardware resources. You may configure the SSL Mode to **dedicated**, where SSL hardware resources are dedicated to a guest in proportion to the vCPUs assigned to a guest. You may also configure **none**, meaning all SSL processing is done in software. -In VELOS there is no **SSL Mode** configuration option. By default, you may configure the **Crypto/Compression Acceleration** option when deploying a VELOS tenant. The choices are **enabled** or **disabled**. When enabled, the system will assign SSL hardware resources in proportion to the number of vCPUs assigned to the tenant. This is conceptually similar to how SSL Mode **Dedicated** works on vCMP guests, but not 100% the same implementation.When disabled, no SSL hardware resources are assigned to the tenant, and all processing is done in software. A environment currently running in the default shared mode will now be running in a mode that essentially mimics the SSL Mode Dedicated. +In VELOS there is no **SSL Mode** configuration option. By default, you may configure the **Crypto/Compression Acceleration** option when deploying a VELOS tenant. The choices are **enabled** or **disabled**. When enabled, the system will assign SSL hardware resources in proportion to the number of vCPUs assigned to the tenant. This is conceptually like SSL Mode **Dedicated** works on vCMP guests, but not 100% the same implementation. When disabled, no SSL hardware resources are assigned to the tenant, and all processing is done in software. An environment currently running in the default shared mode will now be running in a mode that essentially mimics the SSL Mode Dedicated. Lastly, the tenant may be configured to support **Appliance Mode**, which is a security option that disables root and bash access to the tenant. diff --git a/docs/velos_networking.rst b/docs/velos_networking.rst index 4fca4b6..19cd7f4 100644 --- a/docs/velos_networking.rst +++ b/docs/velos_networking.rst @@ -38,7 +38,7 @@ Each chassis partition is a unique entity that has its own set of (local/remote) :align: center :scale: 50% -In addition to management access being completely isolated and unique, in-band networking (for use by tenants) is configured and completely contained within the chassis partition. Each chassis partition will have its own set of networking components such as; PortGroups, VLANs, LAGs, and interfaces. This means that networking within one chassis partition is not accessible or viewable from another chassis partition. +In addition to management access being completely isolated and unique, in-band networking (for use by tenants) is configured and completely contained within the chassis partition. Each chassis partition will have its own set of networking components such as PortGroups, VLANs, LAGs, and interfaces. This means that networking within one chassis partition is not accessible or viewable from another chassis partition. Isolation at the network level is also enforced via the centralized switch fabrics that reside in the dual system controllers. In the VELOS system each blade has multiple connections into the centralized switch fabrics for redundancy and added bandwidth. Each BX110 blade has 2 x 100Gb backplane connections (one to each system controller), that are bonded together in a static LAG (Link Aggregation Group). This star-wired topology provides fast and reliable backplane connections between all the blades, and also allows for complete isolation at the networking layer. @@ -69,7 +69,7 @@ To illustrate the point of how isolated chassis partitions are, the diagram belo Port Groups =========== -The portgroup component is used to control the mode of the physical port. This controls whether the port is bundled or unbundled, and the port speed. Both ports on the BX110 blade must be configured in the same mode currently. The term portgroup is used rather than simply “port” because some front panel ports may accept different types of SFPs. Depending on the portgroup mode value, a different FPGA version is loaded, and the speed of the port is adjusted accordingly. The user can modify the portgroup mode as needed through the F5OS CLI, webUI, or API. +The portgroup component is used to control the mode of the physical port. This controls whether the port is bundled or unbundled, and the port speed. The term portgroup is used rather than simply “port” because some front panel ports may accept different types of optics. Depending on the portgroup mode value, a different FPGA version is loaded, and the speed of the port is adjusted accordingly. The user can modify the portgroup mode as needed through the F5OS CLI, webUI, or API. .. image:: images/velos_networking/image9.png @@ -86,7 +86,7 @@ In releases prior to F5OS-C 1.5.1 both ports on a BX110 blade must be configured -Below is an example of the chassis partition webUI Port Groups screen. Note that any changes in configuration will require a reboot of the blade in order to load a new FPGA bitstream image. +Below is an example of the chassis partition webUI Port Groups screen. Note that any changes in configuration will require a reboot of the blade to load a new FPGA bitstream image. .. image:: images/velos_networking/image11.png :align: center @@ -161,7 +161,7 @@ Below are the current VELOS optic SKUs: | F5-UPG-VEL-QSFP+SR4 | CN | VELOS Field Upgrade: QSFP+ Transceiver (40G-SR4, 850NM, 100M, MPO, DDM Support) | +----------------------+------+---------------------------------------------------------------------------------------+ -The QSFP+ and QSFP28 optics when configured for unbundled mode, will break out into either 4 x 25Gb (with a 100Gb QSFP28 optic) or 4 x 10Gb (with a 40Gb QSFP+ optic). You will need to utilize a breakout cable to allow the single physical port to break out into 4 lower speed ports. The following breakout cable SKUs can be ordered and utilized for either 4 x 25Gb, or 4 x 10GB depending on the optic installed. Note, they come in different lengths (1 meter, 3 meter, or 10 meter) and each of the SKUs is a 2 Pack. +The QSFP+ and QSFP28 optics when configured for unbundled mode, will break out into either 4 x 25Gb (with a 100Gb QSFP28 optic) or 4 x 10Gb (with a 40Gb QSFP+ optic). You will need to utilize a breakout cable to allow the single physical port to break out into 4 lower speed ports. The following breakout cable SKUs can be ordered and utilized for either 4 x 25Gb, or 4 x 10GB depending on the optic installed. Note, they come in different lengths (1 meter, 3 meters, or 10 meters) and each of the SKUs is a 2 Pack. +---------------------+------+--------------------------------------------------------------------------------------------+ | F5-UPGVELSR4XSR3M | CN | VELOS Field Upgrade: QSFP28-QSFP+ Breakout Cable for SR4 ONLY MPO to 4LC (3 Meter 2 Pack) | diff --git a/docs/velos_performance_and_sizing.rst b/docs/velos_performance_and_sizing.rst index 30ce66a..1ad7168 100644 --- a/docs/velos_performance_and_sizing.rst +++ b/docs/velos_performance_and_sizing.rst @@ -18,7 +18,7 @@ In VELOS there are now two distinct FPGAs, the **Application Traffic Services En :align: center :scale: 70% -**Note: In the initial 1.x.x versions of F5OS-C, not all hardware offload functions were enabled. SSL/TLS offload, FASTL4, and compression were all supported, but some features like hardware DDoS mitigation were not, but have since been added in the subsequent TMOS v15.1.4/F5OS 1.2.1 releases. +**Note: In the initial 1.x.x versions of F5OS-C, not all hardware offload functions were enabled. SSL/TLS offload, FASTL4, and compression were all supported, but some features like hardware DDoS mitigation were not, but have since been added in the subsequent TMOS v15.1.4/F5OS 1.2.1 releases.** When comparing performance of VELOS to VIPRION C2400 it is important to note that VELOS is more scalable as it has double the number of slots (8 in VELOS compared to 4 in VIPRION), so the amount of performance in the same RU (Rack Unit) is significantly increased. The VELOS blades are half the width of the current B2250 blades in VIPRION. Looking at comparisons of VIPRION B2150 and B2250 against a single VELOS BX110 blade you can see a 1.5x-3x increase in Layer 7 Requests Per Second (RPS) performance depending on which blade you may be migrating from. From an SSL perspective the increase is 2.3x-10x for RSA based ciphers, and for Elliptical Curve the BX110 can now offload that to hardware, whereas the B2150 and B2250 had to process those ciphers in software consuming more CPU. @@ -85,7 +85,7 @@ Each VELOS BX110 blade has 28 vCPUs, but 6 of those vCPUs are reserved for use b :scale: 70% -When sizing, removing the 6 dedicated vCPUs from the equation will give a better representation of what the per vCPU performance will be. Comparing the performance of a single vCPU can be important for control plane sizing and also for extrapolation of what a tenant’s performance may be. Below is a comparison on the CPUs on the VIPRION B2250, VELOS BX110, and VIPRION B4450. Note that the VELOS sizing is more complex because of the way the CPU’s are used. Since 3 physical / 6 vCPUs are dedicated for use by the platform layer, overall CPU performance can be misleading. +When sizing, removing the 6 dedicated vCPUs from the equation will give a better representation of what the per vCPU performance will be. Comparing the performance of a single vCPU can be important for control plane sizing and for extrapolation of what a tenant’s performance may be. Below is a comparison on the CPUs on the VIPRION B2250, VELOS BX110, and VIPRION B4450. Note that the VELOS sizing is more complex because of the way the CPUs are used. Since 3 physical / 6 vCPUs are dedicated for use by the platform layer, overall CPU performance can be misleading. The graphs below compare 1 and 2 blade configurations of the B2250 vs. a single B4450 blade, and 1 and 2 blade VELOS BX110 configurations. There are comparisons which include all the vCPUs on a BX110, and another set which removes the 6 vCPUs used for the platform layer (more realistic). Instead of showing 14 physical cores and 28 vCPUs, VELOS is sized using 11 physical cores and 22 vCPUs (listed as "minus platform Layer CPU"). @@ -97,7 +97,7 @@ The graphs below compare 1 and 2 blade configurations of the B2250 vs. a single :align: center :scale: 70% -To compare performance of VIPRION against VELOS you can first look at overall CPU capacity of the system, and then break that down to per vCPU performance to get a fair comparison. In a typical sizing exercise, it is normal to look at the overall data sheet metric you are interested in divided by the total number of vCPUs in the system to come up with a per vCPU metric. Because VELOS dedicates some of its processing to the F5OS platform layer, we remove them from the overall sizing metric so that numbers don’t get skewed. As an example, take the overall BX110 blade performance metrics then divide by the total vCPUs on the blades minus the 6 vCPUs for the platform layer (divide by 22). In the past some have used total or aggregate CPU Ghz as a means of comparing different platforms. This may work well when comparing similar generation and architecture platforms, but may not be the best metric given the changes in VELOS. VELOS has more modern processors, which are more efficient and can boost to higher rates than previous generation processors so looking at aggregate processor speed (total Ghz) only is not sufficient to get accurate sizing. +To compare performance of VIPRION against VELOS you can first look at overall CPU capacity of the system, and then break that down to per vCPU performance to get a fair comparison. In a typical sizing exercise, it is normal to look at the overall data sheet metric you are interested in divided by the total number of vCPUs in the system to come up with a per vCPU metric. Because VELOS dedicates some of its processing to the F5OS platform layer, we remove them from the overall sizing metric so that numbers don’t get skewed. As an example, take the overall BX110 blade performance metrics then divide by the total vCPUs on the blades minus the 6 vCPUs for the platform layer (divide by 22). In the past some have used total or aggregate CPU Ghz as a means of comparing different platforms. This may work well when comparing similar generation and architecture platforms but may not be the best metric given the changes in VELOS. VELOS has more modern processors, which are more efficient and can boost to higher rates than previous generation processors so looking at aggregate processor speed (total Ghz) only is not sufficient to get accurate sizing. In the past **Relative CPU Scale** was a numeric grade-based comparison where the overall CPU capacity/horsepower of the system was given a rating. The rating is an easy way to compare different BIG-IP platforms. The Relative CPU Scale is calculated by taking the total # of CPUs in a system (not including those used by VELOS platform layer) and multiplying that times the speed (Ghz) that the processors run. This will result in an aggregate CPU Ghz for the platform or blade. We then take the Aggregate CPU Ghz of a BIG-IP 2000s platform and give it a grade of 1. All other platforms are then given a numeric grade of how many times faster it is than the 2000s. This results in a simple numeric rating system that combines CPU speed with the number of CPUs without having explain Ghz. @@ -110,7 +110,7 @@ In the graph below you can see that a B2250 blade has 10 times more aggregate CP -To see how this really translates into performance, it is good to look at a Layer7 data sheeet metric as that is something that will use a lot of CPU resources. If you look at the per blade Layer7 (Inf-Inf) numbers, you’ll notice VELOS provides higher numbers than a B2250 even though its rating is lower in the chart above. This is likely due to the newer generation of processors, the fact that some processing is dedicated to the platform layer, and the fact that the CPUs can boost higher than previous generations. Generally, a BX110 blade is going to be faster than a B2250 blade (each metric will vary), but it’s safe to propose BX110 blades as direct replacements for B2250 blades. Also keep in mind BX110 has the latest Intel processing and crypto support so things like ECC ciphers are now accelerated in hardware, which was not the case with VIPRION B2xxx blades. +To see how this really translates into performance, it is good to look at a Layer7 data sheet metric as that is something that will use a lot of CPU resources. If you look at the per blade Layer7 (Inf-Inf) numbers, you’ll notice VELOS provides higher numbers than a B2250 even though its rating is lower in the chart above. This is likely due to the newer generation of processors, the fact that some processing is dedicated to the platform layer, and the fact that the CPUs can boost higher than previous generations. Generally, a BX110 blade is going to be faster than a B2250 blade (each metric will vary), but it’s safe to propose BX110 blades as direct replacements for B2250 blades. Also keep in mind BX110 has the latest Intel processing and crypto support so things like ECC ciphers are now accelerated in hardware, which was not the case with VIPRION B2xxx blades. Note a BX110 blade is not intended to replace a single B4450 blade. The B4450 has ~2 times the processing power and vCPU count of a BX110 blade. In most cases it would take 2 BX110 blades to handle the workload of a single B4450. @@ -119,7 +119,7 @@ Note a BX110 blade is not intended to replace a single B4450 blade. The B4450 ha :scale: 80% -Because each blade has a different number of CPUs, a common sizing exercise is to look at the per vCPU performance by using the formulas above to come up with a per vCPU metric. In the graph below it is done for Layer7 RPS (Inf-Inf) but you could use the same math for any metric. Note the graph below is not derived from a per vCPU test, it is taking a published blade metric and dividing it by the number of available vCPUs to come up with a per vCPU metric. As mentioned above using the VELOS metric which is (minus the platform CPUs) is the most realistic. As expected, the BX110 provides a better per vCPU Requests Per Second (RPS) than the B2250, but what may be surprising is that it has a higher RPS on a per vCPU basis than the B4450 as well. This is because the B4450 gets its overall speed due to the total number of vCPUs, and it has 2x more CPUs than the BX110. Even though the BX110 CPUs run slower (1.9Ghz vs. 2.2Ghz) than the B4450, the changes in architecture, more modern CPU etc., make it perform faster at the per vCPU metric. +Because each blade has a different number of CPUs, a common sizing exercise is to look at the per vCPU performance by using the formulas above to come up with a per vCPU metric. In the graph below it is done for Layer7 RPS (Inf-Inf), but you could use the same math for any metric. Note, the graph below is not derived from a per vCPU test, it is taking a published blade metric and dividing it by the number of available vCPUs to come up with a per vCPU metric. As mentioned above, using the VELOS metric which is (minus the platform CPUs) is the most realistic. As expected, the BX110 provides a better per vCPU Requests Per Second (RPS) than the B2250, but what may be surprising is that it has a higher RPS on a per vCPU basis than the B4450 as well. This is because the B4450 gets its overall speed due to the total number of vCPUs, and it has 2x more CPUs than the BX110. Even though the BX110 CPUs run slower (1.9Ghz vs. 2.2Ghz) than the B4450, the changes in architecture, more modern CPU etc., make it perform faster at the per vCPU metric. .. image:: images/velos_performance_and_sizing/image15.png :align: center @@ -133,7 +133,7 @@ Also consider that these extrapolations for the VIPRION blades are for bare meta Memory Sizing ============= -Each VELOS BX110 blade has 128GB of memory, which is double the current memory support of the B2250 blade (64GB) but half the current B4450 blade (256GB). Generally, a BX110 will have more than enough memory to replace a B2250 blade and will actually provide more memory which may help vCMP guests which are pushing memory limits. Just like sizing based on L7 it will likely take 2 BX110 blades to replace a B4450 blade when looking at memory only. +Each VELOS BX110 blade has 128GB of memory, which is double the current memory support of the B2250 blade (64GB) but half the current B4450 blade (256GB). Generally, a BX110 will have more than enough memory to replace a B2250 blade and will provide more memory which may help vCMP guests which are pushing memory limits. Just like sizing based on L7 it will likely take 2 BX110 blades to replace a B4450 blade when looking at memory only. .. image:: images/velos_performance_and_sizing/image16.png :align: center diff --git a/docs/velos_points_of_management.rst b/docs/velos_points_of_management.rst index 7747ee0..2631d70 100644 --- a/docs/velos_points_of_management.rst +++ b/docs/velos_points_of_management.rst @@ -9,14 +9,14 @@ There are three main points of management within the VELOS chassis: the **system :scale: 90% -Additionally, they each run their own version of software; tenants are able to run specific versions of TMOS which have been approved to run on the VELOS platform, and system controllers and chassis partitions each have their own version of F5OS-C software. The supported TMOS tenant versions are; 14.1.4 and later, 15.1.4 and later, and 17.1.x and later. There are no plans to support versions 16.0.x, 16.1.x, or 17.0.x, and there are no plans to support versions prior to 14.1.4. The F5OS-C platform layer in VELOS runs its own version of F5OS, which is unique to the VELOS chassis. On downloads.f5.com, the VELOS versions of F5OS are referred to as F5OS-C, where the C stands for chassis. The rSeries appliances also run F5OS, but that version is designated as F5OS-A, where A stands for appliance. Most of the code and configration interfaces of F5OS are common between VELOS and rSeries, but VELOS has unique F5OS features that are chassis specific. VELOS has two layers of F5OS (system controller and chassis partition), and each of these have their own software images, in addtion to the tenants that run TMOS. +Additionally, they each run their own version of software; tenants are able to run specific versions of TMOS which have been approved to run on the VELOS platform, and system controllers and chassis partitions each have their own version of F5OS-C software. The supported TMOS tenant versions are 14.1.4 and later, 15.1.4 and later, and 17.1.x and later. There are no plans to support versions 16.0.x, 16.1.x, or 17.0.x, and there are no plans to support versions prior to 14.1.4. The F5OS-C platform layer in VELOS runs its own version of F5OS, which is unique to the VELOS chassis. On downloads.f5.com, the VELOS versions of F5OS are referred to as F5OS-C, where the C stands for chassis. The rSeries appliances also run F5OS, but that version is designated as F5OS-A, where A stands for appliance. Most of the code and configuration interfaces of F5OS are common between VELOS and rSeries, but VELOS has unique F5OS features that are chassis specific. VELOS has two layers of F5OS (system controller and chassis partition), and each of these have their own software images, in addition to the tenants that run TMOS. .. image:: images/velos_points_of_management/image2.png :align: center :scale: 90% -At the F5OS-C platform layer, initial configuration consists of out-of-band management IP addresses, routing, and other system parameters like DNS and NTP. Licensing, is also configured at the F5OS layer, and is similar to VIPRION with vCMP configured, in that it is applied at the chassis level and inherited by all tenants. All of these items are configured at the system controller layer. The administrator also configures **chassis partitions** (groupings of VELOS blades/slots), that each have their own management IP address, and CLI, GUI, and API interfaces. +At the F5OS-C platform layer, initial configuration consists of out-of-band management IP addresses, routing, and other system parameters like DNS and NTP. Licensing is also configured at the F5OS layer, and is similar to VIPRION with vCMP configured, in that it is applied at the chassis level and inherited by all tenants. All these items are configured at the system controller layer. The administrator also configures **chassis partitions** (groupings of VELOS blades/slots), that each have their own management IP address, and CLI, GUI, and API interfaces. Inside the chassis partition F5OS layer, interfaces, in-band networking (VLANs, interfaces, Link Aggregation Groups) are configured. Once networking is set up, tenants can be provisioned, and deployed from the F5OS chassis partition management interfaces. Once the tenant is deployed, it is managed like any other BIG-IP instance. This is very similar to how vCMP guests are managed on iSeries or VIPRION. Please refer to the **VELOS Systems Administration Guide** on askf5.com for more detailed information. @@ -25,13 +25,13 @@ Inside the chassis partition F5OS layer, interfaces, in-band networking (VLANs, Differences from iSeries and VIPRION ------------------------------------ -The management of VELOS/F5OS has a lot of similarities to how vCMP is managed on iSeries or VIPRION, in that there are two distinct layers of management. In the diagram below on the left, a typical vCMP environment has a **host** layer and a **guest** layer. At the vCMP host layer, all networking is configured including interfaces, trunks, and VLANs. When vCMP guests are configured they will be assinged a set of VLANs by the administrator that it will have access to. The administrator may not want to give the guest access to all VLANs in the system, and may only assign a small subset of VLANs from the host layer to a specific guest. Inside the TMOS layer of the guest does not require manual configuration of interfaces or trunks, it is the VLANs that are inhertited from the vCMP host configuration that will provide connectivity. The guest will only have access to the VLANs specifically assigned to it when it was created. On the right hand side is an F5OS environment (in this case VELOS), at the F5OS chassis partition platform layer, all networking is configured including interfaces, trunks (now called LAGs), and VLANs. When F5OS tenants are configured they are assinged VLANs by the administrator that they will have access to. Inside the tenant itself does not require configuration of interfaces or LAGs, and VLANs will be inhertited from the F5OS platform layer configuration. The F5OS tenant will only have access to the VLANs specifically assigned to it when it was created. +The management of VELOS/F5OS has a lot of similarities to how vCMP is managed on iSeries or VIPRION, in that there are two distinct layers of management. In the diagram below on the left, a typical vCMP environment has a **host** layer and a **guest** layer. At the vCMP host layer, all networking is configured including interfaces, trunks, and VLANs. When vCMP guests are configured, they will be assigned a set of VLANs by the administrator that it will have access to. The administrator may not want to give the guest access to all VLANs in the system and may only assign a small subset of VLANs from the host layer to a specific guest. Inside the TMOS layer of the guest does not require manual configuration of interfaces or trunks, it is the VLANs that are inherited from the vCMP host configuration that will provide connectivity. The guest will only have access to the VLANs specifically assigned to it when it was created. On the right-hand side is an F5OS environment (in this case VELOS), at the F5OS chassis partition platform layer, all networking is configured including interfaces, trunks (now called LAGs), and VLANs. When F5OS tenants are configured, they are assigned VLANs by the administrator that they will have access to. Inside the tenant itself does not require configuration of interfaces or LAGs, and VLANs will be inherited from the F5OS platform layer configuration. The F5OS tenant will only have access to the VLANs specifically assigned to it when it was created. .. image:: images/velos_points_of_management/image3.png :align: center :scale: 80% -Comparing the management of a non-VCMP (bare metal) iSeries or VIPRION to VELOS is going to be a little bit different with the introduction of the F5OS platform layer. With a bare-metal deployment on iSeries/VIPRION, configuration objects such as interfaces, trunks, and VLANs are directly configurable from within the TMOS layer. Monitoring of the lower layer networking can also be done within the TMOS layer. When moving to VELOS, the configuration and monitoring of the lower level networking objects are done at the F5OS platform layer. For SNMP monitoring there are separate SNMP MIBs for the F5OS layer that can be used to monitor interfaces and platform level statistics. F5OS doesn't use the term trunk to represent aggregated links, it uses the term Link Aggregation Group or LAG. There are also F5OS API's to monitor and configure the platform layer. The F5OS tenants themselves still support monitoring of higher layers. +Comparing the management of a non-VCMP (bare metal) iSeries or VIPRION to VELOS is going to be a little bit different with the introduction of the F5OS platform layer. With a bare-metal deployment on iSeries/VIPRION, configuration objects such as interfaces, trunks, and VLANs are directly configurable from within the TMOS layer. Monitoring of the lower layer networking can also be done within the TMOS layer. When moving to VELOS, the configuration and monitoring of the lower-level networking objects are done at the F5OS platform layer. For SNMP monitoring there are separate SNMP MIBs for the F5OS layer that can be used to monitor interfaces and platform level statistics. F5OS doesn't use the term trunk to represent aggregated links, it uses the term Link Aggregation Group or LAG. There are also F5OS APIs to monitor and configure the platform layer. The F5OS tenants themselves still support monitoring of higher layers. .. image:: images/velos_points_of_management/image4.png diff --git a/docs/velos_security.rst b/docs/velos_security.rst index f51e148..1f8511e 100644 --- a/docs/velos_security.rst +++ b/docs/velos_security.rst @@ -23,9 +23,9 @@ Allow List for F5OS Management F5OS only allows management access via the out-of-band management interfaces on VELOS, there is no in-band access to the F5OS management layer. Within VELOS there are two layers for F5OS; the **system controller** layer, and the **chassis partition** layer. Each of these layers have their own management IP addresses, and access control which can restrict access through the out-of-band network. -Each of the two system controllers has a static IP address assigned, and there is a floating IP address which should be use to access the active system controller. As chassis partitions are deployed they also have a single IP address which is assigned. Access to those F5OS management interfaces may be restricted to specific IP addresses (both IPv4 and IPv6), subnets (via Prefix Length), as well as protocols - 443 (HTTPS), 80 (HTTP), 8888 (RESTCONF), 161 (SNMP), 7001 (VCONSOLE), and 22 (SSH) with version F5OS-C 1.6.0 and later. An administrator can add one or more Allow List entries via the CLI, webUI (webUI will be added in F5OS-C 1.7.0) or API at the system controller layer and the chassis partition layer to lock down access to specific endpoints. +Each of the two system controllers has a static IP address assigned, and there is a floating IP address which should be used to access the active system controller. As chassis partitions are deployed, they also have a single IP address which is assigned. Access to those F5OS management interfaces may be restricted to specific IP addresses (both IPv4 and IPv6), subnets (via Prefix Length), as well as protocols - 443 (HTTPS), 80 (HTTP), 8888 (RESTCONF), 161 (SNMP), 7001 (VCONSOLE), and 22 (SSH) with version F5OS-C 1.6.0 and later. An administrator can add one or more Allow List entries via the CLI, webUI (webUI will be added in F5OS-C 1.7.0) or API at the system controller layer and the chassis partition layer to lock down access to specific endpoints. -By default, all ports except for 161 (SNMP) are enabled for access, meaning ports 80, 443, 8888, 7001, and 22 are allowed access. Port 80 is only open to allow a redirect to port 443 in case someone tries to access the webUI over port 80. The webUI itself is not accessible over port 80. Port 161 is typically viewed as un-secure, and is therefore not accessible until an allow list entry is created for the endpoint trying to access F5OS using SNMP queries. Ideally SNMPv3 should be utilized to provide additional layers of security on an otherwise un-secure protocol. VCONSOLE access also has to be explicitly configured before access to the tenants is possible over port 7001. +By default, all ports except for 161 (SNMP) are enabled for access, meaning ports 80, 443, 8888, 7001, and 22 are allowed access. Port 80 is only open to allow a redirect to port 443 in case someone tries to access the webUI over port 80. The webUI itself is not accessible over port 80. Port 161 is typically viewed as un-secure and is therefore not accessible until an allow list entry is created for the endpoint trying to access F5OS using SNMP queries. Ideally SNMPv3 should be utilized to provide additional layers of security on an otherwise un-secure protocol. VCONSOLE access also must be explicitly configured before access to the tenants is possible over port 7001. To further lock down access you may add an Allow List entry including an IP address and optional prefix for each of the protocols listed above. As an example, if you wanted to restrict API and webUI access to a particular IP address and/or subnet, you can add an Allow List entry for the desired IP or subnet (using the prefix length), specify port 443 and all access from other IP endpoints will be prevented. @@ -34,7 +34,7 @@ The examples below can be applied at either the system controller layer logging Adding Allow List Entries via CLI ----------------------------------- -If you would like to lock down one of the protocols to either a single IP address or subnet, use the **system allowed-ips** command. Be sure to commit any changes. The **prefix-length** parameter is optional. If you omit it, then you will lock down access to a specific IP endpoint, if you add it you can lock down access to a specific subnet. +If you would like to lock down one of the protocols to either a single IP address or subnet, use the **system allowed-ips** command. Be sure to commit any changes. The **prefix-length** parameter is optional. If you omit it, then you will lock down access to a specific IP endpoint, if you add it, you can lock down access to a specific subnet. .. code-block:: bash @@ -43,7 +43,7 @@ If you would like to lock down one of the protocols to either a single IP addres Commit complete. syscon-2-active(config-allowed-ip-snmp)# -Currently you can add one IP address/port pair per **allowed-ip** name with an optional prefix length to specify a CIDR block containing multiple addresses. If you require more than one non-contiguous IP address or subnets you can add it under another name as seen below. +Currently you can add one IP address/port pair per **allowed-ip** name with an optional prefix length to specify a CIDR block containing multiple addresses. If you require more than one non-contiguous IP address or subnets, you can add it under another name as seen below. .. code-block:: bash @@ -110,7 +110,7 @@ Within the body of the API call, specific IP address/port, and optional prefix-l -To view the allowed IP's in the API, use the following call. +To view the allowed IPs in the API, use the following call. .. code-block:: bash @@ -209,7 +209,7 @@ Note that the hash key can be used to check and compare the status of the primar Certificates for Device Management ================================== -F5OS supports TLS device certificates and keys to secure connections to the management interface. You can either create a self-signed certificate, or load your own certificates and keys into the system. In F5OS-C 1.6.0 an admin can now optionally enter a passphrase with the encrypted private key. More details can be found in the link below. +F5OS supports TLS device certificates and keys to secure connections to the management interface. You can either create a self-signed certificate or load your own certificates and keys into the system. In F5OS-C 1.6.0 an admin can now optionally enter a passphrase with the encrypted private key. More details can be found in the link below. `VELOS Certificate Management Overview `_ @@ -217,7 +217,7 @@ F5OS supports TLS device certificates and keys to secure connections to the mana Managing Device Certificates, Keys, CSRs, and CAs via CLI -------------------------------------------------------- -By default, F5OS uses a self-signed certificate and key for device management. If you would like to create your own private key and self-signed certificate use the following CLI command: +By default, F5OS uses a self-signed certificate and key for device management. If you would like to create your own private key and self-signed certificate, use the following CLI command: .. code-block:: bash @@ -366,7 +366,7 @@ The screen below shows the options when creating a self-signed certificate. :align: center :scale: 70% -If you choose the **Store TLS** option of **False** then the certiicate details will be displayed, and you will be given the option to copy them to the clipboard. If you want to store them on the system, then set the **Store TLS** option to **True**. +If you choose the **Store TLS** option of **False** then the certificate details will be displayed, and you will be given the option to copy them to the clipboard. If you want to store them on the system, then set the **Store TLS** option to **True**. .. image:: images/velos_security/imagecert4.png :align: center @@ -382,7 +382,7 @@ You can then use the **Show** options to display the current certificate, key, a :align: center :scale: 70% -You can also create a Certificate Signing Request (CSR) for the self-signed certificate for use when submitting the certificate to the Certificate Authourity (CA). +You can also create a Certificate Signing Request (CSR) for the self-signed certificate for use when submitting the certificate to the Certificate Authority (CA). .. image:: images/velos_security/imagecsr1.png :align: center @@ -394,7 +394,7 @@ After clicking **Save** the CSR will appear, and you will be able to **Copy to C :align: center :scale: 70% -When you install an SSL certificate on the system, you also install a certificate authority (CA) bundle, which is a file that contains root and intermediate certificates. The combination of these two files complete the SSL chain of trust. +When you install an SSL certificate on the system, you also install a certificate authority (CA) bundle, which is a file that contains root and intermediate certificates. The combination of these two files completes the SSL chain of trust. .. image:: images/velos_security/imageca1.png :align: center @@ -456,7 +456,7 @@ In the body of the API call enter the following JSON syntax. Encrypt Management TLS Private Key ======================= -Previously, F5OS allowed an admin to import a TLS certificate and key in clear text. In F5OS-C 1.6.0 an admin can now optionally enter a passphrase with the encrypted private key. This is similar to the BIG-IP functionality defined in the link below. +Previously, F5OS allowed an admin to import a TLS certificate and key in clear text. In F5OS-C 1.6.0 an admin can now optionally enter a passphrase with the encrypted private key. This is like the BIG-IP functionality defined in the link below. `K14912: Adding and removing encryption from private SSL keys (11.x - 16.x) `_ @@ -464,7 +464,7 @@ Previously, F5OS allowed an admin to import a TLS certificate and key in clear t Appliance Mode for F5OS ======================= -If you would like to prevent root / bash level access to the F5OS layer, you can enable **Appliance Mode**, which operates in a similar manner as TMOS appliance mode. Both the F5OS-C system controller and chassis partition layers have a setting where appliance mode can be enabled. Enabling Appliance mode will disable the root account, and access to the underlying bash shell is disabled. The admin account to the F5OS CLI is still enabled. This is viewed as a more secure setting as many vulnerabilities can be avoided by not allowing access to the bash shell. In some heavily audited environments, this setting may be mandatory, but it may prevent lower level debugging from occurring directly in the bash shell. It can be disabled on a temporary basis to do advanced troubleshooting, and then re-enabled when finished. +If you would like to prevent root / bash level access to the F5OS layer, you can enable **Appliance Mode**, which operates in a similar manner as TMOS appliance mode. Both the F5OS-C system controller and chassis partition layers have a setting where appliance mode can be enabled. Enabling Appliance mode will disable the root account, and access to the underlying bash shell is disabled. The admin account to the F5OS CLI is still enabled. This is viewed as a more secure setting as many vulnerabilities can be avoided by not allowing access to the bash shell. In some heavily audited environments, this setting may be mandatory, but it may prevent lower-level debugging from occurring directly in the bash shell. It can be disabled on a temporary basis to do advanced troubleshooting, and then re-enabled when finished. Enabling Appliance Mode via the CLI ----------------------------------- @@ -510,14 +510,14 @@ Appliance mode can be enabled or disabled via the webUI under the **System Setti Enabling Appliance Mode via the API ----------------------------------- -Appliance mode can be enabled or disabled via the API. To view the current status of appliance mode use the following API call. +Appliance mode can be enabled or disabled via the API. To view the current status of appliance mode, use the following API call. .. code-block:: bash GET https://{{velos_chassis1_system_controller_ip}}:8888/restconf/data/openconfig-system:system/f5-security-appliance-mode:appliance-mode -You will see output similar to the response below showing the config and state of appliance mode for F5OS. +You will see output like the response below showing the config and state of appliance mode for F5OS. .. code-block:: json @@ -551,13 +551,13 @@ In the body of the API call add the following: Session Timeouts and Token Lifetime =================================== -Idle timeouts were configurable in previous releases, but the configuration only applied to the current session and was not persistent. F5OS-A 1.3.0 added the ability to configure persistent idle timeouts for F5OS for both the CLI and webUI. The F5OS CLI timeout is configured under system settings, and is controlled via the **idle-timeout** option. This will logout idle sessions to the F5OS CLI whether they are logged in from the console or over SSH. +Idle timeouts were configurable in previous releases, but the configuration only applied to the current session and was not persistent. F5OS-A 1.3.0 added the ability to configure persistent idle timeouts for F5OS for both the CLI and webUI. The F5OS CLI timeout is configured under system settings and is controlled via the **idle-timeout** option. This will logout idle sessions to the F5OS CLI whether they are logged in from the console or over SSH. In F5OS-A 1.4.0, a new **sshd-idle-timeout** option has been added that will control idle-timeouts for both root sessions to the bash shell over SSH, as well as F5OS CLI sessions over SSH. When the idle-timeout and sshd-idle-timeout are both configured, the shorter interval should take precedence. As an example, if the idle-timeout is configured for three minutes, but the sshd-idle-timeout is set to 2 minutes, then an idle connection that is connected over SSH will disconnect in two minutes, which is the shorter of the two configured options. An idle connection to the F5OS CLI over the console will disconnect in three minutes, because the sshd-idle-timeout doesn't apply to console sessions. There is one case that is not covered by either of the above idle-timeout settings. When connecting over the console to the bash shell as root, neither of these settings will disconnect an idle session. Only console connections to the F5OS CLI are covered via the idle-timeout setting. An enhancement has been filed, and in the future this case will be addressed. If this is a concern, then appliance mode could be enabled preventing root/bash access to the system. -For the webUI, a token based timeout is now configurable under the **system aaa** settings. A restconf-token config lifetime option has been added. Once a client to the webUI has a token they are allowed to refresh it up to five times. If the token lifetime is set to 1 minute, then a timeout won't occur until five times that value, or five minutes later. This is because the token refresh has to fail five times before disconnecting the client. +For the webUI, a token-based timeout is now configurable under the **system aaa** settings. A restconf-token config lifetime option has been added. Once a client to the webUI has a token, they are allowed to refresh it up to five times. If the token lifetime is set to 1 minute, then a timeout won't occur until five times that value, or five minutes later. This is because the token refresh has to fail five times before disconnecting the client. Configuring SSH and CLI Timeouts via CLI ----------------------------------------- @@ -646,7 +646,7 @@ Currently only the HTTPS token lifetime is configurable in the webUI. SSH and CL Token Lifetime via CLI ---------------------- -As mentioned in the introduction, the webUI and API use token based authentication and the timeout is based on five token refreshes failing, so the value is essentially five times the configured token lifetime. Use the command **system aaa restconf-token config lifetime ** to set the token lifetime. You may configure the restconf-token lifetime via the CLI. The value is in minutes, and the client is able to refresh the token five times before it expires. As an example, if the restconf-token lifetime is set to 1 minute, an inactive webUI session will have a token expire after one minute, but it can be refreshed a maximum of five times. This will result in a webUI session or API timing out after 5 minutes. +As mentioned in the introduction, the webUI and API use token-based authentication and the timeout is based on five token refreshes failing, so the value is essentially five times the configured token lifetime. Use the command **system aaa restconf-token config lifetime ** to set the token lifetime. You may configure the restconf-token lifetime via the CLI. The value is in minutes, and the client can refresh the token five times before it expires. As an example, if the restconf-token lifetime is set to 1 minute, an inactive webUI session will have a token expire after one minute, but it can be refreshed a maximum of five times. This will result in a webUI session or API timing out after 5 minutes. .. code-block:: bash @@ -733,7 +733,7 @@ To display the current restconf-token lifetime setting, use the command **show s Token Lifetime via webUI ------------------------ -You may configure the restconf-token lifetime via the webUI (new feature added in F5OS-A 1.6.0). The value is in minutes, and the client is able to refresh the token five times before it expires. As an example, if the token lifetime is set to 1 minute, an inactive webUI session will have a token expire after one minute, but it can be refreshed a maximum of five times. This will result in the webUI session timing out after 5 minutes. +You may configure the restconf-token lifetime via the webUI (new feature added in F5OS-A 1.6.0). The value is in minutes, and the client can refresh the token five times before it expires. As an example, if the token lifetime is set to 1 minute, an inactive webUI session will have a token expire after one minute, but it can be refreshed a maximum of five times. This will result in the webUI session timing out after 5 minutes. .. image:: images/velos_security/image6.png :align: center @@ -742,7 +742,7 @@ You may configure the restconf-token lifetime via the webUI (new feature added i Token Lifetime via API ---------------------- -You may configure the restconf-token lifetime via the API. The value is in minutes, and the client is able to refresh the token five times before it expires. As an example, if the token lifetime is set to 1 minute, an inactive webUI session or API session will have a token expire after one minute, but it can be refreshed a maximum of five times. This will result in the webUI session timing out after 5 minutes. +You may configure the restconf-token lifetime via the API. The value is in minutes, and the client can refresh the token five times before it expires. As an example, if the token lifetime is set to 1 minute, an inactive webUI session or API session will have a token expire after one minute, but it can be refreshed a maximum of five times. This will result in the webUI session timing out after 5 minutes. Use the following API PATCH call to set the restconf-token lifetime, or any other password policy parameter. @@ -793,7 +793,7 @@ In the body of the API call adjust the restconf-token lifetime setting to the de Disabling Basic Authentication ============================== -F5OS utilizes basic authentication (username/password) as well as token based authentication for both the API and the webUI. Generally, username/password is issued by the client in order to obtain a token from F5OS, which is then used to make further inquiries or changes. Tokens have a relatively short lifetime for security reasons, and the user is allowed to refresh that token a certain number of times before they are forced to re-authenticate using basic authentication again. Although token based authentication is supported, basic authentication can still be utilized to access F5OS and make changes by default. A new option was added in F5OS-A 1.3.0 to allow basic authentication to be disabled, except for the means of obtaining a token. Once a token is issued to a client, it will be the only way to make changes via the webUI or the API. +F5OS utilizes basic authentication (username/password) as well as token-based authentication for both the API and the webUI. Generally, username/password is issued by the client to obtain a token from F5OS, which is then used to make further inquiries or changes. Tokens have a relatively short lifetime for security reasons, and the user is allowed to refresh that token a certain number of times before they are forced to re-authenticate using basic authentication again. Although token-based authentication is supported, basic authentication can still be utilized to access F5OS and make changes by default. A new option was added in F5OS-A 1.3.0 to allow basic authentication to be disabled, except for the means of obtaining a token. Once a token is issued to a client, it will be the only way to make changes via the webUI or the API. Disabling Basic Auth via the CLI @@ -964,7 +964,7 @@ In the body of the API call adjust the restconf-token:basic setting to **true** Disabling Basic Auth via the webUI ---------------------------------- -Disabling basic authentication via the webUI is a new feature that has been added in F5OS-A 1.4.0. In the webUI go to **User Management -> Authentication Settings** and you'll see a drop down box to enable or disable **Basic Authentication**. +Disabling basic authentication via the webUI is a new feature that has been added in F5OS-A 1.4.0. In the webUI go to **User Management -> Authentication Settings** and you'll see a drop-down box to enable or disable **Basic Authentication**. .. image:: images/velos_security/image5.png :align: center @@ -973,7 +973,7 @@ Disabling basic authentication via the webUI is a new feature that has been adde Confirming Basic Auth is Disallowed ----------------------------------- -With basic authentication enabled (default setting), you can make any API call using username/password (basic auth) autentication. Using the Postman utility this can be demonstrated on any configuration change by setting The Auth Type to **Basic Auth**, and configuring a username and password as seen below. +With basic authentication enabled (default setting), you can make any API call using username/password (basic auth) authentication. Using the Postman utility this can be demonstrated on any configuration change by setting The Auth Type to **Basic Auth**, and configuring a username and password as seen below. .. image:: images/velos_security/imagebasicauth.png :align: center @@ -996,7 +996,7 @@ While basic auth is enabled, any API call using username/password will complete } } -When basic authentication is enabled, a client will be allowed to obtain an auth token using username/password at any URI. The client can then choose to use the auth token for subsequent requests, or they can continue to use basic auth (username/password) authenticaton. As an example, the curl command below uses basic auth successfully to the URI endpoint **restconf/data/openconfig-system:system/config**. In the response you can see the **X-Auth-Token** header, which contains the auth token that can then be used by the client for subsequent requests: +When basic authentication is enabled, a client will be allowed to obtain an auth token using username/password at any URI. The client can then choose to use the auth token for subsequent requests, or they can continue to use basic auth (username/password) authentication. As an example, the curl command below uses basic auth successfully to the URI endpoint **restconf/data/openconfig-system:system/config**. In the response you can see the **X-Auth-Token** header, which contains the auth token that can then be used by the client for subsequent requests: .. code-block:: bash @@ -1055,7 +1055,7 @@ Here is an example of the client issuing the same request with the auth token it } user1$ -If the same exercise is repeated after basic auth is disabled, then the user will not be able to run the intial request using basic auth (username/password). It will fail to any non-root URI as seen below. The response will contain and **access-denied** error. +If the same exercise is repeated after basic auth is disabled, then the user will not be able to run the initial request using basic auth (username/password). It will fail to any non-root URI as seen below. The response will contain and **access-denied** error. .. code-block:: bash @@ -1086,7 +1086,7 @@ If the same exercise is repeated after basic auth is disabled, then the user wil } user1$ -By changing the URI to use the top level API endpoint: (:8888/restconf/data) or (:443/api/data), the client will now be able to obtain a token using basic authentication, but the token will be needed for any other API endpoints. +By changing the URI to use the top-level API endpoint: (:8888/restconf/data) or (:443/api/data), the client will now be able to obtain a token using basic authentication, but the token will be needed for any other API endpoints. .. code-block:: bash @@ -1184,7 +1184,7 @@ Local Password Policies can be set in the **User Management -> Authentication Se Setting Password Policies via API --------------------------------- -Local Password Policies can be viewed or set via the API using the following API calls. To view the current password policy settings issue the following GET API call. +Local Password Policies can be viewed or set via the API using the following API calls. To view the current password policy settings, issue the following GET API call. .. code-block:: bash @@ -1264,7 +1264,7 @@ In the payload of the API call adjust the appropriate parameters under **f5-open Remote Authentication ===================== -The F5OS platform layer supports both local and remote authentication. By default, there are local users enabled for both admin and root access. You will be forced to change passwords for both of these accounts on initial login. Many users will prefer to configure the F5OS layer to use remote authentication via LDAP, RADIUS, AD, or TACACS+. The F5OS TMOS based tenants maintain their own local or remote authentication, and details are covered in standard TMOS documentation. +The F5OS platform layer supports both local and remote authentication. By default, there are local users enabled for both admin and root access. You will be forced to change passwords for both accounts on initial login. Many users will prefer to configure the F5OS layer to use remote authentication via LDAP, RADIUS, AD, or TACACS+. The F5OS TMOS based tenants maintain their own local or remote authentication, and details are covered in standard TMOS documentation. `Configuring Remote User Authentication and Authorization on TMOS `_ @@ -1345,7 +1345,7 @@ To view the current mappings use the **show system aaa authentication roles** CL Login Banner / Message of the Day =================== -Some environments require warning or acceptance messages to be displayed to clients connecting to the F5OS layer at initial connection time and/or upon successful login. The F5OS layer supports configurable Message of the Day (MoTD) and Login Banners that are displayed to clients connecting to the F5OS layer via both CLI and the webUI. The MoTD and Login Banner can be configured via CLI, webUI, or API. The Login Banner is displayed at initial connect time and is commonly used to notify users they are connecting to a specific resource, and that they should not connect if they are not authorized. The MoTD is displayed after successful login, and may also display some information about the resource the user is connecting to. +Some environments require warning or acceptance messages to be displayed to clients connecting to the F5OS layer at initial connection time and/or upon successful login. The F5OS layer supports configurable Message of the Day (MoTD) and Login Banners that are displayed to clients connecting to the F5OS layer via both CLI and the webUI. The MoTD and Login Banner can be configured via CLI, webUI, or API. The Login Banner is displayed at initial connect time and is commonly used to notify users they are connecting to a specific resource, and that they should not connect if they are not authorized. The MoTD is displayed after successful login and may also display some information about the resource the user is connecting to. Configuring Login Banner / MoTD via CLI --------------------------------------- @@ -1476,7 +1476,7 @@ F5OS-A 1.2.0 added support for SNMPv3. Earlier versions of F5OS-A only supported NTP Authentication ================== -NTP Authentication can be enabled to provide a secure communication channel for Network Time Protocol queries from the F5OS platform layer. In order to utilize NTP authentication you must first enable NTP authentication and then add keys in order to secure communication to your NTP servers. +NTP Authentication can be enabled to provide a secure communication channel for Network Time Protocol queries from the F5OS platform layer. To utilize NTP authentication you must first enable NTP authentication and then add keys in order to secure communication to your NTP servers. Enabling NTP Authentication via CLI ----------------------------------- @@ -1816,7 +1816,7 @@ The API call should return output similar to what is seen below. Audit Logging ============ -F5OS has the ability to log all configuration changes and access to the F5OS layer in audit logs. In versions prior to F5OS-C 1.6.0, all access and configuration changes for the system controller layer are logged in one of two separate **audit.log** files. The files reside in the in one of the following paths in the F5OS filesystem when logged in as root; **/var/F5/controller/log/audit.log** or **/var/log/audit/audit.log**. If you are logged into the F5OS CLI as admin, then the actual paths are simplified to **log/controller/audit.log** and **/log/host/audit/audit.log** for the system controller layer. +F5OS can log all configuration changes and access to the F5OS layer in audit logs. In versions prior to F5OS-C 1.6.0, all access and configuration changes for the system controller layer are logged in one of two separate **audit.log** files. The files reside in the in one of the following paths in the F5OS filesystem when logged in as root; **/var/F5/controller/log/audit.log** or **/var/log/audit/audit.log**. If you are logged into the F5OS CLI as admin, then the actual paths are simplified to **log/controller/audit.log** and **/log/host/audit/audit.log** for the system controller layer. In versions prior to F5OS-C 1.6.0, all access and configuration changes for the chassis partition layer are logged in one of two separate **audit.log** files. The files reside in the in one of the following paths in the F5OS filesystem when logged in as root on the system controller; **/var/F5/partition/log/audit.log** or **/var/log/audit/audit.log**. If you are logged into the F5OS CLI as admin, then the actual paths are simplified to **log/audit.log** and **/log/host/audit/audit.log** for the system controller layer. @@ -1825,7 +1825,7 @@ In versions prior to F5OS-C 1.6.0, the audit.log files may only be viewed locall Viewing Audit Logs via F5OS CLI (F5OS-A 1.6.0 and Later) -------------------------------------------------------- -Any information related to login/logout or configuration changes are logged in the **log/controller/audit.log** location. By default these events are not sent to a configured remote syslog location. If you would like to send informational audit level messages to a remote syslog server, then you must explicitly enable audit events. +Any information related to login/logout or configuration changes are logged in the **log/controller/audit.log** location. By default, these events are not sent to a configured remote syslog location. If you would like to send informational audit level messages to a remote syslog server, then you must explicitly enable audit events. First, you must configure the remote syslog destination. As part of that configuration, you will specify the IP address, port, and protocol of the remote syslog server. To send audit.log events to the remote server you must add the command **selectors selector AUTHPRIV DEBUG** as seen below. @@ -1851,7 +1851,7 @@ Then, you can control the level of events that will be logged to the local audit config severity INFORMATIONAL ! -The formatting of audit logs provide the date/time in UTC, the account and ID who performed the action, the type of event, the asset affected, the type of access, and success or failure of the request. Separate log entries provide details on user access (login/login failures) information such as IP address and port and whether access was granted or not. +The formatting of audit logs provides the date/time in UTC, the account and ID who performed the action, the type of event, the asset affected, the type of access, and success or failure of the request. Separate log entries provide details on user access (login/login failures) information such as IP address and port and whether access was granted or not. Viewing Audit Logs via F5OS CLI @@ -2193,7 +2193,7 @@ Most audit events go to the **log/controller/audit.log** location, while a few o syscon-1-active# -To view the contents of the **audit.log** file, use the command **file show path /log/controller/audit.log**. This will show the entire log file from the beginning, but may not be the best way to troubleshoot a recent event. You can append **| more** to the command to go through the file in pages. +To view the contents of the **audit.log** file, use the command **file show path /log/controller/audit.log**. This will show the entire log file from the beginning but may not be the best way to troubleshoot a recent event. You can append **| more** to the command to go through the file in pages. .. code-block:: bash @@ -2232,7 +2232,7 @@ There are options to manipulate the output of the file. Add **| ?** to the comma until End with the line that matches syscon-1-active# file show log/controller/audit.log | -There are other file options that allow the user to tail the log file using **file tail -f** for a live tail, or **file tail -n ** to view a specific number of the most recent lines. +There are other file options that allow the user to tail the log file using **file tail -f** for a live tail or **file tail -n ** to view a specific number of the most recent lines. .. code-block:: bash @@ -2444,7 +2444,7 @@ Below is an example of a client logging out of the F5OS CLI. Note that the logs Account Lockout Audit Logs -------------------------- -In order to capture all events related to account lockout, you will need to configure F5OS to send both standard syslog events as well as host audit log events to a remote server. This is because some of the audit events related to account lockout are captured before the F5OS layer in the host/audit.log and by default that log is not sent remotely. +To capture all events related to account lockout, you will need to configure F5OS to send both standard syslog events as well as host audit log events to a remote server. This is because some of the audit events related to account lockout are captured before the F5OS layer in the host/audit.log and by default that log is not sent remotely. To forward the contents of the host audit logs add **config files file audit/audit.log** to the **system logging host-logs** configuration as seen below. @@ -2561,7 +2561,7 @@ Below is an example audit log of the user **jim-test** using the webUI and then Example Audit Logging of API Changes ------------------------------------ -Below is an example audit log of the user **admin** using the API and then adding a new VLAN to the configuration. In F5OS release prior to F5OS-A 1.4.0 API audit logs captured configuration changes, but did not log the full configuration payload. +Below is an example audit log of the user **admin** using the API and then adding a new VLAN to the configuration. In F5OS release prior to F5OS-A 1.4.0 API audit logs captured configuration changes but did not log the full configuration payload. .. code-block:: bash @@ -2579,7 +2579,7 @@ Below is an example audit log of the user **admin** using the API and then addin Downloading Audit Logs via CLI ------------------------------ -Audit logs can be sent to a remote server as outlined above, but they can also be downloaded from the system if needed. Before transferring a file using the CLI, use the **file list** command to see the contents of the directory and ensure the file is there. There are two audit.log locations: **log/system/audit.log** where most of the audit.log events are logged, and **log/host/audit/audit.log** where some lower level events are logged. +Audit logs can be sent to a remote server as outlined above, but they can also be downloaded from the system if needed. Before transferring a file using the CLI, use the **file list** command to see the contents of the directory and ensure the file is there. There are two audit.log locations: **log/system/audit.log** where most of the audit.log events are logged, and **log/host/audit/audit.log** where some lower-level events are logged. The path below is the main audit.log. @@ -2691,13 +2691,13 @@ In the response the latest file transfer status will be displayed. Downloading Audit Logs via webUI ------------------------------- -You can download either of the audit.log files from the **System -> File Utilities** page in the webUI. In the drop down menu for the **Base Directory** select log/host, and then you can select the audit directory as seen below. +You can download either of the audit.log files from the **System -> File Utilities** page in the webUI. In the drop-down menu for the **Base Directory** select log/host, and then you can select the audit directory as seen below. .. image:: images/velos_security/imageaudit1.png :align: center :scale: 70% -Inside the audit directory you can then select the audit.log and then either **Download** to copy the file to your local machine via the browser, or select **Export** to copy to a remote HTTPS server. +Inside the audit directory you can then select the audit.log and then either **Download** to copy the file to your local machine via the browser or select **Export** to copy to a remote HTTPS server. .. image:: images/velos_security/imageaudit2.png :align: center diff --git a/docs/velos_software_upgrades.rst b/docs/velos_software_upgrades.rst index b8cf22a..701ac6f 100644 --- a/docs/velos_software_upgrades.rst +++ b/docs/velos_software_upgrades.rst @@ -7,9 +7,9 @@ F5OS-C System Controller Upgrades The system controllers are fully redundant, however during software upgrades there can be outages of the entire chassis with the initial 1.1.x releases of F5OS. v1.2.x versions of F5OS-C have introduced a rolling upgrade capability for the system controller upgrade process, which minimizes disruption to the chassis. The chassis must be already running a version of F5OS 1.2.x or later to take advantage of this capability. Upgrades from 1.1.x versions to a 1.2.x version will not see rolling upgrade functionality. -This means that both system controllers will be updated at the same time thus causing an outage for all services within that chassis when running v1.1.x F5OS versions. For this reason, it is recommended you upgrade the system controllers during outage window and failover all services to the other chassis that is paired with the one you’re upgrading. For 1.2.x and later upgrades of F5OS-C on the system controllers, a rolling upgrade occurs where the standby controller is upgraded first, and when completed it will go to an acitve state, and the remaining controller will be upgraded. +This means that both system controllers will be updated at the same time thus causing an outage for all services within that chassis when running v1.1.x F5OS versions. For this reason, it is recommended you upgrade the system controllers during outage window and failover all services to the other chassis that is paired with the one you’re upgrading. For 1.2.x and later upgrades of F5OS-C on the system controllers, a rolling upgrade occurs where the standby controller is upgraded first, and when completed it will go to an active state, and the remaining controller will be upgraded. -When upgrading the system controllers, you will have a choice of upgrading either a bundled release, meaning **OS** and **Services** are **bundled** together in an ISO image, or **unbundled** where you can upgrade service and/or OS independently. F5 recommends using the bundled/ISO option for upgrades at this time. In the future, unbundled options may be utilized for some upgrades. +When upgrading the system controllers, you will have a choice of upgrading either a bundled release, meaning **OS** and **Services** are **bundled** together in an ISO image, or **unbundled** where you can upgrade service and/or OS independently. F5 recommends using the bundled/ISO option for upgrades currently. In the future, unbundled options may be utilized for some upgrades. .. image:: images/velos_software_upgrades/image1.png :align: center @@ -196,7 +196,7 @@ A response like the one below will provide the status of the transfer: } } -After transferring the file you can view the contents of the images/staging directory. The file will then go through an import process before it is ready for use. +After transferring the file, you can view the contents of the images/staging directory. The file will then go through an import process before it is ready for use. .. code-block:: bash @@ -234,7 +234,7 @@ You can then monitor the images/import/iso directory to see when the file is rea "f5-utils-file-transfer:path": "images/import/iso" } -You’ll see output similar to the example below. Once the file shows up here you are ready to upgrade. +You’ll see output like the example below. Once the file shows up here you are ready to upgrade. .. code-block:: json @@ -455,7 +455,7 @@ To upgrade the system controllers via the API you must first run the check versi "f5-system-controller-image:iso-version": "{{Controller_ISO_Image_Full}}" } -If the compatability check passes then you will get a message like the one below, and it is safe to install the new image via the set-version API call: +If the compatibility check passes then you will get a message like the one below, and it is safe to install the new image via the set-version API call: .. code-block:: json @@ -495,7 +495,7 @@ Chassis Partition Upgrades Upgrading a Chassis Partition via the webUI ----------------------------------------- -Upgrade of chassis partitions is performed from the system controller webUI **Partition Management** screen. You must first select the checkmark next to the chassis partition you wish to upgrade and then click **Edit**. You’ll now be able perform either a **bundled** or **unbundled** software upgrade of the chassis partition. At this time a bundled upgrade using the ISO image is recommended. Once you click **Save**, the partition will be brought offline and back online after the upgrade. All tenants will be suspended during this time so an outage will occur for this chassis partition only. +Upgrade of chassis partitions is performed from the system controller webUI **Partition Management** screen. You must first select the checkmark next to the chassis partition you wish to upgrade and then click **Edit**. You’ll now be able perform either a **bundled** or **unbundled** software upgrade of the chassis partition. Currently, a bundled upgrade using the ISO image is recommended. Once you click **Save**, the partition will be brought offline and back online after the upgrade. All tenants will be suspended during this time so an outage will occur for this chassis partition only. .. image:: images/velos_software_upgrades/image8.png :align: center @@ -626,7 +626,7 @@ To upgrade a chassis partition via the API you must first run the check version } } -If the compatability check passes then you will get a message like the one below, and it is safe to install the new image via the set-version API call: +If the compatibility check passes then you will get a message like the one below, and it is safe to install the new image via the set-version API call: .. code-block:: json @@ -752,7 +752,7 @@ To copy a tenant image into a chassis partition, use the following API call to t ] } -To list the current tenant images available on the chassis partition use the following API Call: +To list the current tenant images available on the chassis partition, use the following API Call: .. code-block:: bash @@ -783,7 +783,7 @@ Below is output generated from the previous command: Tenant Upgrades --------------- -Tenants are upgraded via the normal TMOS upgrade process. Find the proper ISO image and ensure it is of a supported VELOS release, and upload it into the TMOS tenant. Once uploaded you can upgrade and boot into the new version. Currently VELOS does not allow an upgrade of the tenant from inside the F5OS layer, you must perform the upgrade from inside the tenant. +Tenants are upgraded via the normal TMOS upgrade process. Find the proper ISO image and ensure it is of a supported VELOS release and upload it into the TMOS tenant. Once uploaded you can upgrade and boot into the new version. Currently VELOS does not allow an upgrade of the tenant from inside the F5OS layer, you must perform the upgrade from inside the tenant. **NOTE: Currently VELOS does not provide a shared image repository for all tenants to upgrade from. With vCMP guests, VIPRION allowed for an image to be loaded once into the host layer, and all tenants had access to that repository to use to upgrade.**