Skip to content

Commit

Permalink
Update DASH HLD to Rev 2.2 (#1791)
Browse files Browse the repository at this point in the history 
Restrict allowed values for the routing_type field in DASH_ROUTE_TABLE
Specify behavior of route groups once all ENI bindings are removed
Deprecate/rename some action_type fields to be consistent with the latest sonic-dash-api changes.
  • Loading branch information
@theasianpianist
theasianpianist authored Aug 30, 2024
1 parent 46cc0c0 commit 7a8d0b1
Showing 1 changed file with 24 additions and 11 deletions.
35 changes: 24 additions & 11 deletions doc/dash/dash-sonic-hld.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# SONiC-DASH HLD
## High Level Design Document
### Rev 2.0
### Rev 2.2

# Table of Contents

Expand Down Expand Up @@ -50,6 +50,7 @@
| 1.7 | 02/20/2024 | Prince Sunny | Introduce Route Group Table |
| 2.0 | 04/08/2024 | Prince Sunny | Schema updates for PL, PL-NSG, metering |
| 2.1 | 08/22/2024 | Mukesh M Velayudhan | Add local Region ID field in appliance |
| 2.2 | 08/28/2024 | Lawrence Lee | Route table `routing_type` restrictions, delete op behavior |

# About this Manual
This document provides more detailed design of DASH APIs, DASH orchestration agent, Config and APP DB Schemas and other SONiC buildimage changes required to bring up SONiC image on an appliance card. General DASH HLD can be found at [dash_hld](https://github.com/sonic-net/DASH/tree/main/documentation/general/dash-high-level-design.md).
Expand Down Expand Up @@ -178,12 +179,11 @@ DASH Sonic implementation is targeted for appliance scenarios and must handles m
8. In normal operation, mappings churn often followed by routes and least for ACLs.
9. ENIs shall have an admin-state that enables normal connections and forwarding only *after* all configurations for an ENI is applied during initial creation. When the ENI is admin-state down, the packets destined to this ENI shall be dropped. Order of operation/configuration shall be enforced by the controller. Sonic implementation shall honor the state set by controller and ENI shall accept and forward traffic only if the admin-state is set to 'up'.
10. ENI must support 1M active bi-directional TCP connections or UDP flows however the connection pool can be oversubscribed. An oversubscription of 2:1 would be expected, so the connection pool can be more optimal if executed as one large table where ENI can be a part of the key.
11. During VNET or ENI delete, implementation must support ability to delete all *mappings* or *routes* in a single API call.
12. Add and Delete APIs are idempotent. As an example, deleting an object that doesn't exists shall not return an error.
13. During a delete operation, if there is a dependency (E.g. mappings still present when a VNET is deleted), implementation shall return *error* and shall not perform any force-deletions or delete dependencies implicitly.
14. During a bulk operation, if any part/subset of API fails, implementation shall return *error* for the entire API. Sonic implementation shall validate the entire API as pre-checks before applying and return accordingly.
15. Implementation must have flexible memory allocation for ENI and not reserve max scale during initial create (e.g 100k routes). This is to allow oversubscription.
16. Implementation must not have silent failures for APIs. E.g accepting an API from controller, returning success and failing in the backend. This is orthogonal to the idempotency of APIs described above for ADD and Delete operations. Intent is to ensure SDN controller and Sonic implementation is in-sync
11. Add and Delete APIs are idempotent. As an example, deleting an object that doesn't exists shall not return an error.
12. With some exceptions, during a delete operation, if there is a dependency (E.g. ), implementation shall return *error* and shall not perform any force-deletions or delete dependencies implicitly. See **Implicit deletion of SAI objects** below for exceptions.
13. During a bulk operation, if any part/subset of API fails, implementation shall return *error* for the entire API. Sonic implementation shall validate the entire API as pre-checks before applying and return accordingly.
14. Implementation must have flexible memory allocation for ENI and not reserve max scale during initial create (e.g 100k routes). This is to allow oversubscription.
15. Implementation must not have silent failures for APIs. E.g accepting an API from controller, returning success and failing in the backend. This is orthogonal to the idempotency of APIs described above for ADD and Delete operations. Intent is to ensure SDN controller and Sonic implementation is in-sync

## 1.7 ACL requirements

Expand Down Expand Up @@ -211,6 +211,17 @@ ACL is essential for NSGs and have different stages. In the current model, there
- ACL rules are not expected to have both tags and prefixes of same type configured in the same rule. For e.g, same rule shall not have both src tag and src prefix configured, but it is possible to have src tag and dst prefix or vice-versa
- Counters can be attached to ACL rules optionally for retrieving the number of connections/flows. It is not required to get the packet/byte counter as in the traditional model. A new SAI counter type shall be required for this.

## 1.8 Implicit deletion of SAI objects
Due to memory constraints, certain high-volume table entries will not be cached in orchagent/SAI. Once programmed to hardware, SONiC is unable to perform operations on specific entries without being explicitly configured. This has the following implications:
- For a given 'parent' which is depended on by uncached 'child' entries, deleting the parent must also delete all child entries in the same SAI API call. This is to avoid needing a separate delete op for each child. For example, `DASH_ROUTE_GROUP_TABLE` is a parent for `DASH_ROUTE_TABLE`, which corresponds to `sai_outbound_routing_entry_t`. A command to delete a route group must, in the same SAI API call, delete all `sai_outbound_routing_entry_t` objects in that group. See the table below for all such dependencies between tables.
- If a previously bound route group is unbound from all ENIs, it must be implicitly deleted (in both orchagent cache and SAI). Since route information is not cached, it is not possible to reprogram routes to hardware, so once a route group is completely unbound it cannot be rebound to another ENI and must be deleted. Adding routes for a route group which has been deleted in this manner will result in an error. (Note that the previous bullet also applies here, so all routes in the unbound route group must also be deleted)

| Parent Table | Child Table | Child SAI Type(s) |
| ------------ | ----------- | ---------------- |
| `DASH_ROUTE_GROUP_TABLE` | `DASH_ROUTE_TABLE` | `sai_outbound_routing_entry_t` |
| `DASH_ACL_GROUP_TABLE` | `DASH_ACL_RULE_TABLE` | ACL rule ID (`sai_object_id_t`) |
| `DASH_VNET_TABLE` | `DASH_VNET_MAPPING_TABLE` | `sai_outbound_ca_to_pa_entry_t`,<br>`sai_pa_validation_entry_t`,<br>`sai_outbound_routing_entry_t` (only if `SAI_OUTBOUND_ROUTING_ENTRY_ATTR_DST_VNET_ID` matches the deleted VNET) |

# 2 Packet Flows

The following section captures at a high-level on the VNET packet flow. Detailed lookup and pipeline behavior can be referenced *here*.
Expand Down Expand Up @@ -519,7 +530,8 @@ DASH_ROUTE_GROUP_TABLE:{{group_id}}

```
DASH_ROUTE_TABLE:{{group_id}}:{{prefix}}
"action_type": {{routing_type}}
"action_type": {{routing_type}} (DEPRECATED)
"routing_type": {{routing_type}}
"vnet":{{vnet_name}} (OPTIONAL)
"appliance":{{appliance_id}} (OPTIONAL)
"overlay_ip":{{ip_address}} (OPTIONAL)
Expand All @@ -535,13 +547,14 @@ DASH_ROUTE_TABLE:{{group_id}}:{{prefix}}
```
key = DASH_ROUTE_TABLE:group_id:prefix ; Route route table with CA prefix for packet Outbound
; field = value
action_type = routing_type ; reference to routing type
action_type = routing_type ; reference to routing type (DEPRECATED)
routing_type = routing_type ; replacement for the deprecated `action_type` field. Must be one of {vnet, vnet_direct, direct, servicetunnel, drop}.
vnet = vnet name ; destination vnet name if routing_type is {vnet, vnet_direct}, a vnet other than eni's vnet means vnet peering
appliance = appliance id ; appliance id if routing_type is {appliance}
overlay_ip = ip_address ; overly_ip to lookup if routing_type is {vnet_direct}, use dst ip from packet if not specified
overlay_sip_prefix = ip_prefix ; overlay ipv6 src ip if routing_type is {servicetunnel}, transform last 32 bits from packet (src ip)
overlay_dip_prefix = ip_prefix ; overlay ipv6 dst ip if routing_type is {servicetunnel}, transform last 32 bits from packet (dst ip)
underlay_sip = ip_address ; underlay ipv4 src ip if routing_type is {servicetunnel,privatelink}; this is the ST GW VIP (for ST traffic) or custom VIP. If specified, overrides pl_underlay_sip from DASH_ENI_TABLE
underlay_sip = ip_address ; underlay ipv4 src ip if routing_type is {servicetunnel} or for DASH_VNET_MAPPING_TABLE entries in this prefix where routing_type is {privatelink}; this is the ST GW VIP (for ST traffic) or custom VIP. If specified, overrides pl_underlay_sip from DASH_ENI_TABLE
underlay_dip = ip_address ; underlay ipv4 dst ip to override if routing_type is {servicetunnel}, use dst ip from packet if not specified
metering_policy_en = bool ; Metering policy lookup enable (optional), default = false (OBSOLETED). If aggregated or/and bits is 0, metering policy is applied
metering_class_or = uint32 ; Metering class-id 'or' bits
Expand Down Expand Up @@ -593,7 +606,7 @@ DASH_VNET_MAPPING_TABLE:{{vnet}}:{{ip_address}}
```
key = DASH_VNET_MAPPING_TABLE:vnet:ip_address ; CA-PA mapping table for Vnet
; field = value
action_type = routing_type ; reference to routing type
routing_type = routing_type ; reference to routing type
underlay_ip = ip_address ; PA address for the CA
mac_address = MAC address as string ; Inner dst mac
metering_class_or = uint32 ; metering class 'or' bits
Expand Down

0 comments on commit 7a8d0b1

Please sign in to comment.