diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 12f26d20e..36cad9a12 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -15,5 +15,5 @@ jobs: npm install -g markdownlint-cli npm install -g markdownlint-rule-search-replace@1.0.9 pwd - markdownlint "**/*.md" --disable MD013 MD033 MD045 MD024 MD041 -r markdownlint-rule-search-replace + markdownlint "**/*.md" --disable MD013 MD033 MD045 MD024 MD041 MD029 MD051 -r markdownlint-rule-search-replace markdownlint -V \ No newline at end of file diff --git a/docs/advance/offload-yunsilicon.en.md b/docs/advance/offload-yunsilicon.en.md index 82c47086a..afa630d6f 100644 --- a/docs/advance/offload-yunsilicon.en.md +++ b/docs/advance/offload-yunsilicon.en.md @@ -54,6 +54,7 @@ bash install.sh ``` ### Setting Up SR-IOV + 1. Find the device id of metaScale device, below is `b3:00.0` ```shell @@ -116,6 +117,7 @@ systemctl disable NetworkManager ### Install SR-IOV Device Plugin 1. Create a ConfigMap that defines SR-IOV resource pool configuration + ```yaml apiVersion: v1 kind: ConfigMap @@ -149,6 +151,7 @@ data: ``` ### Install Multus-CNI + 1. Follow [Multus-CNI](https://github.com/k8snetworkplumbingwg/multus-cni) to deploy Multus-CNI ```bash @@ -218,4 +221,4 @@ recirc_id(0),in_port(8),packet_type(ns=0,id=0),eth(src=00:00:00:ce:cf:b9/01:00:0 ct_state(-new+est-rel+rpl+trk),ct_mark(0/0x1),recirc_id(0x2d278),in_port(15),packet_type(ns=0,id=0),eth(dst=00:00:00:ce:cf:b9/01:00:00:00:00:00),eth_type(0x0800),ipv4(frag=no), packets:6, bytes:588, used:7.277s, actions:8 ``` -You can find some flows if all works well. \ No newline at end of file +You can find some flows if all works well. diff --git a/docs/advance/offload-yunsilicon.md b/docs/advance/offload-yunsilicon.md index 196fd82a6..c7f868502 100644 --- a/docs/advance/offload-yunsilicon.md +++ b/docs/advance/offload-yunsilicon.md @@ -107,7 +107,7 @@ devlink dev eswitch set pci/0000:b3:00.0 mode switchdev ### 安装 SR-IOV Device Plugin 1. 创建 SR-IOV 资源 ConfigMap： - + ```yaml apiVersion: v1 kind: ConfigMap @@ -140,6 +140,7 @@ data: ``` ### 安装 Multus-CNI + 1. 参考 [Multus-CNI](https://github.com/k8snetworkplumbingwg/multus-cni) 来部署 Multus-CNI ```bash @@ -195,11 +196,12 @@ spec: limits: yunsilicon.com/xsc_sriov: '1' ``` + ### Offload 验证 可通过在 Pod 运行节点的 `ovs-ovn` 容器中运行下面的命令观察卸载是否成功： -```bash +```bash ovs-appctl dpctl/dump-flows type=offloaded flow-dump from pmd on cpu core: 9 ct_state(-new+est-rel+rpl+trk),ct_mark(0/0x3),recirc_id(0x2d277),in_port(15),packet_type(ns=0,id=0),eth(src=00:00:00:9d:fb:1a,dst=00:00:00:ce:cf:b9),eth_type(0x0800),ipv4(dst=10.16.0.14,frag=no), packets:6, bytes:588, used:7.276s, actions:ct(zone=4,nat),recirc(0x2d278) @@ -210,4 +212,4 @@ recirc_id(0),in_port(8),packet_type(ns=0,id=0),eth(src=00:00:00:ce:cf:b9/01:00:0 ct_state(-new+est-rel+rpl+trk),ct_mark(0/0x1),recirc_id(0x2d278),in_port(15),packet_type(ns=0,id=0),eth(dst=00:00:00:ce:cf:b9/01:00:00:00:00:00),eth_type(0x0800),ipv4(frag=no), packets:6, bytes:588, used:7.277s, actions:8 ``` -如果有流表内容证明卸载成功。 \ No newline at end of file +如果有流表内容证明卸载成功。 diff --git a/docs/guide/qos.en.md b/docs/guide/qos.en.md index 0cbc464ff..fe4014b3f 100644 --- a/docs/guide/qos.en.md +++ b/docs/guide/qos.en.md @@ -135,5 +135,6 @@ Pod can use annotation below to config `linux-netem` type QoS： `ovn.kubernetes > yum install -y kernel-modules-extra && modprobe sch_netem - `ovn.kubernetes.io/latency`: Set the Pod traffic delay to an integer value in ms. -- `ovn.kubernetes.io/limit`： Set the maximum number of packets that the `qdisc` queue can hold, and takes an integer value, such as 1000. -- `ovn.kubernetes.io/loss`： Set packet loss probability, the value is float type, for example, the value is 20, then it is set 20% packet loss probability. +- `ovn.kubernetes.io/jitter`: Set the Pod traffic jitter to an integer value in ms. +- `ovn.kubernetes.io/limit`: Set the maximum number of packets that the `qdisc` queue can hold, and takes an integer value, such as 1000. +- `ovn.kubernetes.io/loss`: Set packet loss probability, the value is float type, for example, the value is 20, then it is set 20% packet loss probability. diff --git a/docs/guide/qos.md b/docs/guide/qos.md index a82d3c239..64d98617a 100644 --- a/docs/guide/qos.md +++ b/docs/guide/qos.md @@ -136,5 +136,6 @@ Pod 可以使用如下 annotation 配置 `linux-netem` 类型 QoS： `ovn.kubern `ovn.kubernetes.io/loss`。 - `ovn.kubernetes.io/latency`：设置 Pod 流量延迟，取值为整数，单位为 ms。 +- `ovn.kubernetes.io/jitter`：设置 Pod 流量延迟抖动，取值为整数，单位为 ms。 - `ovn.kubernetes.io/limit`： 为 `qdisc` 队列可容纳的最大数据包数，取值为整形数值，例如 1000。 - `ovn.kubernetes.io/loss`： 为设置的报文丢包概率，取值为 float 类型，例如取值为 20，则为设置 20% 的丢包概率。 diff --git a/docs/reference/annotations.en.md b/docs/reference/annotations.en.md new file mode 100644 index 000000000..59202b691 --- /dev/null +++ b/docs/reference/annotations.en.md @@ -0,0 +1,79 @@ +# Annotation Usage + +Kube-OVN uses a large number of Pod and Node Annotations for configuring functionality and transferring information. Users can refer to this document to understand the usage of each Annotation, to better troubleshooting and information retrieval. + +> Note: Some Annotations may change as the code is adjusted. + +## Pod Annotation + +| Key | Value | Description | +| -------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------- | +| ovn.kubernetes.io/allocated | `true` or `false` | If the Pod primary interface has already been allocated an address | +| ovn.kubernetes.io/routed | `true` or `false` | If the Pod primary interface has already been allocated a route | +| ovn.kubernetes.io/mac_address | String | MAC address allocated to Pod primary interface，when creating a Pod, you can set a fixed MAC address by this Annotation | +| ovn.kubernetes.io/ip_address | String | IP address allocated to Pod primary interface，when creating a Pod, you can set a fixed IP address by this Annotation | +| ovn.kubernetes.io/cidr | String | Subnet CIDR that the Pod primary interface belongs to | +| ovn.kubernetes.io/gateway | String | Subnet Gateway address that the Pod primary interface belongs to | +| ovn.kubernetes.io/ip_pool | IP list, separated by comma | Pod primary interface will choose address from this list, used for workload fix address | +| ovn.kubernetes.io/bgp | `true` or `false` | Enable Pod address BGP advertisement 址 | +| ovn.kubernetes.io/snat | String | SNAT address for accessing external address | +| ovn.kubernetes.io/eip | String | EIP address that Pod accesses external clusters and is accessed from external. | +| ovn.kubernetes.io/vip | String | VIP allocated to Pod primary interface | +| ovn.kubernetes.io/virtualmachine | String | The VirtualMachineInstance that the Pod primary interface belongs to | +| ovn.kubernetes.io/logical_router | String | The VPC that the Pod primary interface belongs to | +| ovn.kubernetes.io/layer2_forward | `true` or `false` | Enable add `unknown` address to Pod primary interface in OVN NorthboundDB LSP | +| ovn.kubernetes.io/port_security | `true` or `false` | Enable Pod primary interface port security | +| ovn.kubernetes.io/logical_switch | String | The Subnet that the Pod primary interface belongs to | +| ovn.kubernetes.io/vlan_id | Int | The VlanID that the Pod primary interface belongs to | +| ovn.kubernetes.io/ingress_rate | Int | Pod primary interface ingress rate limit, measured in Mbits/s | +| ovn.kubernetes.io/egress_rate | Int | Pod primary interface egress rate limit, measured in Mbits/s | +| ovn.kubernetes.io/security_groups | String list, separated by comma | The SecurityGroup that the Pod primary interface belongs to | +| ovn.kubernetes.io/allow_live_migration | `true` or `false` | Allow live migration for Pod primary interface, used by KubeVirt | +| ovn.kubernetes.io/default_route | `true` or `false` | Set the default route to the Pod primary interface. | +| ovn.kubernetes.io/provider_network | String | The ProviderNetwork that the Pod primary interface belongs to | +| ovn.kubernetes.io/mirror | `true` or `false` | Enable Pod primary interface traffic mirror | +| ovn.kubernetes.io/latency | Int | The delay injected to the Pod primary interface card, measured in milliseconds | +| ovn.kubernetes.io/limit | Int | Maximum number of packets that the qdisc queue of the primary interface of the Pod | +| ovn.kubernetes.io/loss | Float | The probability of packet loss on the Pod primary interface | +| ovn.kubernetes.io/jitter | Int | The jitter of packet latency on the Pod primary interface, measured in milliseconds | + +## Node Annotation + +| Key | Value | Description | +| ---------------------------------- | ----------------- | -------------------------------------------------------------------------- | +| ovn.kubernetes.io/allocated | `true` or `false` | If the `ovn0` interface has already been allocated an address | +| ovn.kubernetes.io/ip_address | String | IP address allocated to `ovn0` interface | +| ovn.kubernetes.io/mac_address | String | MAC address allocated to `ovn0` interface | +| ovn.kubernetes.io/cidr | String | Subnet CIDR that the node `ovn0` interface belongs to | +| ovn.kubernetes.io/gateway | String | Subnet gateway that the node `ovn0` interface belongs to | +| ovn.kubernetes.io/chassis | String | The Chassis ID in OVN-SouthBoundDB that the node belongs to | +| ovn.kubernetes.io/port_name | String | The LSP name in OVN-NorthboundDB that the node `ovn0` interface belongs to | +| ovn.kubernetes.io/logical_switch | String | Subnet that the node `ovn0` interface belongs to | +| ovn.kubernetes.io/tunnel_interface | String | Network interface used for tunnel encapsulation | + +## Namespace Annotation + +| Key | Value | Description | +| ----------------------------- | --------------------------------------- | ------------------------------------------------- | +| ovn.kubernetes.io/cidr | CIDR list, separated by comma | The CIDRs of subnets bound by this Namespace | +| ovn.kubernetes.io/exclude_ips | excludeIPs list, separated by semicolon | The excludeIPs of subnets bound by this Namespace | + +## Subnet Annotation + +| Key | Value | Description | +| --------------------- | ----------------- | --------------------------------------- | +| ovn.kubernetes.io/bgp | `true` or `false` | Enable Subnet address BGP advertisement | + +## Service Annotation + +| Key | Value | Description | +| ------------------------------- | ----------------- | --------------------------------------------------------- | +| ovn.kubernetes.io/bgp | `true` or `false` | Enable Service address BGP advertisement | +| ovn.kubernetes.io/switch_lb_vip | String | Additional VIP addresses assigned to Service in Kube-OVN. | +| ovn.kubernetes.io/vpc | String | The VPC that the Service belongs to | + +## Networkpolicy Annotation + +| Key | Value | Description | +| ---------------------------- | ----------------- | ------------------------ | +| ovn.kubernetes.io/enable_log | `true` or `false` | Enable NetworkPolicy log | diff --git a/docs/reference/annotations.md b/docs/reference/annotations.md new file mode 100644 index 000000000..9d6a5d100 --- /dev/null +++ b/docs/reference/annotations.md @@ -0,0 +1,80 @@ +# Annotation 使用说明 + +Kube-OVN 使用了大量的 Pod 和 Node Annotation 进行功能设置和信息传递，用户可以通过本文档了解各个 Annotation 的使用，方便问题排查和信息获取。 + +> 注意：部分 Annotation 可能会随着代码调整而进行变化。 + +## Pod Annotation + +| Key | Value | Description | +| -------------------------------------- | ------------------------- | ------------------------------------------------------------------------------- | +| ovn.kubernetes.io/allocated | `true` or `false` | Pod 主网卡是否已被分配地址 | +| ovn.kubernetes.io/routed | `true` or `false` | Pod 主网卡在 OVN 内的路由是否设置完成 | +| ovn.kubernetes.io/mac_address | String | Pod 主网卡所分配到的 Mac 地址，创建 Pod 时可通过该 Annotation 设置固定 Mac 地址 | +| ovn.kubernetes.io/ip_address | String | Pod 主网卡所分配到的 IP 地址，创建 Pod 时可通过该 Annotation 设置固定 IP 地址 | +| ovn.kubernetes.io/cidr | String | Pod 主网卡所属子网的 CIDR | +| ovn.kubernetes.io/gateway | String | Pod 主网卡所属子网的 Gateway 地址 | +| ovn.kubernetes.io/ip_pool | IP 列表，逗号间隔 | Pod 主网卡地址将从列表中选择，适用于 Workload 固定 IP 场景 | +| ovn.kubernetes.io/bgp | `true` or `false` | 是否对外通过 BGP 发布 Pod 地址 | +| ovn.kubernetes.io/snat | String | Pod 访问集群外使用的 SNAT 地址 | +| ovn.kubernetes.io/eip | String | Pod 访问集群外部和被集群外部访问所使用的 EIP 地址 | +| ovn.kubernetes.io/vip | String | Pod 主网卡使用的预留 VIP，可通过该 Annotation 使用预先创建的 VIP 资源 | +| ovn.kubernetes.io/virtualmachine | String | Pod 主网卡所属的 VirtualMachineInstance | +| ovn.kubernetes.io/logical_router | String | Pod 主网卡所属的 VPC | +| ovn.kubernetes.io/layer2_forward | `true` or `false` | Pod 主网卡在 OVN LSP 中是否增加 `unknown` 地址 | +| ovn.kubernetes.io/port_security | `true` or `false` | Pod 主网卡对应端口是否开启 Port Security | +| ovn.kubernetes.io/logical_switch | String | Pod 主网卡所属的 Subnet | +| ovn.kubernetes.io/vlan_id | Int | Pod 主网卡所属 Subnet 的 Vlan ID | +| ovn.kubernetes.io/ingress_rate | Int | Pod 主网卡流入方向限速，单位为 Mbits/s | +| ovn.kubernetes.io/egress_rate | Int | Pod 主网卡流出方向限速，单位为 Mbits/s | +| ovn.kubernetes.io/security_groups | String 列表，使用逗号分隔 | Pod 主网卡所属的 Security Group | +| ovn.kubernetes.io/allow_live_migration | `true` or `false` | Pod 主网卡是否允许 live migration，用于 kubevirt 场景 | +| ovn.kubernetes.io/default_route | `true` or `false` | 是否将主网卡设置为默认路由网卡 | +| ovn.kubernetes.io/provider_network | String | Pod 主网卡所属的 ProviderNetwork | +| ovn.kubernetes.io/mirror | `true` or `false` | Pod 主网卡是否做流量镜像 | +| ovn.kubernetes.io/logical_switch | String | Pod 主网卡所属 Subnet | +| ovn.kubernetes.io/latency | Int | Pod 主网卡注入的延迟，单位为 ms | +| ovn.kubernetes.io/limit | Int | Pod 主网卡 qdisc 队列可容纳的最大数据包数 | +| ovn.kubernetes.io/loss | Float | Pod 主网卡报文丢包概率 | +| ovn.kubernetes.io/jitter | Int | Pod 主网卡注入抖动延迟，单位为 ms | + +## Node Annotation + +| Key | Value | Description | +| ---------------------------------- | ----------------- | ------------------------------------------------- | +| ovn.kubernetes.io/allocated | `true` or `false` | 节点的 `ovn0` 网卡是否已被分配 `join` 子网地址 | +| ovn.kubernetes.io/mac_address | String | Node `ovn0` 网卡分配到的 Mac 地址 | +| ovn.kubernetes.io/ip_address | String | Node `ovn0` 网卡所分配到的 IP 地址 | +| ovn.kubernetes.io/cidr | String | Node `ovn0` 网卡所属 `join` 子网的 CIDR | +| ovn.kubernetes.io/gateway | String | Node `ovn0` 网卡所属 `join` 子网的 Gateway 地址 | +| ovn.kubernetes.io/chassis | String | Node 在 OVN-SouthBoundDB 中的 Chassis ID | +| ovn.kubernetes.io/port_name | String | Node `ovn0` 网卡在 OVN-NorthboundDB 中 LSP 的名称 | +| ovn.kubernetes.io/logical_switch | String | Node `ovn0` 网卡所属 Subnet | +| ovn.kubernetes.io/tunnel_interface | String | 隧道封装使用的网卡 | + +## Namespace Annotation + +| Key | Value | Description | +| ----------------------------- | ------------------------- | ------------------------------------ | +| ovn.kubernetes.io/cidr | CIDR 列表，逗号分隔 | 该 Namespace 所绑定子网的 CIDR | +| ovn.kubernetes.io/exclude_ips | excludeIPs 列表，分号分割 | 该 Namespace 所绑定子网的 excludeIPs | + +## Subnet Annotation + +| Key | Value | Description | +| --------------------- | ----------------- | ----------------------------- | +| ovn.kubernetes.io/bgp | `true` or `false` | 是否对外通过 BGP 发布子网地址 | + +## Service Annotation + +| Key | Value | Description | +| ------------------------------- | ------------------------- | ----------------------------------------- | +| ovn.kubernetes.io/bgp | `true` or `false` | 是否对外通过 BGP 发布 Service 地址 | +| ovn.kubernetes.io/switch_lb_vip | String | Service 在 Kube-OVN 中额外分配的 VIP 地址 | +| ovn.kubernetes.io/vpc | String | Service 所属的 VPC | + +## Networkpolicy Annotation + +| Key | Value | Description | +| ---------------------------- | ----------------- | --------------------------- | +| ovn.kubernetes.io/enable_log | `true` or `false` | 是否开启 NetworkPolicy 日志 | diff --git a/docs/reference/tunnel-protocol.md b/docs/reference/tunnel-protocol.md index 29cad696e..eae0f79ed 100644 --- a/docs/reference/tunnel-protocol.md +++ b/docs/reference/tunnel-protocol.md @@ -18,7 +18,7 @@ CPU 资源。 ## Vxlan -`Vxlan`为上游 OVN 近期支持的协议，该协议在内核中得到了广泛的支持，并可以利用现代网卡的通用 Offload 能力进行加速。。 +`Vxlan` 为上游 OVN 近期支持的协议，该协议在内核中得到了广泛的支持，并可以利用现代网卡的通用 Offload 能力进行加速。。 由于该协议头部长度有限，并且 OVN 需要使用额外的空间进行编排，datapath 的数量存在限制，最多只能创建 4096 个 datapath， 每个 datapath 下最多 4096 个端口。同时由于空间有限，基于 `inport` 的 ACL 没有进行支持。 diff --git a/mkdocs.yml b/mkdocs.yml index 465745153..c56745ecb 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -83,6 +83,7 @@ nav: - 隧道协议说明: 'reference/tunnel-protocol.md' - Kube-OVN 监控指标: 'reference/metrics.md' - Kube-OVN 接口规范: 'reference/kube-ovn-api.md' + - Annotation 使用说明: 'reference/annotation.md' - 文档规范: 'reference/document-convention.md' - 联系方式: 'contact.md' theme: @@ -220,6 +221,7 @@ plugins: 调整日志等级: Change Log Level 使用 Sealos 一键部署 Kubernetes 和 Kube-OVN: Use Sealos to Deploy Kubernetes and Kube-OVN 版本管理: Release Management + Annotation 使用说明: Annotation Usage - locale: zh default: true name: 简体中文