Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor ovn ipsec and remove old ipsec.sh #187

Merged
merged 1 commit into from
Aug 12, 2024
Merged

refactor ovn ipsec and remove old ipsec.sh #187

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 6 additions & 88 deletions docs/advance/ovn-ipsec.en.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,94 +1,12 @@
# Encrypt inter-node communication using IPsec

This function is supported after v1.10.11 and v1.11.4, the kernel version is at least 3.10.0 or above, and UDP ports 500 and 4500 are available.
# Use IPsec to encrypt communication between nodes

## Start IPsec
This function is supported from v1.13.0 onwards, and the host UDP 500 and 4500 ports need to be available.

Copy the script from the Kube-OVN source code [ipsec.sh](https://raw.githubusercontent.com/kubeovn/kube-ovn/master/dist/images/ipsec.sh), execute the command as follows, the script will call ovs-pki to generate and distribute the certificate required for encryption:
## Encryption process

```bash
bash ipsec.sh init
```
kube-ovn-cni is responsible for applying for certificates and will create a certificate signing request to kube-ovn-controller. kube-ovn-controller will automatically approve the certificate application, and then kube-ovn-cni will generate an ipsec configuration file based on the certificate and finally start the ipsec process.

After the execution is completed, the nodes will negotiate for a period of time to establish an IPsec tunnel. The experience value is between ten seconds and one minute.You can check the IPsec status with the following command:
## Configure IPsec

```bash
# bash ipsec.sh status
Pod {ovs-ovn-d7hdt} ipsec status...
Interface name: ovn-a4718e-0 v1 (CONFIGURED)
Tunnel Type: geneve
Local IP: 172.18.0.2
Remote IP: 172.18.0.4
Address Family: IPv4
SKB mark: None
Local cert: /etc/ipsec.d/certs/8aebd9df-46ef-47b9-85e3-73e9a765296d-cert.pem
Local name: 8aebd9df-46ef-47b9-85e3-73e9a765296d
Local key: /etc/ipsec.d/private/8aebd9df-46ef-47b9-85e3-73e9a765296d-privkey.pem
Remote cert: None
Remote name: a4718e55-5b85-4f46-90e6-63527d080590
CA cert: /etc/ipsec.d/cacerts/cacert.pem
PSK: None
Custom Options: {}
Ofport: 2
CFM state: Disabled
Kernel policies installed:
src 172.18.0.2/32 dst 172.18.0.4/32 proto udp sport 6081
src 172.18.0.2/32 dst 172.18.0.4/32 proto udp sport 6081
src 172.18.0.2/32 dst 172.18.0.4/32 proto udp dport 6081
src 172.18.0.2/32 dst 172.18.0.4/32 proto udp dport 6081
Kernel security associations installed:
sel src 172.18.0.2/32 dst 172.18.0.4/32 proto udp sport 6081
sel src 172.18.0.4/32 dst 172.18.0.2/32 proto udp dport 6081
sel src 172.18.0.2/32 dst 172.18.0.4/32 proto udp dport 6081
sel src 172.18.0.4/32 dst 172.18.0.2/32 proto udp sport 6081
IPsec connections that are active:

Pod {ovs-ovn-fvbbj} ipsec status...
Interface name: ovn-8aebd9-0 v1 (CONFIGURED)
Tunnel Type: geneve
Local IP: 172.18.0.4
Remote IP: 172.18.0.2
Address Family: IPv4
SKB mark: None
Local cert: /etc/ipsec.d/certs/a4718e55-5b85-4f46-90e6-63527d080590-cert.pem
Local name: a4718e55-5b85-4f46-90e6-63527d080590
Local key: /etc/ipsec.d/private/a4718e55-5b85-4f46-90e6-63527d080590-privkey.pem
Remote cert: None
Remote name: 8aebd9df-46ef-47b9-85e3-73e9a765296d
CA cert: /etc/ipsec.d/cacerts/cacert.pem
PSK: None
Custom Options: {}
Ofport: 1
CFM state: Disabled
Kernel policies installed:
src 172.18.0.4/32 dst 172.18.0.2/32 proto udp dport 6081
src 172.18.0.4/32 dst 172.18.0.2/32 proto udp dport 6081
src 172.18.0.4/32 dst 172.18.0.2/32 proto udp sport 6081
src 172.18.0.4/32 dst 172.18.0.2/32 proto udp sport 6081
Kernel security associations installed:
sel src 172.18.0.4/32 dst 172.18.0.2/32 proto udp dport 6081
sel src 172.18.0.2/32 dst 172.18.0.4/32 proto udp sport 6081
sel src 172.18.0.4/32 dst 172.18.0.2/32 proto udp sport 6081
sel src 172.18.0.2/32 dst 172.18.0.4/32 proto udp dport 6081
IPsec connections that are active:
```

After the establishment is complete, you can capture packets and observe that the packets have been encrypted:

```bash
# tcpdump -i eth0 -nel esp
10:01:40.349896 IP kube-ovn-worker > kube-ovn-control-plane.kind: ESP(spi=0xcc91322a,seq=0x13d0), length 156
10:01:40.350015 IP kube-ovn-control-plane.kind > kube-ovn-worker: ESP(spi=0xc8df4221,seq=0x1d37), length 156
```

After executing the script, you can turn off IPsec by executing the command:

```bash
# bash ipsec.sh stop
```

Or execute the command to open it again:

```bash
# bash ipsec.sh start
```
Change the args `--enable-ovn-ipsec=false` in kube-ovn-controller and kube-ovn-cni to `--enable-ovn-ipsec=true`.
93 changes: 5 additions & 88 deletions docs/advance/ovn-ipsec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,94 +1,11 @@
# 使用 IPsec 加密节点间通信

该功能从 v1.10.11 和 v1.11.4 后开始支持，kernel 版本至少是 3.10.0 以上，同时需要保证主机 UDP 500 和 4500 端口可用。
该功能从 v1.13.0 后支持，同时需要保证主机 UDP 500 和 4500 端口可用。

## 启动 IPsec
## 加密流程

从 Kube-OVN 源码拷贝脚本 [ipsec.sh](https://raw.githubusercontent.com/kubeovn/kube-ovn/master/dist/images/ipsec.sh)，执行命令如下，该脚本会调用 ovs-pki 生成和分配加密需要的证书：
kube-ovn-cni 负责将证书申请，会创建一个 certificatesigningrequest 给 kube-ovn-controller，kube-ovn-controller 会自动 approve 证书申请，然后 kube-ovn-cni 会根据证书生成 ipsec 配置文件，最后启动 ipsec 进程。

```bash
bash ipsec.sh init
```
## 配置 IPsec

执行完毕后，节点之间会协商一段时间建立 IPsec 隧道，经验值是十几秒到一分钟之间，可以通过如下命令来查看 IPsec 状态：

```bash
# bash ipsec.sh status
Pod {ovs-ovn-d7hdt} ipsec status...
Interface name: ovn-a4718e-0 v1 (CONFIGURED)
Tunnel Type: geneve
Local IP: 172.18.0.2
Remote IP: 172.18.0.4
Address Family: IPv4
SKB mark: None
Local cert: /etc/ipsec.d/certs/8aebd9df-46ef-47b9-85e3-73e9a765296d-cert.pem
Local name: 8aebd9df-46ef-47b9-85e3-73e9a765296d
Local key: /etc/ipsec.d/private/8aebd9df-46ef-47b9-85e3-73e9a765296d-privkey.pem
Remote cert: None
Remote name: a4718e55-5b85-4f46-90e6-63527d080590
CA cert: /etc/ipsec.d/cacerts/cacert.pem
PSK: None
Custom Options: {}
Ofport: 2
CFM state: Disabled
Kernel policies installed:
src 172.18.0.2/32 dst 172.18.0.4/32 proto udp sport 6081
src 172.18.0.2/32 dst 172.18.0.4/32 proto udp sport 6081
src 172.18.0.2/32 dst 172.18.0.4/32 proto udp dport 6081
src 172.18.0.2/32 dst 172.18.0.4/32 proto udp dport 6081
Kernel security associations installed:
sel src 172.18.0.2/32 dst 172.18.0.4/32 proto udp sport 6081
sel src 172.18.0.4/32 dst 172.18.0.2/32 proto udp dport 6081
sel src 172.18.0.2/32 dst 172.18.0.4/32 proto udp dport 6081
sel src 172.18.0.4/32 dst 172.18.0.2/32 proto udp sport 6081
IPsec connections that are active:

Pod {ovs-ovn-fvbbj} ipsec status...
Interface name: ovn-8aebd9-0 v1 (CONFIGURED)
Tunnel Type: geneve
Local IP: 172.18.0.4
Remote IP: 172.18.0.2
Address Family: IPv4
SKB mark: None
Local cert: /etc/ipsec.d/certs/a4718e55-5b85-4f46-90e6-63527d080590-cert.pem
Local name: a4718e55-5b85-4f46-90e6-63527d080590
Local key: /etc/ipsec.d/private/a4718e55-5b85-4f46-90e6-63527d080590-privkey.pem
Remote cert: None
Remote name: 8aebd9df-46ef-47b9-85e3-73e9a765296d
CA cert: /etc/ipsec.d/cacerts/cacert.pem
PSK: None
Custom Options: {}
Ofport: 1
CFM state: Disabled
Kernel policies installed:
src 172.18.0.4/32 dst 172.18.0.2/32 proto udp dport 6081
src 172.18.0.4/32 dst 172.18.0.2/32 proto udp dport 6081
src 172.18.0.4/32 dst 172.18.0.2/32 proto udp sport 6081
src 172.18.0.4/32 dst 172.18.0.2/32 proto udp sport 6081
Kernel security associations installed:
sel src 172.18.0.4/32 dst 172.18.0.2/32 proto udp dport 6081
sel src 172.18.0.2/32 dst 172.18.0.4/32 proto udp sport 6081
sel src 172.18.0.4/32 dst 172.18.0.2/32 proto udp sport 6081
sel src 172.18.0.2/32 dst 172.18.0.4/32 proto udp dport 6081
IPsec connections that are active:
```

建立完成后可以抓包观察报文已经被加密：

```bash
# tcpdump -i eth0 -nel esp
10:01:40.349896 IP kube-ovn-worker > kube-ovn-control-plane.kind: ESP(spi=0xcc91322a,seq=0x13d0), length 156
10:01:40.350015 IP kube-ovn-control-plane.kind > kube-ovn-worker: ESP(spi=0xc8df4221,seq=0x1d37), length 156
```

当执行完脚本后，可以通过执行命令关闭 IPsec：

```bash
# bash ipsec.sh stop
```

或者执行命令再次打开：

```bash
# bash ipsec.sh start
```
将 kube-ovn-controller 和 kube-ovn-cni 中的 args `--enable-ovn-ipsec=false` 修改为 `--enable-ovn-ipsec=true`。