Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix some lints and firewalld policies #323

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,15 +60,15 @@ Setting up a loadbalancer or VIP beforehand to use as the API endpoint is possib
Start provisioning of the cluster using the following command:

```bash
ansible-playbook playbook/site.yml -i inventory.yml
ansible-playbook playbooks/site.yml -i inventory.yml
```

## Upgrading

A playbook is provided to upgrade K3s on all nodes in the cluster. To use it, update `k3s_version` with the desired version in `inventory.yml` and run:

```bash
ansible-playbook playbook/upgrade.yml -i inventory.yml
ansible-playbook playbooks/upgrade.yml -i inventory.yml
```

## Airgap Install
Expand All @@ -77,7 +77,7 @@ Airgap installation is supported via the `airgap_dir` variable. This variable sh

An example folder for an x86_64 cluster:
```bash
$ ls ./playbook/my-airgap/
$ ls ./playbooks/my-airgap/
total 248M
-rwxr-xr-x 1 $USER $USER 58M Nov 14 11:28 k3s
-rw-r--r-- 1 $USER $USER 190M Nov 14 11:30 k3s-airgap-images-amd64.tar.gz
Expand Down
17 changes: 9 additions & 8 deletions inventory-sample.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ k3s_cluster:
extra_server_args: ""
extra_agent_args: ""

# Optional vars
# Optional vars
# cluster_context: k3s-ansible
# api_port: 6443
# k3s_server_location: /var/lib/rancher/k3s
Expand All @@ -29,12 +29,13 @@ k3s_cluster:
# List of locally available manifests to apply to the cluster, useful for PVCs or Traefik modifications.
# extra_manifests: [ '/path/to/manifest1.yaml', '/path/to/manifest2.yaml' ]
# airgap_dir: /tmp/k3s-airgap-images
# user_kubectl: true, by default kubectl is symlinked and configured for use by ansible_user. Set to false to only kubectl via root user.
# user_kubectl: true, by default kubectl is symlinked and configured for use by ansible_user.
# Set to false to only kubectl via root user.
# server_config_yaml: |
# This is now an inner yaml file. Maintain the indentation.
# YAML here will be placed as the content of /etc/rancher/k3s/config.yaml
# See https://docs.k3s.io/installation/configuration#configuration-file
# This is now an inner yaml file. Maintain the indentation.
# YAML here will be placed as the content of /etc/rancher/k3s/config.yaml
# See https://docs.k3s.io/installation/configuration#configuration-file
# registries_config_yaml: |
# Containerd can be configured to connect to private registries and use them to pull images as needed by the kubelet.
# YAML here will be placed as the content of /etc/rancher/k3s/registries.yaml
# See https://docs.k3s.io/installation/private-registry
# Containerd can be configured to connect to private registries and use them to pull images as needed
# by the kubelet. YAML here will be placed as the content of /etc/rancher/k3s/registries.yaml
# See https://docs.k3s.io/installation/private-registry
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
16 changes: 8 additions & 8 deletions roles/airgap/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,31 +15,31 @@
url: https://get.k3s.io/
timeout: 120
dest: "{{ airgap_dir }}/k3s-install.sh"
mode: 0755
mode: "0755"

- name: Distribute K3s install script
ansible.builtin.copy:
src: "{{ airgap_dir }}/k3s-install.sh"
dest: /usr/local/bin/k3s-install.sh
owner: root
group: root
mode: 0755
mode: "0755"

- name: Distribute K3s binary
ansible.builtin.copy:
src: "{{ airgap_dir }}/k3s"
dest: /usr/local/bin/k3s
owner: root
group: root
mode: 0755
mode: "0755"

- name: Distribute K3s SELinux RPM
ansible.builtin.copy:
src: "{{ item }}"
dest: /tmp/
owner: root
group: root
mode: 0755
mode: "0755"
with_fileglob:
- "{{ airgap_dir }}/k3s-selinux*.rpm"
register: selinux_copy
Expand All @@ -57,7 +57,7 @@
- name: Make images directory
ansible.builtin.file:
path: "/var/lib/rancher/k3s/agent/images/"
mode: 0755
mode: "0755"
state: directory

- name: Determine Architecture
Expand All @@ -71,7 +71,7 @@
dest: /var/lib/rancher/k3s/agent/images/{{ item | basename }}
owner: root
group: root
mode: 0755
mode: "0755"
with_first_found:
- files:
- "{{ airgap_dir }}/k3s-airgap-images-amd64.tar.zst"
Expand All @@ -86,7 +86,7 @@
dest: /var/lib/rancher/k3s/agent/images/{{ item | basename }}
owner: root
group: root
mode: 0755
mode: "0755"
with_first_found:
- files:
- "{{ airgap_dir }}/k3s-airgap-images-arm64.tar.zst"
Expand All @@ -101,7 +101,7 @@
dest: /var/lib/rancher/k3s/agent/images/{{ item | basename }}
owner: root
group: root
mode: 0755
mode: "0755"
with_first_found:
- files:
- "{{ airgap_dir }}/k3s-airgap-images-arm.tar.zst"
Expand Down
2 changes: 1 addition & 1 deletion roles/k3s_agent/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
dest: /usr/local/bin/k3s-install.sh
owner: root
group: root
mode: 0755
mode: "0755"

- name: Download K3s binary
ansible.builtin.command:
Expand Down
12 changes: 6 additions & 6 deletions roles/k3s_server/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
dest: /usr/local/bin/k3s-install.sh
owner: root
group: root
mode: 0755
mode: "0755"

- name: Download K3s binary
ansible.builtin.command:
Expand All @@ -46,13 +46,13 @@
- name: Make config directory
ansible.builtin.file:
path: "/etc/rancher/k3s"
mode: 0755
mode: "0755"
state: directory
- name: Copy config values
ansible.builtin.copy:
content: "{{ server_config_yaml }}"
dest: "/etc/rancher/k3s/config.yaml"
mode: 0644
mode: "0644"

- name: Init first server node
when: inventory_hostname == groups['server'][0]
Expand All @@ -64,7 +64,7 @@
dest: "{{ systemd_dir }}/k3s.service"
owner: root
group: root
mode: 0644
mode: "0644"

- name: Copy K3s service file [HA]
when: groups['server'] | length > 1
Expand All @@ -73,7 +73,7 @@
dest: "{{ systemd_dir }}/k3s.service"
owner: root
group: root
mode: 0644
mode: "0644"

- name: Add service environment variables
when: extra_service_envs is defined
Expand Down Expand Up @@ -113,7 +113,7 @@

- name: Change server address in kubeconfig on control node
ansible.builtin.shell: |
KUBECONFIG={{ kubeconfig }} kubectl config set-cluster default --server=https://{{ api_endpoint }}:{{ api_port }}

Check warning on line 116 in roles/k3s_server/tasks/main.yml

View workflow job for this annotation

GitHub Actions / Lint

yaml[line-length]

Line too long (125 > 120 characters)

Check warning on line 116 in roles/k3s_server/tasks/main.yml

View workflow job for this annotation

GitHub Actions / Lint

116:121 [line-length] line too long (125 > 120 characters)
delegate_to: 127.0.0.1
become: false
register: csa_result
Expand All @@ -133,7 +133,7 @@
when: kubeconfig != "~/.kube/config"
ansible.builtin.shell: |
TFILE=$(mktemp)
KUBECONFIG={{ kubeconfig }} kubectl config set-context {{ cluster_context }} --user={{ cluster_context }} --cluster={{ cluster_context }}

Check warning on line 136 in roles/k3s_server/tasks/main.yml

View workflow job for this annotation

GitHub Actions / Lint

yaml[line-length]

Line too long (149 > 120 characters)

Check warning on line 136 in roles/k3s_server/tasks/main.yml

View workflow job for this annotation

GitHub Actions / Lint

136:121 [line-length] line too long (149 > 120 characters)
KUBECONFIG={{ kubeconfig }} kubectl config view --flatten > ${TFILE}
mv ${TFILE} {{ kubeconfig }}
delegate_to: 127.0.0.1
Expand All @@ -154,7 +154,7 @@
dest: "{{ systemd_dir }}/k3s.service"
owner: root
group: root
mode: 0644
mode: "0644"

- name: Enable and check K3s service
ansible.builtin.systemd:
Expand Down
11 changes: 11 additions & 0 deletions roles/prereq/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -1,2 +1,13 @@
---
# Zone for inter-node traffic
k3s_firewalld_node_zone: internal

# List of IP addresses or cidr masks of your nodes
k3s_firewalld_node_cidrs: []

# List of public services
k3s_firewalld_public_ports:
- 80/tcp
- 443/tcp
dereknola marked this conversation as resolved.
Show resolved Hide resolved
Comment on lines +8 to +11
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

K3s does not require any public ports to function, so we should not have any set as default. Only user provided ports will be opened.

Suggested change
# List of public services
k3s_firewalld_public_ports:
- 80/tcp
- 443/tcp
# List of public services
k3s_firewalld_public_ports: []


api_port: 6443
48 changes: 41 additions & 7 deletions roles/prereq/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@
- name: If firewalld enabled, open api port
ansible.posix.firewalld:
port: "{{ api_port }}/tcp"
zone: trusted
zone: "{{ k3s_firewalld_node_zone }}"
state: enabled
permanent: true
immediate: true
Expand All @@ -82,10 +82,44 @@
when: groups['server'] | length > 1
ansible.posix.firewalld:
port: "2379-2381/tcp"
zone: trusted
zone: "{{ k3s_firewalld_node_zone }}"
state: enabled
permanent: true
immediate: true

- name: If firewalld enabled, open inter-node ports
ansible.posix.firewalld:
port: "{{ item }}"
zone: "{{ k3s_firewalld_node_zone }}"
state: enabled
permanent: true
immediate: true
with_items:
- 8472/udp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these be left for the user to provide?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the internal zone, only the nodes would be added. It is not a big risk to open ports to different network backends. I believe this will generate less hassle for users.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, agreed, The user can manage adding or removing ports in a separate task.

- 10250/tcp
- 51820/udp
- 51821/udp
- 5001/tcp

- name: If firewalld enabled, allow node CIDRs
ansible.posix.firewalld:
source: "{{ item }}"
zone: "{{ k3s_firewalld_node_zone }}"
state: enabled
permanent: true
immediate: true
when: k3s_firewalld_node_cidrs is defined
loop: "{{ k3s_firewalld_node_cidrs }}"

- name: If firewalld enabled, open public ports
ansible.posix.firewalld:
port: "{{ item }}"
zone: "public"
state: enabled
permanent: true
immediate: true
when: k3s_firewalld_public_ports is defined
loop: "{{ k3s_firewalld_public_ports }}"

- name: If firewalld enabled, allow default CIDRs
ansible.posix.firewalld:
Expand Down Expand Up @@ -184,7 +218,7 @@
- name: Make rancher directory
ansible.builtin.file:
path: "/var/lib/rancher"
mode: 0755
mode: "0755"
state: directory
- name: Create symlink
ansible.builtin.file:
Expand All @@ -199,13 +233,13 @@
- name: Make manifests directory
ansible.builtin.file:
path: "/var/lib/rancher/k3s/server/manifests"
mode: 0700
mode: "0700"
state: directory
- name: Copy manifests
ansible.builtin.copy:
src: "{{ item }}"
dest: "/var/lib/rancher/k3s/server/manifests"
mode: 0600
mode: "0600"
loop: "{{ extra_manifests }}"

- name: Setup optional private registry configuration
Expand All @@ -214,10 +248,10 @@
- name: Make k3s config directory
ansible.builtin.file:
path: "/etc/rancher/k3s"
mode: 0755
mode: "0755"
state: directory
- name: Copy config values
ansible.builtin.copy:
content: "{{ registries_config_yaml }}"
dest: "/etc/rancher/k3s/registries.yaml"
mode: 0644
mode: "0644"
Loading