Support HA mode with embedded DB #97
Conversation
|
Right now nodes are registering using a non HA endpoint. Either this playbook needs to create such endpoint like when |
St0rmingBr4in
force-pushed
the
k3s-ha
branch
2 times, most recently
from
ffa42b8
to
50ee597
Compare
St0rmingBr4in
changed the title
St0rmingBr4in
force-pushed
the
k3s-ha
branch
5 times, most recently
from
fd61649
to
b2a9f7c
Compare
|
Is there any particular thing that is blocking from this being merged? I am finding that we need the HA functionality in our deployments. Let me know if there is anything I can do to help. |
I think it's ready to merge but if you want to try it first I would be very interested in your feedback :) |
|
I can give it a go. I invoke it as a dependency in a ansible role, so it won't be testing as is, but it could still be a useful to see. |
|
Okay 👌 Let me know how it goes :) |
change variable name(tbm after k3s-io#97)
|
I tested the current version and it works fine when running the first time. It does break when re-running the playbook after the cluster has been successfully setup. I think the error is caused by the node(s) without the |
|
Yes, the playbook tries to verify that all masters joined the cluster. I suspect that they are each creating a 1 node cluster. But I don't know why. I've tried inside vagrant VMs but I don't quite understand where is the problem at the moment. |
|
@itwars I added you as reviewer, I think the change is ready to merge now, if you could take a look I would greatly appreciate. |
|
on 2 fresh ubuntu (for testing purposes with the embedded db) i got the 20 retries any log or test you want me to try ? |
To debug you need to access the logs of the k3s-init service. Using I will add a quick message to state how to debug errors if the verify task fails. |
|
Im cross-posting the resul here from #32 |
|
@mattthhdp Normally k3s-ansible should be able to entirely replace the https://get.k3s.io script. Tell me if I'm wrong but to sum up:
If that's true then I think we should create another issue to keep track of your problem. Since k3s-ansible does not work even without the changes of this PR then we should fix your problem separately of this PR. Would you mind creating a new issue summarizing what you did and the full logs ? |
roles/k3s/master/tasks/main.yml
Outdated
| cmd: "systemd-run -p RestartSec=2 \ | ||
| -p Restart=on-failure \ | ||
| --unit=k3s-init \ | ||
| k3s server {{ server_init_args }}" |
There was a problem hiding this comment.
Does not k3s server require {{ server_init_args }} in order to connect to the "cluster master" and if so isn't more suitable to add {{ server_init_args }} to k3s/master/template/k3s.service.j2?
There was a problem hiding this comment.
No, the {{ server_init_args }} are only used for the inital startup to create the embedded etcd cluster. After that the startup works the same as always.
Take a look at the defaults file and the rancher docs for reference.
At least that's my understanding of this.
There was a problem hiding this comment.
I thought it was a bit unclear, however I got it clarified from a k3s maintainer. That these are only required for the initial setup.
|
According to this article, CentOS 7 can now run systemd 231. Has anyone tried this? |
|
Hi guys, any news here? I am new to GitHub and stuff but it seams to @itwars has to approve the PR? |
|
Hello guys, any plans to merge this? |
|
Also hoping for this to get merged |
|
I have a workaround for this, namely, run the playbook twice. The first invocation is limited "master[0]". Additionally, leverage inventory/$CLUSTER/group_vars/master (example below using MySQL as external database): Run 1: ansible-playbook site.yml --limit master[0] |
|
Is it maybe an option to not support centos 7? |
|
Does this just need to be re-tested with centos7 and synced to be integrated? Is there anything I can help with? |
| {% if ansible_host == hostvars[groups['master'][0]]['ansible_host'] | default(groups['master'][0]) %} | ||
| --cluster-init | ||
| {% else %} | ||
| --server https://{{ hostvars[groups['master'][0]]['ansible_host'] | default(groups['master'][0]) }}:6443 |
There was a problem hiding this comment.
Similar to the above, wouldn't this be equivalent to apiserver_endpoint?
There was a problem hiding this comment.
The apiserver_endpoint could be something other than a master host. It can be a load balancer, VIP, or something similar. You'll want to use that endpoint once the cluster is running but just to start it you just connect to any existing master host.
roles/k3s/master/tasks/main.yml
Outdated
| args: | ||
| warn: false # The ansible systemd module does not support transient units |
|
@St0rmingBr4in are you still looking to merge this, or is this PR effectively dead at this point? |
|
I don't have the time to do it, if someone wants to open a new PR I'll gladly review it and merge it. |
This enables initializing a cluster in HA mode with an embedded DB. https://rancher.com/docs/k3s/latest/en/installation/ha-embedded/ When multiple masters are specified in the master group, k3s-ansible will add the necessary flags during the initialization phase. (i.e. --cluster-init and --server) For the embedded HA mode to work the k3s version must be >= v1.19.1 Signed-off-by: Julien DOCHE <[email protected]>
This replaces the `master_ip` var by `apiserver_endpoint` for genericity. The init service is deployed only when k3s.service is not present on the machine to ensure idempotence. Signed-off-by: Julien DOCHE <[email protected]>
Signed-off-by: Julien DOCHE <[email protected]>
Signed-off-by: Julien DOCHE <[email protected]>
Signed-off-by: Julien DOCHE <[email protected]>
Signed-off-by: Julien DOCHE <[email protected]>
Allows specifying an alternate port value for the loadbalanced apiserver endpoint using a new 'apiserver_port' variable. This is required if port 6443 is already in use on the loadbalancer. Signed-off-by: Brian Brookman <[email protected]>
Signed-off-by: Brian Brookman <[email protected]>
* Remove unsupported command module "warn" parameter As of ansible-core 2.14.0, the builtin command module no longer has a "warn" parameter. Using it causes a fatal error that stops playbook completion. Signed-off-by: Brian Brookman <[email protected]> Signed-off-by: Derek Nola <[email protected]> * Relax yamllint rules on spaces inside braces Signed-off-by: Derek Nola <[email protected]> * Fix lint Signed-off-by: Derek Nola <[email protected]> --------- Signed-off-by: Brian Brookman <[email protected]> Signed-off-by: Derek Nola <[email protected]> Co-authored-by: Julien DOCHE <[email protected]> Co-authored-by: Derek Nola <[email protected]>
|
Closed in favor of #210 |
This enables initializing a cluster in HA mode with an embedded DB.
https://rancher.com/docs/k3s/latest/en/installation/ha-embedded/
When multiple masters are specified in the master group, k3s-ansible will add the necessary flags during the initialization phase (i.e. --cluster-init and --server)
For the embedded HA mode to work the k3s version must be >= v1.19.1
Closes #32