terraform upcloud: Added possibility to set up nodes with only private networks

[Xartos]

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind api-change
/kind bug
/kind cleanup
/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

This is just for review purpuse, I'll take it upstream when this is approved

This also changes the output variables master_ip and worker_ip from having the IPs in private_ip and public_ip for each node to be private and public. I tried to look and that doesn't seem to be used anywhere so it should be fine

Does this PR introduce a user-facing change?:

[robinAwallace]

LGTM, But I will try it later today 🙂

[robinAwallace]

Should there be some migration steps to move from public to private and vice verse?

[Xartos]

Hmm, yea that makes sense. I'll write some small section about that. But it's basically to just replace all nodes.

[simonklb]

When trying to apply these changes on an existing cluster I'm seeing will be updated in-place, is this not true or what do you mean by "replace all nodes"?

[Xartos]

The issue is that the indexing for the interfaces in terraform matter. So if you remove the public IP it will look like it's in place. But in reality it will recreate the interface completely, so it will get a new IP. See this issue

[simonklb]

That's scary... :o

[robinAwallace]

I tried to create vms with just private ips and a bastion host and it works 👍

[simonklb]

The plans after switching to this branch from the base branch of this PR looks like this:

With use_public_ips=true:

  ~ master_ip           = {
      ~ simonklb-control-plane-0 = {
          + private    = "172.16.123.3"
          - private_ip = "172.16.123.3"
          + public     = "185.26.51.68"
          - public_ip  = "185.26.51.68"
        }
    }

I would not have expected any diffs here if it was completely backwards compatible. I'm not sure if there is something relying on the output variables but if there are it might be Example of backwards incompatibility https://github.com/elastisys/ck8s-devbox/pull/535 which makes it worth noting that it's not backwards compatible.

With use_public_ips=false:

  ~ master_ip           = {
      ~ simonklb-control-plane-0 = {
          + private    = "185.26.51.68"
          - private_ip = "172.16.123.3"
          - public_ip  = "185.26.51.68"
        }
    }

This looks wrong or maybe I'm not understanding what "private IPs" means here. I would have assumed the private IP would have been the local address.

Response: #20 (comment)

[simonklb]

When trying to apply these changes on an existing cluster I'm seeing will be updated in-place, is this not true or what do you mean by "replace all nodes"?

[robinAwallace]

@davidumea @Xartos any updates for this? Or should we try to review this again and merge it?

[davidumea]

@davidumea @Xartos any updates for this? Or should we try to review this again and merge it?

I thought about making proxy_protocol an optional variable, thoughts? Other than that I have nothing more to add here.

[Xartos]

This looks wrong or maybe I'm not understanding what "private IPs" means here. I would have assumed the private IP would have been the local address.

@simonklb Yea, this is because of this bug. If you run plan after it has been applied it should show you the correct private IP.

[simonklb]

Only a non-blocker that I think would make this change look a bit less "migration-oriented". It looks like it does the job though and I appreciate the work done to not break existing machines!