terraform upcloud: Added possibility to set up nodes with only private networks

[Xartos]

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind api-change
/kind bug
/kind cleanup
/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

This is just for review purpuse, I'll take it upstream when this is approved

This also changes the output variables master_ip and worker_ip from having the IPs in private_ip and public_ip for each node to be private and public. I tried to look and that doesn't seem to be used anywhere so it should be fine

Does this PR introduce a user-facing change?:

[robinAwallace]

LGTM, But I will try it later today 🙂

[robinAwallace]

Should there be some migration steps to move from public to private and vice verse?

[Xartos]

Hmm, yea that makes sense. I'll write some small section about that. But it's basically to just replace all nodes.

[simonklb]

When trying to apply these changes on an existing cluster I'm seeing will be updated in-place, is this not true or what do you mean by "replace all nodes"?

[robinAwallace]

I tried to create vms with just private ips and a bastion host and it works 👍

[simonklb]

The plans after switching to this branch from the base branch of this PR looks like this:

With use_public_ips=true:

  ~ master_ip           = {
      ~ simonklb-control-plane-0 = {
          + private    = "172.16.123.3"
          - private_ip = "172.16.123.3"
          + public     = "185.26.51.68"
          - public_ip  = "185.26.51.68"
        }
    }

I would not have expected any diffs here if it was completely backwards compatible. I'm not sure if there is something relying on the output variables but if there are it might be Example of backwards incompatibility https://github.com/elastisys/ck8s-devbox/pull/535 which makes it worth noting that it's not backwards compatible.

With use_public_ips=false:

  ~ master_ip           = {
      ~ simonklb-control-plane-0 = {
          + private    = "185.26.51.68"
          - private_ip = "172.16.123.3"
          - public_ip  = "185.26.51.68"
        }
    }

This looks wrong or maybe I'm not understanding what "private IPs" means here. I would have assumed the private IP would have been the local address.

[simonklb]

The overview at the top of the README looks a bit out of date, maybe add the bastion to the ASCII diagram or just remove it?

It also mentions the network setup so maybe also mention the private only setup or get rid of that as well?

[simonklb]

Being able to both have use_public_ips=false and force_public_ip=true is a bit confusing.

Without digging into the code I wouldn't really get what this meant. It kind of reads as "create a public NIC but don't use it", is that the case?

Could the documentation be clearer somehow to explain why this is useful when migrating. Is it to keep some machines in the same state as before but recreating the rest?

[simonklb]

When trying to apply these changes on an existing cluster I'm seeing will be updated in-place, is this not true or what do you mean by "replace all nodes"?

[simonklb]

Suggested change

	* `force_no_user_data`: If `private_network_dns` is set existing nodes will be deleted. This forces this mahcine to not add the user_data. Useful if you're migrating from public nodes to only private. Defaults to `false`
	* `force_no_user_data`: If `private_network_dns` is set existing nodes will be deleted. This forces this machine to not add the user_data. Useful if you're migrating from public nodes to only private. Defaults to `false`

[simonklb]

Same with this. It's not clear to me why I would need to use this variable.