Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Single node support #319

Merged
merged 7 commits into from
Aug 29, 2024
Merged

Single node support #319

merged 7 commits into from
Aug 29, 2024

Conversation

masnax
Copy link
Contributor

@masnax masnax commented Jun 12, 2024

Adds basic single-node support.

This adds an initial question to microcloud init asking if the user wants to set up just one node.

Do you want to set up more than one cluster member? (yes/no) [default=yes]: 

If the user enters no, the system will follow the ordinary setup with ZFS and OVN (or optionally a FAN network instead), but the user will not be prompted for a lookup subnet and will not be presented a list of systems.

The cluster can be grown after this point by using microcloud add.

TODO:

  • Add tests
  • Support preseed
  • Support adding missing services (maybe, if it makes sense)
  • Test MicroOVN functions properly with 1 node.

@masnax
Copy link
Contributor Author

masnax commented Jul 23, 2024

Right now, this assumes the current behaviour where MicroCeph requires 3 nodes. This can be configured, but will require changing a lot of the tests which check for MicroCeph to be skipped with fewer than 3 nodes.

Preseed is also ignored here. We arbitrarily require at least 2 nodes for preseed but this doesn't necessarily have to be the case anymore.

@masnax masnax force-pushed the single-node branch 2 times, most recently from dc35488 to b41c76f Compare August 20, 2024 02:55
@masnax masnax marked this pull request as ready for review August 27, 2024 14:52
Copy link
Contributor

@roosterfish roosterfish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, only two small comments.

What is missing when it comes to adding existing services? Do you think about something like deploying a single node MicroCloud without OVN and then adding OVN to it later using service add?

Should the option of single node MicroCloud also be reflected in the docs? I was thinking about something like https://canonical-microcloud.readthedocs-hosted.com/en/latest/microovn/tutorial/single-node/.

The PRs description still says Ceph will be skipped but after #378 got merged I think this sentence can be removed too.

cmd/microcloud/main_init.go Outdated Show resolved Hide resolved
cmd/microcloud/main_init_preseed.go Show resolved Hide resolved
@roosterfish
Copy link
Contributor

As mentioned earlier the only other thing I found with this PR is adding services (like MicroOVN) after the single node MicroCloud is deployed seems to be not working currently.

It reports Error: All services have already been set up:

root@m1:~# microcloud init
Waiting for LXD to start ...
Do you want to set up more than one cluster member? (yes/no) [default=yes]: no
Select an address for MicroCloud's internal traffic:

 Using address "10.165.133.71" for MicroCloud

Gathering system information ...
Would you like to set up local storage? (yes/no) [default=yes]:  
Select exactly one disk from each cluster member:

Select which disks to wipe:

 Using "/dev/disk/by-id/scsi-SQEMU_QEMU_HARDDISK_lxd_volm1--local" on "m1" for local storage pool

Would you like to set up distributed storage? (yes/no) [default=yes]: 
Select from the available unpartitioned disks:

Select which disks to wipe:

Disk configuration does not meet recommendations for fault tolerance. At least 3 systems must supply disks.
Continuing with this configuration will leave MicroCloud susceptible to data loss
Change disk selection? (yes/no) [default=yes]: no
 Using 1 disk(s) on "m1" for remote storage pool

Do you want to encrypt the selected disks? (yes/no) [default=no]: 
Would you like to set up CephFS remote storage? (yes/no) [default=yes]: 
What subnet (either IPv4 or IPv6 CIDR notation) would you like your Ceph internal traffic on? [default: 10.165.133.0/24] 
Configure distributed networking? (yes/no) [default=yes]: no
Initializing new services
 Local MicroCloud is ready
 Local LXD is ready
 Local MicroOVN is ready
 Local MicroCeph is ready
Awaiting cluster formation ...
Configuring cluster-wide devices ...
MicroCloud is ready
root@m1:~# microcloud service add
Error: All services have already been set up

@masnax
Copy link
Contributor Author

masnax commented Aug 28, 2024

@roosterfish Ah I see, it's just a miscommunication. microcloud service add does not let you re-configure services that are already clustered, it initializes new services that were not previously initialized.

For example, if MicroOVN was not installed when microcloud init was first run, but installed later. You can then use microcloud service add to add MicroOVN to the MicroCloud cluster.

You get the error Error: All services have already been set up because MicroOVN has already been initialized by MicroCloud during microcloud init, the configuration on the LXD side is what was skipped.

For example:

snap remove microovn --purge
microcloud init
...
snap install microovn
microcloud service add

We've been discussing adding a microcloud service edit to change the configuration of the already-installed services, but that likely won't be part of the LTS release.

@roosterfish
Copy link
Contributor

roosterfish commented Aug 29, 2024

You get the error Error: All services have already been set up because MicroOVN has already been initialized by MicroCloud during microcloud init, the configuration on the LXD side is what was skipped.

Thanks I see but is it a bug that MicroCloud initializes MicroOVN as you can see in the output above I have proactively answered no on the Configure distributed networking? question?

It gave me a single node MicroCloud using fan networking so I would suspect MicroCloud doesn't even look at MicroOVN in this situation.

@masnax
Copy link
Contributor Author

masnax commented Aug 29, 2024

You get the error Error: All services have already been set up because MicroOVN has already been initialized by MicroCloud during microcloud init, the configuration on the LXD side is what was skipped.

Thanks I see but is it a bug that MicroCloud initializes MicroOVN as you can see in the output above I have proactively answered no on the Configure distributed networking? question?

It gave me a single node MicroCloud using fan networking so I would suspect MicroCloud doesn't even look at MicroOVN in this situation.

Configure != Initialize. MicroCloud will form the MicroOVN cluster if the snap is installed irrespective of whether or not you want to configure the network in LXD at the precise moment you ran microcloud init. The primary objective is to create the clusters. If a user wants to configure their network some other way, then at least they will have the MicroOVN cluster formed.

Copy link
Contributor

@roosterfish roosterfish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@masnax masnax merged commit 8fe8668 into canonical:main Aug 29, 2024
14 of 15 checks passed
@token47
Copy link

token47 commented Aug 30, 2024

Sunbeam uses very similar approach and can do microceph in one node. Can't the same be done for microcloud?

@masnax
Copy link
Contributor Author

masnax commented Aug 30, 2024

Sunbeam uses very similar approach and can do microceph in one node. Can't the same be done for microcloud?

If you're referring to the line in the PR description about MicroCeph being skipped, that was out of date and is now removed. Single node does support MicroCeph.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants