Include cloud-init in images

[JedMeister]

As per inquiry by exoscale - cloud-init could be a good addition to TKL images (more info on exoscale blog post).

It is available in wheezy-backports

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[JedMeister]

I imagine that this could meld quite nicely with TKL inithooks (although I'm only guessing... I don't know enough about it all to be sure...)

[lirazsiri]

Supporting cloud-init looks like a good move. There's some overlap with inithooks which would need to be taken care of and tested. I say we look into this for 14.0. The main benefit is that it would make deployment of the images on other platforms that much more practical which should be a win for hosting providers / private clouds. Our current inithooks solution is AWS/Hub specific (processing the userdata).

[a3s7p]

@JedMeister, are we looking at this for 14.1?

[JedMeister]

TBH I'm not sure why we didn't do it for v14.0 but I recall there was a reason. Maybe it was just that we decided to postpone so we could do some testing to ensure that there is no negative interplay between InitHooks and CloudInit?

My thought is that if you are out of things to do and want something to look at; then please feel free to start having a look at this. As mentioned above the main issue is potential overlap with inithooks and how that may be problematic...

[JedMeister]

As a bit of explanation to the repinning of this issue to v14.2:

I had a bit of a chat to Alon about this and basically it'd be nice to include; but as there is overlap between cloud-init and inithooks, this isn't a simple "add it to the plan" type scenario (as I sort of had in my head).

We'll need to actually put a bit of effort into trying to integrate the 2 systems. We'll want current inithooks usage to remain unchanged (so current users don't have to retool); although it would be good to allow cloud-init to support more of inithooks functionality.

[a3s7p]

@JedMeister, as I've started working with this, it is fairly straight-forward to disable all the functionality that could interfere with inithooks leaving just what the end users might want to use (Puppet, Chef, Salt, custom scripts) and it also provides some general-case functionality like the message on root login (#541) and blocking the access to the metadata server after launch (security) so it would be a beneficial addition.

It's not a simple «add to the plan» but it's not much more than changing the default config — there are (optional) modules for running regular scripts which are parallel to inithooks and some that replicate the functionality but if we leave the rest on it wouldn't do any harm in my POV. :-)

Now, if we're actually to integrate the two systems, which would probably mean delegating more of inithooks' responsibilities to cloud-init that is indeed more sensible as a goal for v14.2. If it's to be just an add-on I don't see why we can't do it for v14.1 (yet).

However, another thing that should be mentioned is that by default, cloud-init precedes inithooks in boot ordering. I am not sure which would be the better way.

[JedMeister]

FWIW Alon contributed to cloud-init back when TurnKey was Ubuntu based. My understanding was that the original plan was to essentially merge cloud-init and inithooks; but then we diverged from Ubuntu and the plan was abandoned. (Although I perhaps have the story wrong...)

It was only because of a hosting provider's interest in hosting TurnKey (but unwillingness to do it without cloud-init support) that I lodged this feature request. Also it seemed like if it was done right it could extend the power of TurnKey's initialisation without too much work...

But as you see I probably didn't really have a good understanding of cloud-init, nor what it does ...

Anyway, back to the job at hand; you raise a really good point. And if you are already close and so long as it doesn't interfere with existing inithooks that seems totally reasonable.

The only downside is that we'll really need to give it a good testing as if there is a bug somewhere that interferes with things then we risk breaking all the appliances...! IMO we really need to ensure that the inclusion has no effect on current inithooks functionality (even if that means reducing the usefulness/functionality of cloud-init).

[a3s7p]

Also it seemed like if it was done right it could extend the power of TurnKey's initialisation without too much work...

That's what it looks like to me, too.

I am fairly close but I did not test extensively yet which I agree is important. Provided I find no issues, should it be good for inclusion in v14.1?

As I said, it brings benefit even if everything user-controlled is disabled so we could go that route if it turns out to be problematic otherwise. But I think it should be fine with a minimum of user-controlled modules just as well and I would like to see it in as such ASAP which would be v14.1.

[vondrt4]

@qrntz Could I please see what you have? I'm planning to start work on OpenStack bugs (mainly filesystem resizing and password setting) and wouldn't like to duplicate your work.

[JedMeister]

@vondrt4 - That'd be awesome! It's @qrntz 's birthday this weekend and so he's gone away. I'm sure he'll post back early mid to late next week.

Re filesystem resizing does the autogrow-fs inithook not suit your purpose (or not run/work in OpenStack builds)?

[vondrt4]

Nope. I've already researched that. That works with LVM. This one for EC2 looks promising. But if we had cloud-init, we would not have to care about that.
Currently, the OpenStack images in EC2 compatibility mode (AMI) have only a rootfs on /dev/vda. You can just issue resize2fs /dev/vda. But I would like to build the Openstack builds as qcow2 with a single partition on /dev/vda1, as is customary now. I believe Amazon is moving towards Xen HVM and this image architecture as well.
Bear with me while I learn TKLDEV.

[vondrt4]

One more comment on cloud-init.
The OpenStack [image requirements](http://docs.openstack.org/image-guide/openstack-
images.html) page lists all things and image should do. They are usually covered by cloud-init. However the Inithooks already do most if not all of this work. So to have an OpenStack-compliant image, cloud-init is not a strict requirement.
In this doc and this presentation are examples of things cloud-init can do. Of course any of the modules can be turned of in its config file on the VM image.
I don't have Heat on my cloud for example, because there are few publicly available templates for it, but one use-case for cloud-init is customization of a VM to be a part of a greater stack. In the templates I've seen, bash scripts were about as common as cloud-config snippets, if not more. Inithooks can do that as well. And IMHO TKL is will not usually be used as a part of a Heat stack.

[JedMeister]

Awesome! 👍

Its clear to me that you have a great understanding and are well researched. Definitely know much more about it than me (which TBH wouldn't be too hard...). It'd be so great for our OpenStack images to get some love because I'm well aware that they are a poor cousin at the moment...

Thanks tons.and I really look forward to seeing what happens. 😄

[JedMeister]

@qrntz - could you please share what you have when you get a chance mate.

[vondrt4]

Prototype for OpenStack here. turnkeylinux/buildtasks#23
On EC2, there may be some more work, because of the Hub.

[a3s7p]

Sorry for leaving you hanging, @vondrt4!

Anyway, I see that your approach was more radical than mine. When I worked on cloud-init, we agreed to rather restrict the scope of operation of cloud-init so that inithooks would be unaffected. So the changes were rather minimal and should not contribute much to your effort since you've went way further with it.

You can see the code at https://github.com/qrntz/buildtasks/tree/cloud-init.

[JedMeister]

We still haven't got this done yet unfortunately, moving to v15.1. Hopefully we can actually implement it then!

[l-arnold]

This appears that it would be very helpful. When I look at the mirrors it appears that OpenStack (in TKL usage) stopped at version 15x. XEN at the same time.
I used to be able to easily take a XEN build and modify it so that it would load to Linode. Now the XEN build tasks fail in 16.x and I expect similar in OpenStack (qcow is thought to be an Image Drop in - but I have not done it that way yet).
Perhaps there is a workaround with OVA or VMDK, but really a "straight up" easy install would be best.
A few Linode Links re cloud-init
https://github.com/displague/terraform-linode-cloudinit-example
and
"Linode will also support Cloud Init and a Metadata Service."...
but: I also see that "cloud-init" is not there yet.

Thinking outloud here a bit. Wouldn't be cool if you could initialize a new Debian server with Linode/Rackspace etc. Login via SSH/Weblish etc. Type "Install Turnkey Core" (etc) and be on your way.

Looks like I need to spin up a server to run TKLDEV 17 just now.

[l-arnold]

Debian Centric: https://packages.debian.org/sid/cloud-init

[JedMeister]

it appears that OpenStack (in TKL usage) stopped at version 15x. XEN at the same time.

That is because we don't have any Xen infrastructure ourselves and when we put the word out for testing, no one responded. So we just never published them.

Thinking outloud here a bit. Wouldn't be cool if you could initialize a new Debian server with Linode/Rackspace etc. Login via SSH/Weblish etc. Type "Install Turnkey Core" (etc) and be on your way.

Yes that would! FWIW, it's something that we have discussed internally, but we're nowhere near a point where we could reliably package that (nor are we 100% clear on how we would package it).

Debian Centric: https://packages.debian.org/sid/cloud-init

Yep, Debian have cloud-init packaged, so installing it is easy. Unfortunately though, it overlaps/conflicts with inithooks. As you can see in the history of this issue, there have been attempts to work around that conflict - but we need to revisit this... I certainly hope to at some point, but currently I have a huge backlog and are really only at the start of the v17.0 release...

[l-arnold]

Been working, slowly, to get an ISO built TKL Odoo Linode built. I mentioned several times that I as using the XEN build to get through. It was not perfect but way easier than the ISO process. (For the life of me I don't know why TKL's Grub is not recognzized by Linodes).. Anyway, trying to help. If I can get this worked through I can try TKLDEV 17 rc1.

[l-arnold]

As soon as I sent that I got it building.
I do have the issue that none of the "support structure" from Linode is there when I build it this way. I can't use their backups or easy migrations.
I remain optimistic though.. Might be my first v 16.x to go through this way.

I'll see what I can do to help the TKL VM's a long. (I forked the XEN BT a while ago, but as I noted, it stopped working in 16.x)