Start from checkpointed container?

[alanruttenberg]

Docker has the ability to checkpoint = save state of a container which can then be restarted (and resumed, stopped, and restarted from checkpoint).

I'd like to use this to reduce my boot up time using ABCL, which is JVM based and doesn't have a save image function.

Is this something you might be interested in implementing? If not, could you suggest some places in the code which would have to be modified, and then I can give it a go?

Thanks,
Alan

[daewok]

Definitely sounds interesting, but I don't think I've ever used this feature of Docker before. Is what you're talking about accomplished with the docker pause/unpause commands? Or something else?

[alanruttenberg]

https://docs.docker.com/engine/reference/commandline/checkpoint/

It's based on https://criu.org/

As I understand pause/resume, the container stays running, but the processes inside are paused.

With checkpoint the container is stopped, but with the state of memory preserved - dumped as files into the same storage that the container uses. You can then restart the container and have it resume with that state. Once the container is stopped, you can restart it again from the same checkpointed state.

A note: This isn't fully included in the mac edition currently. For now you have to install CRIU each time the Docker app starts using these instructions. The containers and their checkpoints persist across restarts, though.

Here's a post I sent to the ABCL dev list, and it includes a recipe. It's missing one detail that became necessary in the recent docker version - you need to have to start the container with --net=host

[daewok]

Definitely a cool feature! Looks like something I'd like to add. I should be able start looking at it late next week.

I suspect the most difficult part will be just handling the swank socket. I've never disconnected from a swank server and left it running, but I know it should be possible. Curious, is running on the host network required for the container you want to checkpoint or the container you're using to install criu?

[alanruttenberg]

I think we could set things up so that the last thing the container does before the checkpoint is spawn a thread that sleeps a few seconds and then starts slime. The thread will be frozen when the checkpoint happens, but when the container restarts, it should run and start slime for us.

I'm not sure about shutting slime down, as one might want to do before a checkpoint - I'll look into that.

The host network seems to be required when starting the container that resumes the checkpoint. This was a change from the previous docker version. TBH, I don't really understand the networking stuff - for what I've been doing "it just worked". Here's the discussion where I was told about the --net=host thing.

I'm also not sure about whether the port mapping can once the container is already created. I see code that starts a container from an image exposing the port, and I've googled and found conversations that say it has to be done at that time, though those conversations were a couple of years ago. In any case, if we adopt the strategy that the container starts slime once resuming we can set the ports once when the container is created and use the same ports for slime.

[alanruttenberg]

If I understand correctly, this conversation suggests that a port can indeed be opened in a running container.

[alanruttenberg]

See restart-server in swank.lisp for how to stop and restart slime. I guess that the emacs side will complain it lost connection when you shut down the lisp side, and can then be restarted in the usual way (M-x slime - do you want to create another lisp buffer - no.)