crun should never checkpoint the netns

[Luap99]

Basically the same as 2a0947e but this made the exception to checkpoint the netns when the netns path is empty in the runtime spec. This works only for the case where podman creates a netns in advance but this is not always the case, i.e. when a custom userns is used (which also doesn't work in crun right now but this is a different issue #1207).

The problem now is that I want to consolidate the network setup code in podman containers/podman#18468 to only use one setup path instead of two for with and without userns. So going forward I always want to let the runtime create the netns (empty netns path in config) and after the create call configure the netns in podman. This works just fine except for the checkpoint/restore case. On restore criu tries to restore the netns which fails:

(00.048662)      1: Try to restore a link 10:2:eth0
(00.048676)      1: Restoring link eth0 type 2
(00.048691)      1: Restoring netdev eth0 idx 2
(00.048705)      1: Restore ll addr (8e:../6) for device
(00.048712)      1: Error (criu/net.c:1462): Unknown peer net namespace
(00.063166)      1: Error (criu/libnetlink.c:54): -16 reported by netlink: Device or resource busy
(00.063205)      1: Error (criu/net.c:1816): Can't restore link: -16
(00.063270)      1: Error (criu/util.c:1411): Can't wait or bad status: errno=0, status=65280
(00.064717) Error (criu/cr-restore.c:2536): Restoring FAILED.

The same commands work just fine with runc because they always ignore the netns. opencontainers/runc@8187fb7

So my ask is to always ignore the netns to match runc behavior and allow poman to work correctly.

[adrianreber]

I had a look at it and can provide a fix. Different then in runc, but similar.

[adrianreber]

This first needs changes in libcriu. The needed interface has not been exported, yet. I will open a PR in CRIU first.

[adrianreber]

See checkpoint-restore/criu#2175 for the CRIU changes.

[Luap99]

@adrianreber Is there a way we can move forward here or do we need to wait on a new criu release?

[adrianreber]

I see CI runs based on Ubuntu. I think we can update CRIU in Fedora and the Ubuntu PPA to include the necessary patch in the current release without waiting for a new release.

@rst0git Can you update the CRIU PPA to 3.18 with the patch from checkpoint-restore/criu#2175 (maybe also the Sapphire Rapids patch)?

[Luap99]

Any update here? It would be great if we could get this fixed.

[giuseppe]

@adrianreber is the new API usable by crun?

[rst0git]

@giuseppe Yes, it should be available with something like libcriu_wrapper->criu_set_empty_ns(CLONE_NEWNET)

The following patches address similar problems in runc:

• opencontainers/runc@acef746
• opencontainers/runc@22d60d9
• opencontainers/runc@8187fb7
• opencontainers/runc@4803faf

[giuseppe]

thanks, if the network namespace is set by default to empty, how can you set a different value? I see only --empty-ns in runc checkpoint --help

[rst0git]

The --empty-ns option is used to tell CRIU to create a namespace during restore, but without restoring its properties. As far as I understand, the reason this option is used by default for the network namespace is because container networking is not managed by runc/crun.

I see only --empty-ns in runc checkpoint --help

$ criu --help | grep empty-ns
  --empty-ns net        Create a namespace, but don't restore its properties

$ runc checkpoint --help | grep empty-ns
   --empty-ns value             create a namespace, but don't restore its properties

$ runc restore --help | grep empty-ns
   --empty-ns value             create a namespace, but don't restore its properties

how can you set a different value?

What do you mean by setting a different value? CRIU's CLI option is intended to be used as follows: --empty-ns net:
https://github.com/checkpoint-restore/criu/blob/fd353fdd8eba6f5e094a3bd474edf9b39a58cca6/criu/config.c#L969

[giuseppe]

I mean, there is no way to checkpoint the current network namespace now, because there is only an --empty-ns, but that is already the default value for the network namespace. So how can someone say "checkpoint the current network namespace" (which is the behavior crun has right now)?

[rst0git]

I believe the network namespace will be fully checkpointed and restored when this option is not used. According to the message in commit opencontainers/runc@8187fb7, this option is set by default:

cr: don't dump network devices and their configuration

RunC doesn't manage network devices and their configuration,
so it is impossible to describe external dependencies to restore them
back.

This means that all users have to set --empty-ns network, so let's do
this by default.

However, I believe the current behaviour of crun and runc is to set the network namespace as external:

• criu: tell CRIU to not touch the network namespace #322
• Add support to checkpoint and restore into external network namespaces opencontainers/runc#1849