Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

24.0.7 rp #2

Open
wants to merge 447 commits into
base: master
Choose a base branch
from
Open

24.0.7 rp #2

wants to merge 447 commits into from

Conversation

ef0xa
Copy link

@ef0xa ef0xa commented Apr 4, 2024

- What I did

- How I did it

- How to verify it

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

thaJeztah and others added 30 commits July 2, 2023 12:41
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 18d77ff)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit b770a50)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This field's documentation was still referring to the Swarm V1 API, which
is deprecated, and the link redirects to SwarmKit.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit ed0dbb8)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This has been around for a long time - since v17.04 (API v1.28)
but was never documented.

It allows removing a plugin even if it's still in use.

Signed-off-by: Milas Bowman <[email protected]>
(cherry picked from commit eb0edea)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This option was added in 8cb2229 for
API version 1.28, but forgot to update the documentation and version
history.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit aba8e04)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This option was added in 8cb2229 for
API version 1.28, but forgot to update the documentation and version
history.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 85ccb25)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This option was added in 8cb2229 for
API version 1.28, but forgot to update the documentation and version
history.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit a4bdfb9)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This option was added in 8cb2229 for
API version 1.28, but forgot to update the documentation and version
history.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 892e9f2)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This option was added in 8cb2229 for
API version 1.28, but forgot to update the documentation and version
history.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit f6258f7)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
…_deprecated_swarm

[24.0 backport] docs: api: remove outdated information from ServerVersion
…n_disable_force_carry

[24.0 backport] docs: api: add missing "force" query arg on plugin disable
…oups

[24.0 backport] vendor: github.com/containerd/cgroups/v3 v3.0.2
The daemon.lazyInitializeVolume() function only handles restoring Volumes
if a Driver is specified. The Container's MountPoints field may also
contain other kind of mounts (e.g., bind-mounts). Those were ignored, and
don't return an error; https://github.com/moby/moby/blob/1d9c8619cded4657af1529779c5771127e8ad0e7/daemon/volumes.go#L243-L252C2

However, the prepareMountPoints() assumed each MountPoint was a volume,
and logged an informational message about the volume being restored;
https://github.com/moby/moby/blob/1d9c8619cded4657af1529779c5771127e8ad0e7/daemon/mounts.go#L18-L25

This would panic if the MountPoint was not a volume;

    github.com/docker/docker/daemon.(*Daemon).prepareMountPoints(0xc00054b7b8?, 0xc0007c2500)
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/mounts.go:24 +0x1c0
    github.com/docker/docker/daemon.(*Daemon).restore.func5(0xc0007c2500, 0x0?)
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:552 +0x271
    created by github.com/docker/docker/daemon.(*Daemon).restore
            /root/rpmbuild/BUILD/src/engine/.gopath/src/github.com/docker/docker/daemon/daemon.go:530 +0x8d8
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x564e9be4c7c0]

This issue was introduced in 647c2a6

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit a490248)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
…_npe

[24.0 backport] daemon: daemon.prepareMountPoints(): fix panic if mount is not a volume
TestClientWithRequestTimeout has been observed to flake in CI. The
timing in the test is quite tight, only giving the client a 10ms window
to time out, which could potentially be missed if the host is under
load and the goroutine scheduling is unlucky. Give the client a full
five seconds of grace to time out before failing the test.

Signed-off-by: Cory Snider <[email protected]>
(cherry picked from commit 9cee34b)
Signed-off-by: Cory Snider <[email protected]>
Multiple daemons starting/running concurrently can collide with each
other when editing iptables rules. Most integration tests which opt into
parallelism and start daemons work around this problem by starting the
daemon with the --iptables=false option. However, some of the tests
neglect to pass the option when starting or restarting the daemon,
resulting in those tests being flaky.

Audit the integration tests which call t.Parallel() and (*Daemon).Stop()
and add --iptables=false arguments where needed.

Signed-off-by: Cory Snider <[email protected]>
(cherry picked from commit cdcb7c2)
Signed-off-by: Cory Snider <[email protected]>
…-flakiness

[24.0 backport] make tests less flaky
A panic would happen when converting an config that had ports exposed, because
the ExposedPorts map in the OCI-spec was not initialized. This could happen
when committing a container, or when using the classic builder and the
parent image had ports exposed, for example

    FROM busybox AS stage0
    EXPOSE 80

    FROM stage0 AS stage1
    RUN echo hello

Example of the panic:

    2023/07/07 15:13:02 http: panic serving @: assignment to entry in nil map
    goroutine 1944 [running]:
    net/http.(*conn).serve.func1()
    	/usr/local/go/src/net/http/server.go:1854 +0xbf
    panic({0x45f660, 0xb6a8d0})
    	/usr/local/go/src/runtime/panic.go:890 +0x263
    github.com/docker/docker/daemon/containerd.containerConfigToOciImageConfig(...)
    	/go/src/github.com/docker/docker/daemon/containerd/image_import.go:397
    github.com/docker/docker/daemon/containerd.generateCommitImageConfig({0xc001470498, {0x0, 0x0}, {0xc000c437d8, 0x5}, {0x0, 0x0}, {0xc000c43b27, 0x5}, {0x0, ...}, ...}, ...)
    	/go/src/github.com/docker/docker/daemon/containerd/image_commit.go:138 +0x40e
    github.com/docker/docker/daemon/containerd.(*ImageService).CommitImage(0xc0008853e0, {0xb8f660, 0xc000c4f7c0}, {{0x0, 0x0}, {0x0, 0x0}, 0xc00104b900, 0xc00104b180, {0xc0011a7640, ...}, ...})
    	/go/src/github.com/docker/docker/daemon/containerd/image_commit.go:82 +0x73b
    github.com/docker/docker/daemon/containerd.(*ImageService).CommitBuildStep(0xc0008853e0, {0xb8f660, 0xc000c4f7c0}, {{0x0, 0x0}, {0x0, 0x0}, 0xc00104b900, 0xc00104b180, {0xc0011a7640, ...}, ...})
    	/go/src/github.com/docker/docker/daemon/containerd/image_commit.go:308 +0x110
    github.com/docker/docker/builder/dockerfile.(*Builder).commitContainer(0xc0012b8cc0, {0xb8f660, 0xc000c4f7c0}, 0xc0010b2b60, {0xc0011a7640, 0x40}, 0xc00104b180)
    	/go/src/github.com/docker/docker/builder/dockerfile/internals.go:61 +0x168
    github.com/docker/docker/builder/dockerfile.(*Builder).commit(0xc0012b8cc0, {0xb8f660, 0xc000c4f7c0}, 0xc0010b2b60, {0xc0012a7d80?, 0xc001340060?})
    	/go/src/github.com/docker/docker/builder/dockerfile/internals.go:45 +0x1aa
    github.com/docker/docker/builder/dockerfile.dispatchLabel({0xb8f660, 0xc000c4f7c0}, {0xc0010b2b60, 0xc000c6b628, 0xc0012b8cc0, {0xb80f60, 0xc0011a46c0}, 0xc000bc2560}, 0x1e24a85?)
    	/go/src/github.com/docker/docker/builder/dockerfile/dispatchers.go:83 +0x258
    github.com/docker/docker/builder/dockerfile.dispatch({0xb8f660, 0xc000c4f7c0}, {0xc0010b2b60, 0xc000c6b628, 0xc0012b8cc0, {0xb80f60, 0xc0011a46c0}, 0xc000bc2560}, {0xb7be40, 0xc00111cde0})
    	/go/src/github.com/docker/docker/builder/dockerfile/evaluator.go:74 +0x529
    github.com/docker/docker/builder/dockerfile.(*Builder).dispatchDockerfileWithCancellation(0xc0012b8cc0, {0xb8f660, 0xc000c4f7c0}, {0xc000b1d380, 0x1, 0xc0011a4660?}, {0x0, 0x0, 0x0?}, 0x5c, ...)
    	/go/src/github.com/docker/docker/builder/dockerfile/builder.go:296 +0x8f2
    github.com/docker/docker/builder/dockerfile.(*Builder).build(0xc0012b8cc0, {0xb8f660, 0xc000c4f7c0}, {0xb80f60, 0xc0011a46c0}, 0xc0011a49f0)
    	/go/src/github.com/docker/docker/builder/dockerfile/builder.go:211 +0x2e5
    github.com/docker/docker/builder/dockerfile.(*BuildManager).Build(0xc0008868c0, {0xb8f708, 0xc0011a44b0}, {{0xb789c0, 0xc0011a4540}, {{0xb6b940, 0xc000c22a50}, {0xb6c5e0, 0xc000c22a68}, {0xb6c5e0, ...}, ...}, ...})
    	/go/src/github.com/docker/docker/builder/dockerfile/builder.go:98 +0x358
    github.com/docker/docker/api/server/backend/build.(*Backend).Build(0xc0007d0870, {0xb8f708, 0xc0011a44b0}, {{0xb789c0, 0xc0011a4540}, {{0xb6b940, 0xc000c22a50}, {0xb6c5e0, 0xc000c22a68}, {0xb6c5e0, ...}, ...}, ...})
    	/go/src/github.com/docker/docker/api/server/backend/build/backend.go:69 +0x186
    github.com/docker/docker/api/server/router/build.(*buildRouter).postBuild(0xc0008333c0, {0xb8f708, 0xc0011a44b0}, {0xb8e130, 0xc0000ed500}, 0xc0010d4800, 0xc0012df760?)
    	/go/src/github.com/docker/docker/api/server/router/build/build_routes.go:280 +0x7a6
    github.com/docker/docker/api/server/middleware.ExperimentalMiddleware.WrapHandler.func1({0xb8f708, 0xc0011a44b0}, {0xb8e130?, 0xc0000ed500?}, 0x36cf80?, 0xc0010ab550?)
    	/go/src/github.com/docker/docker/api/server/middleware/experimental.go:26 +0x15b
    github.com/docker/docker/api/server/middleware.VersionMiddleware.WrapHandler.func1({0xb8f708, 0xc0011a4480}, {0xb8e130, 0xc0000ed500}, 0xc000d787e8?, 0xc000d787a0?)
    	/go/src/github.com/docker/docker/api/server/middleware/version.go:62 +0x4d7
    github.com/docker/docker/pkg/authorization.(*Middleware).WrapHandler.func1({0xb8f708, 0xc0011a4480}, {0xb8e130?, 0xc0000ed500?}, 0xc0010d4800, 0xc0010ab500?)
    	/go/src/github.com/docker/docker/pkg/authorization/middleware.go:59 +0x649
    github.com/docker/docker/api/server.(*Server).makeHTTPHandler.func1({0xb8e130, 0xc0000ed500}, 0xc0010d4700)
    	/go/src/github.com/docker/docker/api/server/server.go:53 +0x1ce
    net/http.HandlerFunc.ServeHTTP(0xc0010d4600?, {0xb8e130?, 0xc0000ed500?}, 0xc000d789e8?)
    	/usr/local/go/src/net/http/server.go:2122 +0x2f
    github.com/docker/docker/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc0001a7e00, {0xb8e130, 0xc0000ed500}, 0xc000d37600)
    	/go/src/github.com/docker/docker/vendor/github.com/gorilla/mux/mux.go:210 +0x1cf
    net/http.serverHandler.ServeHTTP({0xb7ec58?}, {0xb8e130, 0xc0000ed500}, 0xc000d37600)
    	/usr/local/go/src/net/http/server.go:2936 +0x316
    net/http.(*conn).serve(0xc0012661b0, {0xb8f708, 0xc000fd0360})
    	/usr/local/go/src/net/http/server.go:1995 +0x612
    created by net/http.(*Server).Serve
    	/usr/local/go/src/net/http/server.go:3089 +0x5ed

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit a0e1155)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
…_npe

[24.0 backport] daemon/containerd: fix assignment to entry in nil map during commit
Signed-off-by: CrazyMax <[email protected]>
(cherry picked from commit ee9fe2c)
Calling function returned from setupTest (which calls testEnv.Clean) in
a defer block inside a test that spawns parallel subtests caused the
cleanup function to be called before any of the subtest did anything.

Change the defer expressions to use `t.Cleanup` instead to call it only
after all subtests have also finished.
This only changes tests which have parallel subtests.

Signed-off-by: Paweł Gronowski <[email protected]>
(cherry picked from commit f9e2eed)
Signed-off-by: Paweł Gronowski <[email protected]>
[24.0 backport] integration-cli: Add t.Helper() to the cli test helper functions
[backport 24.0] integration: Don't env cleanup before parallel subtests
Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 2a59188)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.

The current code used the client's `addr` as hostname in some cases, which
could contain the path for the unix-socket (`/var/run/docker.sock`), which
gets rejected by go1.20.6 and go1.19.11 because of a security fix for
[CVE-2023-29406 ][1], which was implemented in  https://go.dev/issue/60374.

Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.

This patch introduces a `DummyHost` const, and uses this dummy host for
cases where we don't need an actual hostname.

Before this patch (using go1.20.6):

    make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
    === RUN   TestAttachWithTTY
        attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
    --- FAIL: TestAttachWithTTY (0.11s)
    === RUN   TestAttachWithoutTTy
        attach_test.go:46: assertion failed: error is not nil: http: invalid Host header
    --- FAIL: TestAttachWithoutTTy (0.02s)
    FAIL

With this patch applied:

    make GO_VERSION=1.20.6 TEST_FILTER=TestAttach test-integration
    INFO: Testing against a local daemon
    === RUN   TestAttachWithTTY
    --- PASS: TestAttachWithTTY (0.12s)
    === RUN   TestAttachWithoutTTy
    --- PASS: TestAttachWithoutTTy (0.02s)
    PASS

[1]: GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 92975f0)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname.

The current code used the socket path as hostname, which gets rejected by
go1.20.6 and go1.19.11 because of a security fix for [CVE-2023-29406 ][1],
which was implemented in  https://go.dev/issue/60374.

Prior versions go Go would clean the host header, and strip slashes in the
process, but go1.20.6 and go1.19.11 no longer do, and reject the host
header.

Before this patch, tests would fail on go1.20.6:

    === FAIL: pkg/authorization TestAuthZRequestPlugin (15.01s)
    time="2023-07-12T12:53:45Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 1s"
    time="2023-07-12T12:53:46Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 2s"
    time="2023-07-12T12:53:48Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 4s"
    time="2023-07-12T12:53:52Z" level=warning msg="Unable to connect to plugin: //tmp/authz2422457390/authz-test-plugin.sock/AuthZPlugin.AuthZReq: Post \"http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq\": http: invalid Host header, retrying in 8s"
        authz_unix_test.go:82: Failed to authorize request Post "http://%2F%2Ftmp%2Fauthz2422457390%2Fauthz-test-plugin.sock/AuthZPlugin.AuthZReq": http: invalid Host header

[1]: GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 6b7705d)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit e1db9e9)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
thaJeztah and others added 30 commits October 18, 2023 15:32
full diff: golang/crypto@v0.2.0...v0.3.0

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 7b66ae1)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
full diff: golang/sys@v0.8.0...v0.10.0

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit a046616)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
full diff: golang/crypto@v0.3.0...v0.9.0

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 9670db3)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
full diff: golang/text@v0.9.0...v0.11.0

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 87de23a)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
full diff: golang/net@v0.10.0...v0.13.0
full diff: golang/crypto@v0.9.0...v0.11.0

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 2edb858)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
full diff: golang/sys@v0.10.0...v0.13.0

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit bc33c27)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
full diff: golang/text@v0.11.0...v0.13.0

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 0679ae9)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
full diff: golang/crypto@v0.11.0...v0.14.0

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 1946aea)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
full diff: golang/net@v0.13.0...v0.17.0

This fixes the same CVE as go1.21.3 and go1.20.10;

- net/http: rapid stream resets can cause excessive work

  A malicious HTTP/2 client which rapidly creates requests and
  immediately resets them can cause excessive server resource consumption.
  While the total number of requests is bounded to the
  http2.Server.MaxConcurrentStreams setting, resetting an in-progress
  request allows the attacker to create a new request while the existing
  one is still executing.

  HTTP/2 servers now bound the number of simultaneously executing
  handler goroutines to the stream concurrency limit. New requests
  arriving when at the limit (which can only happen after the client
  has reset an existing, in-flight request) will be queued until a
  handler exits. If the request queue grows too large, the server
  will terminate the connection.

  This issue is also fixed in golang.org/x/net/http2 v0.17.0,
  for users manually configuring HTTP/2.

  The default stream concurrency limit is 250 streams (requests)
  per HTTP/2 connection. This value may be adjusted using the
  golang.org/x/net/http2 package; see the Server.MaxConcurrentStreams
  setting and the ConfigureServer function.

  This is CVE-2023-39325 and Go issue https://go.dev/issue/63417.
  This is also tracked by CVE-2023-44487.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 1800dd0)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Add IP_NF_MANGLE to "Generally Required" kernel features, since it appears to be necessary for Docker Swarm to work.

Closes moby#46636

Signed-off-by: Stephan Henningsen <[email protected]>
(cherry picked from commit cf90733)
Signed-off-by: Bjorn Neergaard <[email protected]>
[24.0 backport] Add IP_NF_MANGLE to check-config.sh
[24.0 backport] vendor: golang.org/x/net v0.17.0
Copy the implementation of `context.WithoutCancel` introduced in Go 1.21
to be able to use it when building with older versions.
This will use the stdlib directly when building with Go 1.21+.

Signed-off-by: Paweł Gronowski <[email protected]>
(cherry picked from commit f6e44bc)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Log a warning if we encounter an error when releasing leases. While it
may not have direct consequences, failing to release the lease should be
unexpected, so let's make them visible.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 28d201f)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Also rename variables that collided with imports.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit aefbd49)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit e10eca3)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Fixes leases not being released when operation was cancelled.

Signed-off-by: Paweł Gronowski <[email protected]>
(cherry picked from commit 6f27bef)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
…efer_noncancellable_context

[24.0 backport] daemon/c8d: Use non cancellable context in defers
When the default bridge is disabled by setting dockerd's `--bridge=none`
option, the daemon still creates a sandbox for containers with no
network attachment specified. In that case `NetworkDisabled` will be set
to true.

However, currently the `releaseNetwork` call will early return if
NetworkDisabled is true. Thus, these sandboxes won't be deleted until
the daemon is restarted. If a high number of such containers are
created, the daemon would then take few minutes to start.

See moby#42461.

Signed-off-by: payall4u <[email protected]>
Signed-off-by: Albin Kerouanton <[email protected]>
(cherry picked from commit 9664f33)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
When the daemon process or the host running it is abruptly terminated,
the layer metadata file can become inconsistent on the file system.
Specifically, `link` and `lower` files may exist but be empty, leading
to overlay mounting errors during layer extraction, such as:
"failed to register layer: error creating overlay mount to <path>:
too many levels of symbolic links."

This commit introduces the use of `AtomicWriteFile` to ensure that the
layer metadata files contain correct data when they exist on the file system.

Signed-off-by: Mike <[email protected]>
(cherry picked from commit de2447c)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
full diff: klauspost/compress@v1.16.3...v1.16.5

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 0354791)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
fixes data corruption with zstd output in "best"

- 1.17.2 diff: klauspost/compress@v1.17.1...v1.17.2
- full diff: klauspost/compress@v1.16.5...v1.17.2

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit f2c67ea)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
…work_NetworkDisabled

[24.0 backport] daemon: release sandbox even when NetworkDisabled
…er-data-write

[24.0 backport] daemon: overlay2: Write layer metadata atomically
commit def549c passed through the context
to the daemon.ContainerStart function. As a result, restarting containers
no longer is an atomic operation, because a context cancellation could
interrupt the restart (between "stopping" and "(re)starting"), resulting
in the container being stopped, but not restarted.

Restarting a container, or more factually; making a successful request on
the `/containers/{id]/restart` endpoint, should be an atomic operation.

This patch uses a context.WithoutCancel for restart requests.

It's worth noting that daemon.containerStop already uses context.WithoutCancel,
so in that function, we'll be wrapping the context twice, but this should
likely not cause issues (just redundant for this code-path).

Before this patch, starting a container that bind-mounts the docker socket,
then restarting itself from within the container would cancel the restart
operation. The container would be stopped, but not started after that:

    docker run -dit --name myself -v /var/run/docker.sock:/var/run/docker.sock docker:cli sh
    docker exec myself sh -c 'docker restart myself'

    docker ps -a
    CONTAINER ID   IMAGE         COMMAND                  CREATED          STATUS                       PORTS     NAMES
    3a2a741c65ff   docker:cli    "docker-entrypoint.s…"   26 seconds ago   Exited (128) 7 seconds ago             myself

With this patch: the stop still cancels the exec, but does not cancel the
restart operation, and the container is started again:

    docker run -dit --name myself -v /var/run/docker.sock:/var/run/docker.sock docker:cli sh
    docker exec myself sh -c 'docker restart myself'
    docker ps
    CONTAINER ID   IMAGE        COMMAND                  CREATED              STATUS         PORTS     NAMES
    4393a01f7c75   docker:cli   "docker-entrypoint.s…"   About a minute ago   Up 4 seconds             myself

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit aeb8972)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
[24.0 backport] vendor: github.com/klauspost/compress v1.17.2
[24.0 backport] deny /sys/devices/virtual/powercap
…cancel

[24.0 backport] daemon: daemon.containerRestart: don't cancel restart on context cancel
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.