Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cryptic Error with Different User: 'ERROR: parsing time "{\x0a \"registry-1.dock" as "2006-01-02T15:04:05Z07:00": cannot parse "{\x0a \"registry-1.dock" as "2006"' #3066

Open
3 tasks done
JWCS opened this issue Mar 14, 2025 · 7 comments

Comments

@JWCS
Copy link

JWCS commented Mar 14, 2025

Contributing guidelines

I've found a bug and checked that ...

  • ... the documentation does not mention anything about my problem
  • ... there are no open or closed issues that are related to my problem

Description

I have docker installed via the recommended steps on website, for ubuntu. (If relevant, I've used docker for years, so I don't think there's an installation issue; see details below).
This is the "main" user I have:

> id
uid=1000(mainuser) gid=1000(mainuser) groups=1000(mainuser),4(adm),20(dialout),27(sudo),998(docker) # truncated

Now, for another process (simplified), there's another needed user, but its setup to be nearly identical to main user. It was created using adduser, with the same uid and gid and groups as main user; you can't even tell the difference (this was called by "otheruser").

> id
uid=1000(azureuser) gid=1000(azureuser) groups=1000(azureuser),27(sudo),998(docker) # no truncation

Logged into both users, pulling works just fine, and status commands give the same, so I don't think it's the cli. But when I try buildx build (or buildx ls), that's where the cryptic error comes. (Note: it used to work... then something changed, and I can't figure out what).

> sudo su otheruser
> BUILDX_GIT_INFO=false docker buildx build -t wip -f Dockerfile .
ERROR: parsing time "{\x0a  \"registry-1.dock" as "2006-01-02T15:04:05Z07:00": cannot parse "{\x0a  \"registry-1.dock" as "2006"

I feel like I must've caused it somehow... but I have no clue how or why, and I can't seem to find any relevant issues.

Expected behaviour

> whoami
mainuser
> BUILDX_GIT_INFO=false docker buildx build -t wip -f Dockerfile .
# success

Actual behaviour

> sudo su otheruser
> BUILDX_GIT_INFO=false docker buildx build -t wip -f Dockerfile .
ERROR: parsing time "{\x0a  \"registry-1.dock" as "2006-01-02T15:04:05Z07:00": cannot parse "{\x0a  \"registry-1.dock" as "2006"

Buildx version

(Both users): github.com/docker/buildx v0.21.1 7c2359c

Docker info

docker version
docker version
Client: Docker Engine - Community
 Version:           28.0.1
 API version:       1.48
 Go version:        go1.23.6
 Git commit:        068a01e
 Built:             Wed Feb 26 10:41:09 2025
 OS/Arch:           linux/arm64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          28.0.1
  API version:      1.48 (minimum version 1.24)
  Go version:       go1.23.6
  Git commit:       bbd0a17
  Built:            Wed Feb 26 10:41:09 2025
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.25
  GitCommit:        bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc:
  Version:          1.2.4
  GitCommit:        v1.2.4-0-g6c52b3f
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
docker info
docker info
Client: Docker Engine - Community
 Version:    28.0.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.21.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.33.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 7
 Server Version: 28.0.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: false
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc version: v1.2.4-0-g6c52b3f
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
 Kernel Version: 5.15.0-1081-azure
 Operating System: Ubuntu 20.04.6 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 2
 Total Memory: 963MiB
 Name: azeuss01019
 ID: c583da7a-512d-40ef-8fef-91d450c602ff
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

Builders list

mainuser:
 bash
> docker buildx ls
NAME/NODE     DRIVER/ENDPOINT   STATUS    BUILDKIT   PLATFORMS
default*      docker
 \_ default    \_ default       running   v0.20.0    linux/arm64, linux/arm (+2)

otheruser:
 bash
ERROR: parsing time "{\x0a  \"registry-1.dock" as "2006-01-02T15:04:05Z07:00": cannot parse "{\x0a  \"registry-1.dock" as "2006"

Configuration

cat .dockerignore

*

cat Dockerfile

# syntax = docker/dockerfile:1.4.0
FROM python:3.10-alpine3.21

The build directory is on an xfs mount, but the docker data is at the original /var/something/docker, and there's no daemon file configured.

# sudo su otheruser
BUILDX_GIT_INFO=false docker buildx build -t wip -f Dockerfile .

(I've tried with/without --load; can't think of any relevant other flags. --progress is useless, doesn't get that far)

Build logs

The successful build is completely normal. The failed build doesn't even start, but immediately quits.

Additional info

I've checked that the filesystem is OK / uncorrupted, but I can't think of anything else.

@crazy-max
Copy link
Member

I didn't manage to repro with your Dockerfile:

$ BUILDX_GIT_INFO=false docker buildx build -t wip -f Dockerfile .
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 100B done
#1 DONE 0.1s

#2 resolve image config for docker-image://docker.io/docker/dockerfile:1.4.0
#2 ...

#3 [auth] docker/dockerfile:pull token for registry-1.docker.io
#3 DONE 0.0s

#2 resolve image config for docker-image://docker.io/docker/dockerfile:1.4.0
#2 DONE 1.3s

#4 docker-image://docker.io/docker/dockerfile:1.4.0@sha256:178c4e4a93795b9365dbf6cf10da8fcf517fcb4a17f1943a775c0f548e9fc2ff
#4 resolve docker.io/docker/dockerfile:1.4.0@sha256:178c4e4a93795b9365dbf6cf10da8fcf517fcb4a17f1943a775c0f548e9fc2ff 0.0s done
#4 sha256:178c4e4a93795b9365dbf6cf10da8fcf517fcb4a17f1943a775c0f548e9fc2ff 2.00kB / 2.00kB done
#4 sha256:0fd968d6b87118e5c321970e04567ad319a583d3ff7ea88fb57015f14fbeeaa7 528B / 528B done
#4 sha256:c45593c04fe618ad08c8f3e6cdd0d6d42c7e1dd8e103adb6c88bed6fe6c5d2c6 2.37kB / 2.37kB done
#4 sha256:96cc9c4d9b06a8798ac4c21bd88d4a32f3737594d10cd176ac4f2549bea93c71 0B / 9.93MB 0.1s
#4 sha256:96cc9c4d9b06a8798ac4c21bd88d4a32f3737594d10cd176ac4f2549bea93c71 5.24MB / 9.93MB 0.3s
#4 sha256:96cc9c4d9b06a8798ac4c21bd88d4a32f3737594d10cd176ac4f2549bea93c71 8.39MB / 9.93MB 0.4s
#4 sha256:96cc9c4d9b06a8798ac4c21bd88d4a32f3737594d10cd176ac4f2549bea93c71 9.93MB / 9.93MB 0.4s done
#4 extracting sha256:96cc9c4d9b06a8798ac4c21bd88d4a32f3737594d10cd176ac4f2549bea93c71 0.1s done
#4 DONE 0.6s

#5 [internal] load .dockerignore
#5 transferring context: 42B done
#5 DONE 0.1s

#6 [internal] load metadata for docker.io/library/python:3.10-alpine3.21
#6 ...

#7 [auth] library/python:pull token for registry-1.docker.io
#7 DONE 0.0s

#6 [internal] load metadata for docker.io/library/python:3.10-alpine3.21
#6 DONE 1.0s

#8 [1/1] FROM docker.io/library/python:3.10-alpine3.21@sha256:4c4097e46608e9b9025a486e3b72d628ac8947a3caa20f142c2e8c0029a12b21
#8 resolve docker.io/library/python:3.10-alpine3.21@sha256:4c4097e46608e9b9025a486e3b72d628ac8947a3caa20f142c2e8c0029a12b21 0.0s done
#8 sha256:1271aa09ec5214b342e2ec3ba46ea8650122a9fe38417ff8ace806ebc5d80bc2 1.74kB / 1.74kB done
#8 sha256:2a80925da4cee73f146fcad502168a78f3d8871c2cf2f23a3de9280a4c4a3baf 5.09kB / 5.09kB done
#8 sha256:3588bd7cf5ec4d8e71913c7fe40450d447a9253f82bc9ad5100305085feaddc5 0B / 458.62kB 0.1s
#8 sha256:3d2e6691f0b27c1c33f84ca62d06edf41cdb2c335f1ede2e87d9c01db1f44fb6 0B / 15.64MB 0.1s
#8 sha256:e5c20ecf2705f9d84b27fbbda5bd9542a596fb5f2d2941ae5310896b0e7e9e08 0B / 249B 0.1s
#8 sha256:4c4097e46608e9b9025a486e3b72d628ac8947a3caa20f142c2e8c0029a12b21 10.30kB / 10.30kB done
#8 sha256:3588bd7cf5ec4d8e71913c7fe40450d447a9253f82bc9ad5100305085feaddc5 458.62kB / 458.62kB 0.2s done
#8 extracting sha256:3588bd7cf5ec4d8e71913c7fe40450d447a9253f82bc9ad5100305085feaddc5 0.1s
#8 sha256:3d2e6691f0b27c1c33f84ca62d06edf41cdb2c335f1ede2e87d9c01db1f44fb6 4.19MB / 15.64MB 0.4s
#8 sha256:e5c20ecf2705f9d84b27fbbda5bd9542a596fb5f2d2941ae5310896b0e7e9e08 249B / 249B 0.4s done
#8 extracting sha256:3588bd7cf5ec4d8e71913c7fe40450d447a9253f82bc9ad5100305085feaddc5 0.1s done
#8 sha256:3d2e6691f0b27c1c33f84ca62d06edf41cdb2c335f1ede2e87d9c01db1f44fb6 15.64MB / 15.64MB 0.6s
#8 sha256:3d2e6691f0b27c1c33f84ca62d06edf41cdb2c335f1ede2e87d9c01db1f44fb6 15.64MB / 15.64MB 0.6s done
#8 extracting sha256:3d2e6691f0b27c1c33f84ca62d06edf41cdb2c335f1ede2e87d9c01db1f44fb6 0.1s
#8 extracting sha256:3d2e6691f0b27c1c33f84ca62d06edf41cdb2c335f1ede2e87d9c01db1f44fb6 0.6s done
#8 extracting sha256:e5c20ecf2705f9d84b27fbbda5bd9542a596fb5f2d2941ae5310896b0e7e9e08
#8 extracting sha256:e5c20ecf2705f9d84b27fbbda5bd9542a596fb5f2d2941ae5310896b0e7e9e08 done
#8 DONE 1.5s

#9 exporting to image
#9 exporting layers done
#9 writing image sha256:57835aa74c130090327dd9d19570516642dbc73c396894ba618645e90c4e4fe9 done
#9 naming to docker.io/library/wip done
#9 DONE 0.0s

Looking at the error when running with otheruser user this might be an issue in docker config?
Can you check the content of ~/.docker/config.json for otheruser?

@JWCS
Copy link
Author

JWCS commented Mar 18, 2025

Yeah, I can't normally produce it either, except in this odd circumstance. I was hoping there was some sort of hidden logs or config you could point me to.
I never configure the ~/.docker folder, at most /etc/docker/daemon.json (which isn't). There is no ~/.docker/config.json file.

@crazy-max
Copy link
Member

Can you try with debug enabled?

BUILDX_GIT_INFO=false docker --debug buildx build -t wip -f Dockerfile .

@JWCS
Copy link
Author

JWCS commented Mar 19, 2025

Exact same output; which is why I'm stumped for debugging.

$ BUILDX_GIT_INFO=false docker --debug buildx build -t wip -f Dockerfile .
ERROR: parsing time "{\x0a  \"registry-1.dock" as "2006-01-02T15:04:05Z07:00": cannot parse "{\x0a  \"registry-1.dock" as "2006"

@thaJeztah
Copy link
Member

I wonder if the error is some code trying to parse a (JWT) access-token, and trying to get the expiration date from it. The { being the opening brace of the JSON, {\x0a is a newline, and registry-1.dock is the first part of the registry for which the token is;

For example, this is what i have stored as a token;

{"ServerURL":"https://index.docker.io/v1/","Username":"svstijn","Secret":"<REDACTED>"}

There output from the error could be if it was printed (only showing the field values?)

The "2006-01-02T15:04:05Z07:00" is the format used for RFC3339;
https://pkg.go.dev/time#RFC3339

And while searching if I found something related to this, I found various examples trying to get the Expires time from tokens, for example; https://github.com/Versent/saml2aws/blob/36983fbdc0e4fbfa59713e5a1206556ab0919c40/cmd/saml2aws/commands/script.go#L15-L45

@thaJeztah
Copy link
Member

thaJeztah commented Mar 19, 2025

Did a quick try with some naive code trying to do so, and what it could look like; https://go.dev/play/p/-tOY8WNVFQj

package main

import (
	"encoding/json"
	"fmt"
	"time"
)

func main() {
	const resp = `{"ServerURL":"https://index.docker.io/v1/","Username":"my-username","Secret":"REDACTED"}`

	type Token struct {
		ServerURL string
		Username  string
		Secret    string
	}

	var out2 Token
	_ = json.Unmarshal([]byte(resp), &out2)

	jsonString := fmt.Sprintf("%v", out2)
	fmt.Println(jsonString)

	_, err := time.Parse(time.RFC3339, jsonString)
	if err != nil {
		fmt.Println("ERROR:", err)
	}

}

That produces;

{https://index.docker.io/v1/ my-username REDACTED}
ERROR: parsing time "{https://index.docker.io/v1/ my-username REDACTED}" as "2006-01-02T15:04:05Z07:00": cannot parse "{https://index.docker.io/v1/ my-username REDACTED}" as "2006"

Also wondering if it could be something in the registry (or in between); there's some partying happening in here; https://github.com/distribution/distribution/blob/98dd92206f26f9d57535f79cb214c86ec07978b3/registry/storage/linkedblobstore.go#L175-L196

@thaJeztah
Copy link
Member

@crazy-max if it's buildx; could it be something like this code? (not sure how a JSON would end up in that file though);

buildx/store/store.go

Lines 160 to 173 in 18ccba0

func (t *Txn) UpdateLastActivity(ng *NodeGroup) error {
return t.s.cfg.AtomicWriteFile(filepath.Join(activityDir, ng.Name), []byte(time.Now().UTC().Format(time.RFC3339)), 0600)
}
func (t *Txn) GetLastActivity(ng *NodeGroup) (la time.Time, _ error) {
dt, err := os.ReadFile(filepath.Join(t.s.cfg.Dir(), activityDir, ng.Name))
if err != nil {
if os.IsNotExist(errors.Cause(err)) {
return la, nil
}
return la, err
}
return time.Parse(time.RFC3339, string(dt))
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants