Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: disable artifacts cache with composefs #23274

Merged
merged 3 commits into from
Jul 24, 2024
Merged

test: disable artifacts cache with composefs #23274

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
90411b2
vendor: bump c/storage
giuseppe Jul 17, 2024
8403f4c
test: fix podman pull tests
giuseppe Jul 18, 2024
fef125c
test: disable artifacts cache with composefs
giuseppe Jul 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@ require (
github.com/containers/libhvee v0.7.1
github.com/containers/ocicrypt v1.2.0
github.com/containers/psgo v1.9.0
github.com/containers/storage v1.54.1-0.20240712125645-98ad80d6d165
github.com/containers/storage v1.54.1-0.20240724150347-86a0c425388b
github.com/containers/winquit v1.1.0
github.com/coreos/go-systemd/v22 v22.5.1-0.20231103132048-7d375ecc2b09
github.com/coreos/stream-metadata-go v0.4.4
github.com/crc-org/crc/v2 v2.38.0
github.com/crc-org/vfkit v0.5.1
github.com/cyphar/filepath-securejoin v0.3.0
github.com/cyphar/filepath-securejoin v0.3.1
github.com/digitalocean/go-qemu v0.0.0-20230711162256-2e3d0186973e
github.com/docker/distribution v2.8.3+incompatible
github.com/docker/docker v27.0.3+incompatible
Expand Down Expand Up @@ -87,7 +87,7 @@ require (
require (
dario.cat/mergo v1.0.0 // indirect
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 // indirect
github.com/Microsoft/hcsshim v0.12.4 // indirect
github.com/Microsoft/hcsshim v0.12.5 // indirect
github.com/VividCortex/ewma v1.2.0 // indirect
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d // indirect
github.com/aead/serpent v0.0.0-20160714141033-fba169763ea6 // indirect
Expand Down Expand Up @@ -170,7 +170,7 @@ require (
github.com/moby/buildkit v0.12.5 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/patternmatcher v0.6.0 // indirect
github.com/moby/sys/mountinfo v0.7.1 // indirect
github.com/moby/sys/mountinfo v0.7.2 // indirect
github.com/moby/sys/sequential v0.5.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
Expand Down
16 changes: 8 additions & 8 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ github.com/BurntSushi/toml v1.4.0 h1:kuoIxZQy2WRRk1pttg9asf+WVv6tWQuBNVmK8+nqPr0
github.com/BurntSushi/toml v1.4.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/Microsoft/hcsshim v0.12.4 h1:Ev7YUMHAHoWNm+aDSPzc5W9s6E2jyL1szpVDJeZ/Rr4=
github.com/Microsoft/hcsshim v0.12.4/go.mod h1:Iyl1WVpZzr+UkzjekHZbV8o5Z9ZkxNGx6CtY2Qg/JVQ=
github.com/Microsoft/hcsshim v0.12.5 h1:bpTInLlDy/nDRWFVcefDZZ1+U8tS+rz3MxjKgu9boo0=
github.com/Microsoft/hcsshim v0.12.5/go.mod h1:tIUGego4G1EN5Hb6KC90aDYiUI2dqLSTTOCjVNpOgZ8=
github.com/Netflix/go-expect v0.0.0-20220104043353-73e0943537d2 h1:+vx7roKuyA63nhn5WAunQHLTznkw5W8b1Xc0dNjp83s=
github.com/Netflix/go-expect v0.0.0-20220104043353-73e0943537d2/go.mod h1:HBCaDeC1lPdgDeDbhX8XFpy1jqjK0IBG8W5K+xYqA0w=
github.com/VividCortex/ewma v1.2.0 h1:f58SaIzcDXrSy3kWaHNvuJgJ3Nmz59Zji6XoJR/q1ow=
Expand Down Expand Up @@ -97,8 +97,8 @@ github.com/containers/ocicrypt v1.2.0 h1:X14EgRK3xNFvJEfI5O4Qn4T3E25ANudSOZz/sir
github.com/containers/ocicrypt v1.2.0/go.mod h1:ZNviigQajtdlxIZGibvblVuIFBKIuUI2M0QM12SD31U=
github.com/containers/psgo v1.9.0 h1:eJ74jzSaCHnWt26OlKZROSyUyRcGDf+gYBdXnxrMW4g=
github.com/containers/psgo v1.9.0/go.mod h1:0YoluUm43Mz2UnBIh1P+6V6NWcbpTL5uRtXyOcH0B5A=
github.com/containers/storage v1.54.1-0.20240712125645-98ad80d6d165 h1:9WbIxink8kCOoMNh9ju4CMdrrxwnbQMV1YJD8sUXt+k=
github.com/containers/storage v1.54.1-0.20240712125645-98ad80d6d165/go.mod h1:EyuSB0B1ddqXN0pXGNKPrtxzma80jhRCeVl7/J/JAhE=
github.com/containers/storage v1.54.1-0.20240724150347-86a0c425388b h1:xWuoWniyeweN/rNmTQJTaGSth5rKQ3V6EcQ4V3XVa/A=
github.com/containers/storage v1.54.1-0.20240724150347-86a0c425388b/go.mod h1:NbpcPTBSlREa7GfqzHqPaAD28CtBsZu+xsryPkOLnHg=
github.com/containers/winquit v1.1.0 h1:jArun04BNDQvt2W0Y78kh9TazN2EIEMG5Im6/JY7+pE=
github.com/containers/winquit v1.1.0/go.mod h1:PsPeZlnbkmGGIToMPHF1zhWjBUkd8aHjMOr/vFcPxw8=
github.com/coreos/go-oidc/v3 v3.10.0 h1:tDnXHnLyiTVyT/2zLDGj09pFPkhND8Gl8lnTRhoEaJU=
Expand All @@ -118,8 +118,8 @@ github.com/creack/pty v1.1.18 h1:n56/Zwd5o6whRC5PMGretI4IdRLlmBXYNjScPaBgsbY=
github.com/creack/pty v1.1.18/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=
github.com/cyberphone/json-canonicalization v0.0.0-20231217050601-ba74d44ecf5f h1:eHnXnuK47UlSTOQexbzxAZfekVz6i+LKRdj1CU5DPaM=
github.com/cyberphone/json-canonicalization v0.0.0-20231217050601-ba74d44ecf5f/go.mod h1:uzvlm1mxhHkdfqitSA92i7Se+S9ksOn3a3qmv/kyOCw=
github.com/cyphar/filepath-securejoin v0.3.0 h1:tXpmbiaeBrS/K2US8nhgwdKYnfAOnVfkcLPKFgFHeA0=
github.com/cyphar/filepath-securejoin v0.3.0/go.mod h1:F7i41x/9cBF7lzCrVsYs9fuzwRZm4NQsGTBdpp6mETc=
github.com/cyphar/filepath-securejoin v0.3.1 h1:1V7cHiaW+C+39wEfpH6XlLBQo3j/PciWFrgfCLS8XrE=
github.com/cyphar/filepath-securejoin v0.3.1/go.mod h1:F7i41x/9cBF7lzCrVsYs9fuzwRZm4NQsGTBdpp6mETc=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
Expand Down Expand Up @@ -363,8 +363,8 @@ github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3N
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/patternmatcher v0.6.0 h1:GmP9lR19aU5GqSSFko+5pRqHi+Ohk1O69aFiKkVGiPk=
github.com/moby/patternmatcher v0.6.0/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc=
github.com/moby/sys/mountinfo v0.7.1 h1:/tTvQaSJRr2FshkhXiIpux6fQ2Zvc4j7tAhMTStAG2g=
github.com/moby/sys/mountinfo v0.7.1/go.mod h1:IJb6JQeOklcdMU9F5xQ8ZALD+CUr5VlGpwtX+VE0rpI=
github.com/moby/sys/mountinfo v0.7.2 h1:1shs6aH5s4o5H2zQLn796ADW1wMrIwHsyJ2v9KouLrg=
github.com/moby/sys/mountinfo v0.7.2/go.mod h1:1YOa8w8Ih7uW0wALDUgT1dTTSBrZ+HiBLGws92L2RU4=
github.com/moby/sys/sequential v0.5.0 h1:OPvI35Lzn9K04PBbCLW0g4LcFAJgHsvXsRyewg5lXtc=
github.com/moby/sys/sequential v0.5.0/go.mod h1:tH2cOOs5V9MlPiXcQzRC+eEyab644PWKGRYaaV5ZZlo=
github.com/moby/sys/user v0.1.0 h1:WmZ93f5Ux6het5iituh9x2zAG7NFY9Aqi49jjE1PaQg=
Expand Down
47 changes: 33 additions & 14 deletions test/e2e/common_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -400,6 +400,30 @@ func imageTarPath(image string) string {
return filepath.Join(cacheDir, imageCacheName)
}

func (p *PodmanTestIntegration) pullImage(image string, toCache bool) {
if toCache {
oldRoot := p.Root
p.Root = p.ImageCacheDir
defer func() {
p.Root = oldRoot
}()
}
for try := 0; try < 3; try++ {
podmanSession := p.PodmanBase([]string{"pull", image}, toCache, true)
pull := PodmanSessionIntegration{podmanSession}
pull.Wait(440)
if pull.ExitCode() == 0 {
break
}
if try == 2 {
Expect(pull).Should(Exit(0), "Failed after many retries")
}

GinkgoWriter.Println("Will wait and retry")
time.Sleep(time.Duration(try+1) * 5 * time.Second)
}
}

// createArtifact creates a cached image tarball in a local directory
func (p *PodmanTestIntegration) createArtifact(image string) {
if os.Getenv("NO_TEST_CACHE") != "" {
Expand All @@ -408,19 +432,8 @@ func (p *PodmanTestIntegration) createArtifact(image string) {
destName := imageTarPath(image)
if _, err := os.Stat(destName); os.IsNotExist(err) {
GinkgoWriter.Printf("Caching %s at %s...\n", image, destName)
for try := 0; try < 3; try++ {
pull := p.PodmanNoCache([]string{"pull", image})
pull.Wait(440)
if pull.ExitCode() == 0 {
break
}
if try == 2 {
Expect(pull).Should(Exit(0), "Failed after many retries")
}

GinkgoWriter.Println("Will wait and retry")
time.Sleep(time.Duration(try+1) * 5 * time.Second)
}
p.pullImage(image, false)

save := p.PodmanNoCache([]string{"save", "-o", destName, image})
save.Wait(90)
Expand Down Expand Up @@ -1036,8 +1049,14 @@ func (p *PodmanTestIntegration) RestoreArtifactToCache(image string) error {

func populateCache(podman *PodmanTestIntegration) {
for _, image := range CACHE_IMAGES {
err := podman.RestoreArtifactToCache(image)
Expect(err).ToNot(HaveOccurred())
// FIXME: Remove this hack once composefs can be used with images
// pulled from sources other than a registry.
if strings.Contains(podman.StorageOptions, "overlay.use_composefs=true") {
podman.pullImage(image, true)
} else {
err := podman.RestoreArtifactToCache(image)
Expect(err).ToNot(HaveOccurred())
}
}
// logformatter uses this to recognize the first test
GinkgoWriter.Printf("-----------------------------\n")
Expand Down
32 changes: 18 additions & 14 deletions test/e2e/pull_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,37 +216,39 @@ var _ = Describe("Podman pull", func() {
})

It("podman pull by instance digest (image list)", func() {
SkipIfRemote("podman-remote does not support disabling external imagestore")

session := podmanTest.Podman([]string{"pull", "-q", "--arch=arm64", ALPINEARM64DIGEST})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())

// inspect using the digest of the list
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoTags}}", ALPINELISTDIGEST})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoTags}}", ALPINELISTDIGEST})
session.WaitWithDefaultTimeout()
Expect(session).To(ExitWithError(125, fmt.Sprintf(`no such object: "%s"`, ALPINELISTDIGEST)))
// inspect using the digest of the list
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINELISTDIGEST})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINELISTDIGEST})
session.WaitWithDefaultTimeout()
Expect(session).To(ExitWithError(125, fmt.Sprintf(`no such object: "%s"`, ALPINELISTDIGEST)))

// inspect using the digest of the arch-specific image's manifest
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoTags}}", ALPINEARM64DIGEST})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoTags}}", ALPINEARM64DIGEST})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())
Expect(string(session.Out.Contents())).To(HavePrefix("[]"))
// inspect using the digest of the arch-specific image's manifest
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINEARM64DIGEST})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINEARM64DIGEST})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())
Expect(string(session.Out.Contents())).To(Not(ContainSubstring(ALPINELISTDIGEST)))
Expect(string(session.Out.Contents())).To(ContainSubstring(ALPINEARM64DIGEST))
// inspect using the image ID
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoTags}}", ALPINEARM64ID})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoTags}}", ALPINEARM64ID})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())
Expect(string(session.Out.Contents())).To(HavePrefix("[]"))
// inspect using the image ID
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINEARM64ID})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINEARM64ID})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())
Expect(string(session.Out.Contents())).To(Not(ContainSubstring(ALPINELISTDIGEST)))
Expand All @@ -258,49 +260,51 @@ var _ = Describe("Podman pull", func() {
})

It("podman pull by tag (image list)", func() {
SkipIfRemote("podman-remote does not support disabling external imagestore")

session := podmanTest.Podman([]string{"pull", "-q", "--arch=arm64", ALPINELISTTAG})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())
// inspect using the tag we used for pulling
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoTags}}", ALPINELISTTAG})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoTags}}", ALPINELISTTAG})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())
Expect(string(session.Out.Contents())).To(ContainSubstring(ALPINELISTTAG))
// inspect using the tag we used for pulling
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINELISTTAG})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINELISTTAG})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())
Expect(string(session.Out.Contents())).To(ContainSubstring(ALPINELISTDIGEST))
Expect(string(session.Out.Contents())).To(ContainSubstring(ALPINEARM64DIGEST))
// inspect using the digest of the list
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoTags}}", ALPINELISTDIGEST})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoTags}}", ALPINELISTDIGEST})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())
Expect(string(session.Out.Contents())).To(ContainSubstring(ALPINELISTTAG))
// inspect using the digest of the list
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINELISTDIGEST})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINELISTDIGEST})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())
Expect(string(session.Out.Contents())).To(ContainSubstring(ALPINELISTDIGEST))
Expect(string(session.Out.Contents())).To(ContainSubstring(ALPINEARM64DIGEST))
// inspect using the digest of the arch-specific image's manifest
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoTags}}", ALPINEARM64DIGEST})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoTags}}", ALPINEARM64DIGEST})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())
Expect(string(session.Out.Contents())).To(ContainSubstring(ALPINELISTTAG))
// inspect using the digest of the arch-specific image's manifest
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINEARM64DIGEST})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINEARM64DIGEST})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())
Expect(string(session.Out.Contents())).To(ContainSubstring(ALPINELISTDIGEST))
Expect(string(session.Out.Contents())).To(ContainSubstring(ALPINEARM64DIGEST))
// inspect using the image ID
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoTags}}", ALPINEARM64ID})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoTags}}", ALPINEARM64ID})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())
Expect(string(session.Out.Contents())).To(ContainSubstring(ALPINELISTTAG))
// inspect using the image ID
session = podmanTest.Podman([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINEARM64ID})
session = podmanTest.PodmanNoCache([]string{"inspect", "--format", "{{.RepoDigests}}", ALPINEARM64ID})
session.WaitWithDefaultTimeout()
Expect(session).Should(ExitCleanly())
Expect(string(session.Out.Contents())).To(ContainSubstring(ALPINELISTDIGEST))
Expand Down
46 changes: 46 additions & 0 deletions vendor/github.com/Microsoft/hcsshim/hnsaccelnet.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

60 changes: 60 additions & 0 deletions vendor/github.com/Microsoft/hcsshim/internal/hns/hnsaccelnet.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

23 changes: 23 additions & 0 deletions vendor/github.com/Microsoft/hcsshim/internal/hns/hnsendpoint.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading