DeferCleanup() or AfterAll() is not triggered if specs in nested container node is skipped

[psturc]

Hello! I know it is probably not a recommended practice to write tests with so many nested container nodes (and I'm currently rewriting specs in my test to be independent), but I thought it's worth to report this issue anyway.

This is the code

var _ = Describe("Describe level 1", Ordered, func() {
	BeforeAll(func() {
		log.Println("Executing BeforeAll, level 1")
		DeferCleanup(log.Println, "Executing DeferCleanup, level 1")
	})
	It("It1 level 1", func() {
		log.Println("Executing It1 level 1")
		Expect(1).To(Equal(1))
	})
	Describe("Describe level 2", func() {
		BeforeAll(func() {
			if os.Getenv("SKIP") == "true" {
				Skip("Skipping Describe level 2")
			}
			log.Println("Executing BeforeAll, level 2")
			DeferCleanup(log.Println, "Executing DeferCleanup, level 2")
		})
		It("It1 level 2", func() {
			log.Println("Executing It1 level 2")
			Expect(1).To(Equal(1))
		})
		// Describe("Describe level 3", func() {
		// 	BeforeAll(func() {
		// 		log.Println("Executing BeforeAll, level 3")
		// 		DeferCleanup(log.Println, "Executing DeferCleanup, level 3")
		// 	})
		// 	It("It1 level 3", func() {
		// 		log.Println("Executing It1 level 3")
		// 		Expect(1).To(Equal(1))
		// 	})
		// })
	})
})

When I run the tests with env var SKIP=true (which results in all specs in container node Describe level 2 to be skipped), the DeferCleanup, level 1 is called, which is expected (see the log 2022/05/18 11:06:30 Executing DeferCleanup, level 1)

➜  ginkgo SKIP=true ginkgo 
Running Suite: Ginkgo Suite - /Users/psturc/learn/go/ginkgo
===========================================================
Random Seed: 1652864789

Will run 2 of 2 specs
•2022/05/18 11:06:30 Executing BeforeAll, level 1
2022/05/18 11:06:30 Executing It1 level 1

------------------------------
S [SKIPPED] [0.000 seconds]
Describe level 1
/Users/psturc/learn/go/ginkgo/main_test.go:11
  Describe level 2 [BeforeAll]
  /Users/psturc/learn/go/ginkgo/main_test.go:21
    It1 level 2
    /Users/psturc/learn/go/ginkgo/main_test.go:28

  Skipping Describe level 2
  In [BeforeAll] at: /Users/psturc/learn/go/ginkgo/main_test.go:23
------------------------------
2022/05/18 11:06:30 Executing DeferCleanup, level 1

Ran 1 of 2 Specs in 0.001 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 1 Skipped
PASS

Ginkgo ran 1 suite in 1.190130184s
Test Suite Passed

However if I uncomment the code for Describe level 3, and run the test with the same command, the cleanup (DeferCleanup, level 1) doesn't happen.

➜  ginkgo SKIP=true ginkgo
Running Suite: Ginkgo Suite - /Users/psturc/learn/go/ginkgo
===========================================================
Random Seed: 1652865119

Will run 3 of 3 specs
•2022/05/18 11:12:01 Executing BeforeAll, level 1
2022/05/18 11:12:01 Executing It1 level 1

------------------------------
S [SKIPPED] [0.000 seconds]
Describe level 1
/Users/psturc/learn/go/ginkgo/main_test.go:11
  Describe level 2 [BeforeAll]
  /Users/psturc/learn/go/ginkgo/main_test.go:21
    It1 level 2
    /Users/psturc/learn/go/ginkgo/main_test.go:28

  Skipping Describe level 2
  In [BeforeAll] at: /Users/psturc/learn/go/ginkgo/main_test.go:23
------------------------------
S [SKIPPED] [0.000 seconds]
Describe level 1
/Users/psturc/learn/go/ginkgo/main_test.go:11
  Describe level 2
  /Users/psturc/learn/go/ginkgo/main_test.go:20
    Describe level 3
    /Users/psturc/learn/go/ginkgo/main_test.go:32
      [It] It1 level 3
      /Users/psturc/learn/go/ginkgo/main_test.go:37

  Spec skipped because Skip() was called in BeforeAll
  In [It] at: /Users/psturc/learn/go/ginkgo/main_test.go:37
------------------------------

Ran 1 of 3 Specs in 0.001 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 2 Skipped
PASS

Ginkgo ran 1 suite in 1.892848673s
Test Suite Passed

It happens only in case the specs in second container node (Describe level 2) are skipped. (If I run the tests with SKIP=false ginkgo, the cleanup works as expected).

[onsi]

hey @psturc thanks for reporting this. Looks like it might be an edge-case bug that you've found. I'll dig into it and see how if I can push a fix out soon.

[psturc]

Hi @onsi did you get a chance to look into this (no pressure, just curious :)?

[onsi]

hey sorry, no. i've been super behind!