Skip to content
This repository has been archived by the owner on Nov 30, 2024. It is now read-only.

Commit

Permalink
Omit redandunt execution with rspec --bisect
Browse files Browse the repository at this point in the history 
Since bisect is a command that finds minimum repro that fails depending on the order,
      examples after the last failure example doesn't need to be executed.
  • Loading branch information
@makicamel
makicamel committed Jul 10, 2024
1 parent 4be46f6 commit e2e1a1f
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 38 deletions.
5 changes: 4 additions & 1 deletion lib/rspec/core/bisect/example_minimizer.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,10 @@ def prep
end

def non_failing_example_ids
@non_failing_example_ids ||= all_example_ids - failed_example_ids
return @non_failing_example_ids if defined?(@non_failing_example_ids)

last_failed_example_id_index = all_example_ids.rindex(failed_example_ids.last)
@non_failing_example_ids = all_example_ids[0..last_failed_example_id_index] - failed_example_ids
end

def get_expected_failures_for?(ids)
Expand Down
53 changes: 16 additions & 37 deletions spec/rspec/core/bisect/coordinator_spec.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,12 @@ def find_minimal_repro(output, formatter=Formatters::BisectProgressFormatter, bi
expect(output).to eq(<<-EOS.gsub(/^\s+\|/, ''))
|Bisect started using options: ""
|Running suite to find failures... (n.nnnn seconds)
|Starting bisect with 2 failing examples and 6 non-failing examples.
|Starting bisect with 2 failing examples and 3 non-failing examples.
|Checking that failure(s) are order-dependent... failure appears to be order-dependent
|
|Round 1: bisecting over non-failing examples 1-6 .. ignoring examples 4-6 (n.nnnn seconds)
|Round 2: bisecting over non-failing examples 1-3 .. multiple culprits detected - splitting candidates (n.nnnn seconds)
|Round 3: bisecting over non-failing examples 1-2 .. ignoring example 2 (n.nnnn seconds)
|Bisect complete! Reduced necessary non-failing examples from 6 to 2 in n.nnnn seconds.
|Round 1: bisecting over non-failing examples 1-3 .. multiple culprits detected - splitting candidates (n.nnnn seconds)
|Round 2: bisecting over non-failing examples 1-2 .. ignoring example 2 (n.nnnn seconds)
|Bisect complete! Reduced necessary non-failing examples from 3 to 2 in n.nnnn seconds.
|
|The minimal reproduction command is:
| rspec 1.rb[1:1] 2.rb[1:1] 4.rb[1:1] 5.rb[1:1]
Expand All @@ -54,40 +53,26 @@ def find_minimal_repro(output, formatter=Formatters::BisectProgressFormatter, bi
| - Failing examples (2):
| - 2.rb[1:1]
| - 5.rb[1:1]
| - Non-failing examples (6):
| - Non-failing examples (3):
| - 1.rb[1:1]
| - 3.rb[1:1]
| - 4.rb[1:1]
| - 6.rb[1:1]
| - 7.rb[1:1]
| - 8.rb[1:1]
|Checking that failure(s) are order-dependent..
| - Running: rspec 2.rb[1:1] 5.rb[1:1] (n.nnnn seconds)
| - Failure appears to be order-dependent
|Round 1: bisecting over non-failing examples 1-6
| - Running: rspec 2.rb[1:1] 5.rb[1:1] 6.rb[1:1] 7.rb[1:1] 8.rb[1:1] (n.nnnn seconds)
| - Running: rspec 1.rb[1:1] 2.rb[1:1] 3.rb[1:1] 4.rb[1:1] 5.rb[1:1] (n.nnnn seconds)
| - Examples we can safely ignore (3):
| - 6.rb[1:1]
| - 7.rb[1:1]
| - 8.rb[1:1]
| - Remaining non-failing examples (3):
| - 1.rb[1:1]
| - 3.rb[1:1]
| - 4.rb[1:1]
|Round 2: bisecting over non-failing examples 1-3
|Round 1: bisecting over non-failing examples 1-3
| - Running: rspec 2.rb[1:1] 4.rb[1:1] 5.rb[1:1] (n.nnnn seconds)
| - Running: rspec 1.rb[1:1] 2.rb[1:1] 3.rb[1:1] 5.rb[1:1] (n.nnnn seconds)
| - Multiple culprits detected - splitting candidates
|Round 3: bisecting over non-failing examples 1-2
|Round 2: bisecting over non-failing examples 1-2
| - Running: rspec 2.rb[1:1] 3.rb[1:1] 4.rb[1:1] 5.rb[1:1] (n.nnnn seconds)
| - Running: rspec 1.rb[1:1] 2.rb[1:1] 4.rb[1:1] 5.rb[1:1] (n.nnnn seconds)
| - Examples we can safely ignore (1):
| - 3.rb[1:1]
| - Remaining non-failing examples (2):
| - 1.rb[1:1]
| - 4.rb[1:1]
|Bisect complete! Reduced necessary non-failing examples from 6 to 2 in n.nnnn seconds.
|Bisect complete! Reduced necessary non-failing examples from 3 to 2 in n.nnnn seconds.
|
|The minimal reproduction command is:
| rspec 1.rb[1:1] 2.rb[1:1] 4.rb[1:1] 5.rb[1:1]
Expand All @@ -105,10 +90,10 @@ def find_minimal_repro(output, formatter=Formatters::BisectProgressFormatter, bi
expect(output).to eq(<<-EOS.gsub(/^\s+\|/, ''))
|Bisect started using options: ""
|Running suite to find failures... (n.nnnn seconds)
|Starting bisect with 1 failing example and 7 non-failing examples.
|Starting bisect with 1 failing example and 1 non-failing example.
|Checking that failure(s) are order-dependent... failure(s) do not require any non-failures to run first
|
|Bisect complete! Reduced necessary non-failing examples from 7 to 0 in n.nnnn seconds.
|Bisect complete! Reduced necessary non-failing examples from 1 to 0 in n.nnnn seconds.
|
|The minimal reproduction command is:
| rspec 2.rb[1:1]
Expand All @@ -125,7 +110,7 @@ def find_minimal_repro(output, formatter=Formatters::BisectProgressFormatter, bi
expect(output).to eq(<<-EOS.gsub(/^\s+\|/, ''))
|Bisect started using options: ""
|Running suite to find failures... (n.nnnn seconds)
|Starting bisect with 1 failing example and 7 non-failing examples.
|Starting bisect with 1 failing example and 1 non-failing example.
|Checking that failure(s) are order-dependent... failure(s) do not require any non-failures to run first
|
|================================================================================
Expand All @@ -135,7 +120,7 @@ def find_minimal_repro(output, formatter=Formatters::BisectProgressFormatter, bi
|is unexpected, consider setting `config.bisect_runner = :shell` and trying again.
|================================================================================
|
|Bisect complete! Reduced necessary non-failing examples from 7 to 0 in n.nnnn seconds.
|Bisect complete! Reduced necessary non-failing examples from 1 to 0 in n.nnnn seconds.
|
|The minimal reproduction command is:
| rspec 2.rb[1:1]
Expand All @@ -154,18 +139,12 @@ def find_minimal_repro(output, formatter=Formatters::BisectProgressFormatter, bi
|Running suite to find failures... (n.nnnn seconds)
| - Failing examples (1):
| - 2.rb[1:1]
| - Non-failing examples (7):
| - Non-failing examples (1):
| - 1.rb[1:1]
| - 3.rb[1:1]
| - 4.rb[1:1]
| - 5.rb[1:1]
| - 6.rb[1:1]
| - 7.rb[1:1]
| - 8.rb[1:1]
|Checking that failure(s) are order-dependent..
| - Running: rspec 2.rb[1:1] (n.nnnn seconds)
| - Failure is not order-dependent
|Bisect complete! Reduced necessary non-failing examples from 7 to 0 in n.nnnn seconds.
|Bisect complete! Reduced necessary non-failing examples from 1 to 0 in n.nnnn seconds.
|
|The minimal reproduction command is:
| rspec 2.rb[1:1]
Expand Down Expand Up @@ -207,10 +186,10 @@ def bisect_round_started(notification)
expect(output).to eq(<<-EOS.gsub(/^\s+\|/, ''))
|Bisect started using options: ""
|Running suite to find failures... (n.nnnn seconds)
|Starting bisect with 2 failing examples and 6 non-failing examples.
|Starting bisect with 2 failing examples and 3 non-failing examples.
|Checking that failure(s) are order-dependent... failure appears to be order-dependent
|
|Round 1: bisecting over non-failing examples 1-6 .. ignoring examples 4-6 (n.nnnn seconds)
|Round 1: bisecting over non-failing examples 1-3 .. multiple culprits detected - splitting candidates (n.nnnn seconds)
|
|Bisect aborted!
|
Expand Down

0 comments on commit e2e1a1f

Please sign in to comment.