Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: add example feature files #7

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

tests: add example feature files #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 77 additions & 0 deletions tests/log-messages.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Tests for the expected workflow log messages

Feature: Log messages

As a researcher,
I want to be able to see the log messages of my workflow execution,
So that I can verify that the workflow ran correctly.

Scenario: The compiling step has produced the expected messages
When the workflow is finished
Then the engine logs should contain
"""
Publishing step:0, cmd: g++ -g -O3 -Wall -Wextra -Wpedantic -o skim skim.cxx `root-config --cflags --libs`, total steps 4 to MQ
"""

Scenario: The skimming step has produced the expected messages
When the workflow is finished
Then the engine logs should contain "Publishing step:1, cmd: ./skim"
And the job logs for the "skimming" step should contain
"""
>>> Process final state FourElectrons for sample Run2012B_DoubleElectron:
At least our electrons: pass=258449 all=21474287 -- eff=1.20 % cumulative eff=1.20 %
Require good isolation: pass=113 all=258449 -- eff=0.04 % cumulative eff=0.00 %
Good Electron kinematics: pass=67 all=113 -- eff=59.29 % cumulative eff=0.00 %
"""

Scenario: The histogramming step has produced the expected messages
When the workflow is finished
Then the engine logs should contain "Publishing step:2, cmd: python histograms.py"
And the job logs for the "histogramming" step should contain
"""
>>> Process skimmed sample SMHiggsToZZTo4L and final state FourMuons
>>> Process skimmed sample SMHiggsToZZTo4L and final state FourElectrons
>>> Process skimmed sample SMHiggsToZZTo4L and final state TwoMuonsTwoElectrons
>>> Process skimmed sample ZZTo4mu and final state FourMuons
>>> Process skimmed sample ZZTo4e and final state FourElectrons
>>> Process skimmed sample ZZTo2e2mu and final state TwoMuonsTwoElectrons
>>> Process skimmed sample Run2012B_DoubleMuParked and final state FourMuons
>>> Process skimmed sample Run2012B_DoubleMuParked and final state TwoMuonsTwoElectrons
>>> Process skimmed sample Run2012C_DoubleMuParked and final state FourMuons
>>> Process skimmed sample Run2012C_DoubleMuParked and final state TwoMuonsTwoElectrons
>>> Process skimmed sample Run2012B_DoubleElectron and final state FourElectrons
>>> Process skimmed sample Run2012B_DoubleElectron and final state TwoMuonsTwoElectrons
>>> Process skimmed sample Run2012C_DoubleElectron and final state FourElectrons
>>> Process skimmed sample Run2012C_DoubleElectron and final state TwoMuonsTwoElectrons
"""

Scenario: The plotting step has produced the expected messages
When the workflow is finished
Then the engine logs should contain "Publishing step:3, cmd: python plot.py"
And the job logs for the "plotting" step should contain
"""
Info in <TCanvas::Print>: pdf file FourMuons_Higgs_mass.pdf has been created
Info in <TCanvas::Print>: png file FourMuons_Higgs_mass.png has been created
Info in <TCanvas::Print>: pdf file FourElectrons_Higgs_mass.pdf has been created
Info in <TCanvas::Print>: png file FourElectrons_Higgs_mass.png has been created
Info in <TCanvas::Print>: pdf file TwoMuonsTwoElectrons_Higgs_mass.pdf has been created
Info in <TCanvas::Print>: png file TwoMuonsTwoElectrons_Higgs_mass.png has been created
Info in <TCanvas::Print>: pdf file combined_Higgs_mass.pdf has been created
Info in <TCanvas::Print>: png file combined_Higgs_mass.png has been created
Info in <TCanvas::Print>: pdf file FourMuons_Z1_mass.pdf has been created
Info in <TCanvas::Print>: png file FourMuons_Z1_mass.png has been created
Info in <TCanvas::Print>: pdf file FourElectrons_Z1_mass.pdf has been created
Info in <TCanvas::Print>: png file FourElectrons_Z1_mass.png has been created
Info in <TCanvas::Print>: pdf file TwoMuonsTwoElectrons_Z1_mass.pdf has been created
Info in <TCanvas::Print>: png file TwoMuonsTwoElectrons_Z1_mass.png has been created
Info in <TCanvas::Print>: pdf file combined_Z1_mass.pdf has been created
Info in <TCanvas::Print>: png file combined_Z1_mass.png has been created
Info in <TCanvas::Print>: pdf file FourMuons_Z2_mass.pdf has been created
Info in <TCanvas::Print>: png file FourMuons_Z2_mass.png has been created
Info in <TCanvas::Print>: pdf file FourElectrons_Z2_mass.pdf has been created
Info in <TCanvas::Print>: png file FourElectrons_Z2_mass.png has been created
Info in <TCanvas::Print>: pdf file TwoMuonsTwoElectrons_Z2_mass.pdf has been created
Info in <TCanvas::Print>: png file TwoMuonsTwoElectrons_Z2_mass.png has been created
Info in <TCanvas::Print>: pdf file combined_Z2_mass.pdf has been created
Info in <TCanvas::Print>: png file combined_Z2_mass.png has been created
"""
27 changes: 27 additions & 0 deletions tests/run-duration.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Tests for the workflow execution duration time

Feature: Run duration

As a researcher,
I want to verify that my workflow finishes in a reasonable amount of time,
so that I can stay assured that there are no unusual problems with computing resources.

Scenario: The workflow terminates in a reasonable amount of time
When the workflow execution completes
Then the workflow run duration should be less than 25 minutes

Scenario: The compiling step terminates in a reasonable amount of time
When the workflow execution completes
Then the duration of the step "compiling" should be less than 5 minutes

Scenario: Skimming the data takes a reasonable amount of time
When the workflow execution completes
Then the duration of the step "skimming" should be less than 15 minutes

Scenario: The histograms are created in a reasonable amount of time
When the workflow execution completes
Then the duration of the step "histogramming" should be less than 15 minutes

Scenario: Combining the histograms into plot takes a reasonable amount of time
When the workflow execution completes
Then the duration of the step "plotting" should be less than 5 minutes
73 changes: 73 additions & 0 deletions tests/workspace-files.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Tests for the presence of files in the workspace

Feature: Workspace files

As a researcher,
I want to make sure that my workflow produces the expected files,
and that the relevant input files are present in the workspace,
so that I can be sure that the workflow outputs are correct.

Scenario: The workspace contains all the relevant input files
When the workflow execution completes
Then the workspace should contain "skim.cxx"
And the workspace should include "histograms.py"
And the workspace should include "plot.py"

Rule: The output files and reports are generated correctly

Scenario: The expected plots are present in the workspace
When the workflow is finished
Then the workspace should include "combined_Higgs_mass.pdf"
And the workspace should include "combined_Higgs_mass.png"
And the workspace should include "combined_Z1_mass.pdf"
And the workspace should include "combined_Z1_mass.png"
And the workspace should include "combined_Z2_mass.pdf"
And the workspace should include "combined_Z2_mass.png"
And the workspace should include "FourElectrons_Higgs_mass.pdf"
And the workspace should include "FourElectrons_Higgs_mass.png"
And the workspace should include "FourElectrons_Z1_mass.pdf"
And the workspace should include "FourElectrons_Z1_mass.png"
And the workspace should include "FourElectrons_Z2_mass.pdf"
And the workspace should include "FourElectrons_Z2_mass.png"
And the workspace should include "FourMuons_Higgs_mass.pdf"
And the workspace should include "FourMuons_Higgs_mass.png"
And the workspace should include "FourMuons_Z1_mass.pdf"
And the workspace should include "FourMuons_Z1_mass.png"
And the workspace should include "FourMuons_Z2_mass.pdf"
And the workspace should include "FourMuons_Z2_mass.png"
And the workspace should include "TwoMuonsTwoElectrons_Higgs_mass.pdf"
And the workspace should include "TwoMuonsTwoElectrons_Higgs_mass.png"
And the workspace should include "TwoMuonsTwoElectrons_Z1_mass.pdf"
And the workspace should include "TwoMuonsTwoElectrons_Z1_mass.png"
And the workspace should include "TwoMuonsTwoElectrons_Z2_mass.pdf"
And the workspace should include "TwoMuonsTwoElectrons_Z2_mass.png"

Scenario: The expected plots represent the expected data
When the workflow is finished
Then the size of the file "FourMuons_Z2_mass.pdf" should be between "16 KiB" and "17 KiB"
And the size of the file "TwoMuonsTwoElectrons_Higgs_mass.png" should be between "13 KiB" and "14 KiB"
And the size of the file "FourMuons_Z1_mass.png" should be between "14 KiB" and "15 KiB"
And the size of the file "plot.py" should be between "6 KiB" and "7 KiB"
And the size of the file "FourMuons_Higgs_mass.png" should be between "13 KiB" and "14 KiB"
And the size of the file "histograms.py" should be between "3 KiB" and "4 KiB"
And the size of the file "combined_Z2_mass.png" should be between "15 KiB" and "16 KiB"
And the size of the file "TwoMuonsTwoElectrons_Higgs_mass.pdf" should be between "16 KiB" and "17 KiB"
And the size of the file "TwoMuonsTwoElectrons_Z2_mass.png" should be between "14 KiB" and "15 KiB"
And the size of the file "FourElectrons_Z1_mass.pdf" should be between "15 KiB" and "16 KiB"
And the size of the file "combined_Z1_mass.png" should be between "14 KiB" and "15 KiB"
And the size of the file "FourElectrons_Z1_mass.png" should be between "13 KiB" and "14 KiB"
And the size of the file "combined_Higgs_mass.pdf" should be between "17 KiB" and "18 KiB"
And the size of the file "FourElectrons_Z2_mass.png" should be between "14 KiB" and "15 KiB"
And the size of the file "FourElectrons_Higgs_mass.pdf" should be between "15 KiB" and "16 KiB"
And the size of the file "combined_Z1_mass.pdf" should be between "16 KiB" and "17 KiB"
And the size of the file "FourMuons_Z1_mass.pdf" should be between "16 KiB" and "17 KiB"
And the size of the file "TwoMuonsTwoElectrons_Z2_mass.pdf" should be between "17 KiB" and "18 KiB"
And the size of the file "TwoMuonsTwoElectrons_Z1_mass.pdf" should be between "15.5 KiB" and "16.5 KiB"
And the size of the file "combined_Higgs_mass.png" should be between "15 KiB" and "16 KiB"
And the size of the file "FourElectrons_Z2_mass.pdf" should be between "16 KiB" and "17 KiB"
And the size of the file "FourElectrons_Higgs_mass.png" should be between "12 KiB" and "13 KiB"
And the size of the file "combined_Z2_mass.pdf" should be between "17 KiB" and "18 KiB"
And the size of the file "TwoMuonsTwoElectrons_Z1_mass.png" should be between "13 KiB" and "14 KiB"
And the size of the file "FourMuons_Z2_mass.png" should be between "14 KiB" and "15 KiB"
And the size of the file "FourMuons_Higgs_mass.pdf" should be between "16 KiB" and "17 KiB"