Skip to content

Commit

Permalink
Modularise sim WF
Browse files Browse the repository at this point in the history
* offers
    * explicitly select modules and detectors with --modules and
      --readout-detectors flags, respectively
    * better overview and managing of dependencies

* introduce a SimulationWorkflow class
    * automatically resolves dependencies
    * introduce posiibility to identify a task by a stage and detector
      source such as ("RECO", "TPC")
      to make dependencies more clear based on what is actually going on
      NOTE: Still room for improvement
          * could do this for as many tasks as possible, atm only DIGI,
            RECO and MATCH tasks use this feature
          * task names could automatically be built from that
    * remove explicit TF suffix from task names and add automatically
    * does not add tasks for inactive detectors
    * possibility to compose source strings such as "ITS,TPC,TPC-TRD"
      automatically from proposed sources --> non-existent sources would
      be trimmed

* some tasks adjusted to their minimal dependencies

* small adjustments in o2dpg_qc_finalization_workflow.py to comply with
  <taskname>_<timeframe> format of tasknames per timeframe
  • Loading branch information
Benedikt Volkel committed Aug 1, 2022
1 parent dc72622 commit 51d384a
Show file tree
Hide file tree
Showing 2 changed files with 414 additions and 265 deletions.
4 changes: 2 additions & 2 deletions MC/bin/o2dpg_qc_finalization_workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ def add_QC_finalization(taskName, qcConfigPath, needs=None):
if standalone == True:
needs = []
elif needs == None:
needs = [taskName + '_local' + str(tf) for tf in range(1, ntimeframes + 1)]
needs = [taskName + '_local_' + str(tf) for tf in range(1, ntimeframes + 1)]

task = createTask(name=QC_finalize_name(taskName), needs=needs, cwd=qcdir, lab=["QC"], cpu=1, mem='2000')
task['cmd'] = f'o2-qc --config {qcConfigPath} --remote-batch {taskName}.root' + \
Expand Down Expand Up @@ -74,7 +74,7 @@ def add_QC_postprocessing(taskName, qcConfigPath, needs, runSpecific, prodSpecif
## The list of remote-batch workflows (reading the merged QC tasks results, applying Checks, uploading them to QCDB)
MFTDigitsQCneeds = []
for flp in range(5):
MFTDigitsQCneeds.extend(['mftDigitsQC'+str(flp)+'_local'+str(tf) for tf in range(1, ntimeframes + 1)])
MFTDigitsQCneeds.extend(['mftDigitsQC'+str(flp)+'_local_'+str(tf) for tf in range(1, ntimeframes + 1)])
add_QC_finalization('mftDigitsQC', 'json://${O2DPG_ROOT}/MC/config/QC/json/qc-mft-digit-0.json', MFTDigitsQCneeds)
add_QC_finalization('mftClustersQC', 'json://${O2DPG_ROOT}/MC/config/QC/json/qc-mft-cluster.json')
add_QC_finalization('mftAsyncQC', 'json://${O2DPG_ROOT}/MC/config/QC/json/qc-mft-async.json')
Expand Down
Loading

0 comments on commit 51d384a

Please sign in to comment.