Skip to content

Commit

Permalink
Document that certain faro steps now need a band constraint.
Browse files Browse the repository at this point in the history
When a task has band in its quantum dimensions but does not have band
in the dimensions of any of its inputs, it will run over all bands
known to any instrument unless those bands are constrained by the data
query.  It was an accident of the old QG generation algorithm that this
wasn't happening before (unrelated tasks in the same step were
providing a band constraint, but the middleware now sees them as
unrelated).
  • Loading branch information
TallJimbo committed Aug 23, 2023
1 parent ad30346 commit 5810ee8
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 3 deletions.
7 changes: 4 additions & 3 deletions pipelines/HSC/DRP-RC2_subset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -400,10 +400,11 @@ subsets:
- skyObjectStd
description: >
Tasks that can be run together, but only after the 'step1', 'step2',
'step3', and 'step4' subsets
'step3', and 'step4' subsets.
This step includes patch-level aggregation Tasks. These should be
run with explicit 'tract' constraints in the data query, otherwise
quanta will be created for jobs with only partial visit coverage.
run with explicit 'tract' and 'band' constraints in the data query, or
quanta will be created for jobs with only partial visit coverage and
for bands with no columns in the object table.
nightlyStep8:
subset:
- validateObjectTableCore
Expand Down
5 changes: 5 additions & 0 deletions pipelines/_ingredients/HSC/DRP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,8 @@ subsets:
description: |
Set of tract-level faro metrics to be run on coadd products. These use Object
Tables which are available after consolidateObjectTable (step3) is run.
These must have a band constraint in their data query to avoid generating
quanta for bands that do not have columns in the object table, since
using an object table dataset as input does not provide information to
the middleware about which tasks went into making the object table.
5 changes: 5 additions & 0 deletions pipelines/_ingredients/LSSTCam-imSim/DRP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -300,3 +300,8 @@ subsets:
description: |
Set of tract-level faro metrics to be run on coadd products. These use Object
Tables which are available after consolidateObjectTable (step3) is run.
These must have a band constraint in their data query to avoid generating
quanta for bands that do not have columns in the object table, since
using an object table dataset as input does not provide information to
the middleware about which tasks went into making the object table.

0 comments on commit 5810ee8

Please sign in to comment.