Add HealSparse property maps into Merian DRP

pipelines/_ingredients/DECam/DRP.yaml

@@ -167,53 +167,73 @@ subsets:
       Jointcal uses PreSources, generated by consolidatePreSourceTable,
       for all visits that overlap a tract. It also produces solutions
       per-tract, per-visit.
-  step2d:
+  step2cde:
     subset:
       - finalizeCharacterization
-      - writeRecalibratedSourceTable
-      - transformSourceTable
-      - consolidateSourceTable
       - updateVisitSummary
-    description: |
-      Visit-level tasks.
-      Allowed data query constraints: visit
-
-      writeRecalibratedSourceTable, transformSourceTable run per-detector
-      consolidateSourceTable produces one data product per visit.
-      finalizeCharacterization will eventually model full focal plane PSFs.
-  step2e:
-    subset:
       - makeCcdVisitTable
       - makeVisitTable
     description: |
-      Global-level tasks that must not be run with any data query constraints
-      Can be run anytime after subset step2d.
-
-      Allowed data query constraints: instrument
-
-      Tasks generate one data product per collection.
-      make[Ccd]VisitTable produces per-collection summary of the Visits
-      and CcdVisits.
+      Per-visit and per-collection tasks that can be run together after step2b
+      with no data query constraints other than instrument.
   step3:
     subset:
-      - selectGoodSeeingVisits
-      - makeWarp
-      - templateGen
+      - mergeMeasurements
       - deblend
+      - transformObjectTable
+      - forcedPhotCoadd
+      - healSparsePropertyMaps
+      - templateGen
+      - consolidateObjectTable
       - detection
+      - makeWarp
       - writeObjectTable
+      - selectGoodSeeingVisits
+      - mergeDetections
       - measure
-      - mergeMeasurements
       - assembleCoadd
-      - forcedPhotCoadd
-      - transformObjectTable
-      - consolidateObjectTable
-      - mergeDetections
     description: |
       Tract-level tasks that can be run together, but only after the 'step1'
       and 'step2' subsets.
 
       These should be run with explicit 'tract' constraints essentially all the
-      time, because otherwise quanta will be created for jobs with only partial visit
-      coverage.
-  # Steps 4-7 are not currently implemented in this DECam DRP processing pipeline.
+      time, because otherwise quanta will be created for jobs with only partial
+      visit coverage.
+  step4:
+    subset:
+      - writeRecalibratedSourceTable
+      - transformSourceTable
+    description: |
+      Tasks that can be run together, but only after the 'step1', 'step2' and
+      'step3' subsets
+
+      These detector-level tasks should not be run with 'tract' or 'patch' as
+      part of the data ID expression if all reference catalogs or diffIm
+      templates that cover these detector-level quanta are desired.
+  # Step 5 is not currently implemented in this DECam DRP processing pipeline.
+  step6:
+    subset:
+      - consolidateSourceTable
+    description: |
+      Tasks that can be run together, but only after the 'step1', 'step2',
+      'step3', and 'step4' subsets.
+
+      This step includes visit-level aggregation tasks. Running without tract
+      or patch in the data query is recommended, otherwise the outputs of
+      consolidateSourceTable will not contain complete visits.
+
+      This subset is separate from step4 to signal to operators to pause to
+      assess unexpected image differencing failures before these aggregation
+      steps. Otherwise, if run in the same quantum graph, aggregated data
+      products (e.g. sourceTable_visit) would not be created if one or more of
+      the expected inputs is missing.
+  step7:
+    subset:
+      - consolidateHealSparsePropertyMaps
+    description: |
+      Tasks that should be run as the final step that require global inputs,
+      and can be run after the 'step3' subset.
+
+      This step has global aggregation tasks to run over all visits, detectors,
+      tracts, etc.  This step should be run only with the instrument constraint
+      in the data query.