Pipeline hierarchy discussions

[rcannood]

Option 1: Current

%%| label: fig-example4
%%| fig-cap: "Example of how the concat and merges work.  \nGEX: Gene-expression. ADT: Antibody-Derived Tags."

flowchart LR


  Raw1[Raw counts 1] --> Split1[/Split\nmodalities/] --> ProcGEX1 & ProcADT1
  Raw2[Raw counts 2] --> Split2[/Split\nmodalities/] --> ProcGEX2 & ProcADT2
  ProcGEX1[/Process GEX\nprofile/] & ProcGEX2[/Process GEX\nprofile/] --> ConcatGEX[/Concatenate\nprofiles/] --> ProcGEX[/Process GEX\nprofiles/]
  ProcADT1[/Process ADT\nprofile/] & ProcADT2[/Process ADT\nprofile/] --> ConcatADT[/Concatenate\nprofiles/] --> ProcADT[/Process ADT\nprofiles/]
  ProcGEX & ProcADT --> Merge[/Merge\nmodalities/] --> Integration[/Integration/]

Loading

• Problematic because Process ADT profile (e.g. DSB) requires info from the Raw or GEX profile.

Option 2: Sequential

Process everything sequentially.

flowchart LR
  Raw1[Raw counts 1] --> ProcessGEX1[/Process GEX profile/] --> ProcessADT1[/Process ADT profile/] --> Concat
  Raw2[Raw counts 2] --> ProcessGEX2[/Process GEX profile/] --> ProcessADT2[/Process ADT profile/] --> Concat
  Concat --> ProcessGEX[/Process GEX profiles/] --> ProcessADT[/Process ADT profiles/] --> Integration

Loading

Option 3: Hierarchy of modalities

Treat GEX as the 'main' modality and pass the objects to other modalities.

flowchart LR
  Raw1[Raw counts 1] --> ProcessGEX1[/Process GEX profile/] --> ConcatGEX
  Raw2[Raw counts 2] --> ProcessGEX2[/Process GEX profile/] --> ConcatGEX
  ConcatGEX --> ProcessGEX[/Process GEX profiles/]

  Raw1 --> ProcessADT1[/Process ADT profile/] --> ConcatADT
  Raw2 --> ProcessADT2[/Process ADT profile/] --> ConcatADT
  ConcatADT --> ProcessADT[/Process ADT profiles/]

  ProcessGEX --> ProcessADT1
  ProcessGEX --> ProcessADT2

  ProcessGEX --> Merge[/Merge modalities/]
  ProcessADT --> Merge
  Merge --> Integration

Loading

Option 4: Pass cell calling

Extract the cell calling after the single-sample GEX processing and pass it to other modalities.

flowchart LR
  Raw1[Raw counts 1] --> ProcessGEX1[/Process GEX profile/] --> ConcatGEX
  Raw2[Raw counts 2] --> ProcessGEX2[/Process GEX profile/] --> ConcatGEX
  ConcatGEX --> ProcessGEX[/Process GEX profiles/]

  Raw1 --> ProcessADT1[/Process ADT profile/] --> ConcatADT
  Raw2 --> ProcessADT2[/Process ADT profile/] --> ConcatADT
  ConcatADT --> ProcessADT[/Process ADT profiles/]

  ProcessGEX1 --> CellCall1[/Extract cell\ncalling/] --> ProcessADT1
  ProcessGEX2 --> CellCall2[/Extract cell\ncalling/] --> ProcessADT2

  ProcessGEX --> Merge[/Merge modalities/]
  ProcessADT --> Merge
  Merge --> Integration

Loading

Option 5: Unfiltered processing before splitting

flowchart LR


  Unfilt1[Unfiltered\ncounts 1] --> UnfiltProc1[/Process\nunfiltered/] --> Split1[/Split\nmodalities/] --> ProcGEX1 & ProcADT1
  Unfilt2[Unfiltered\ncounts 2] --> UnfiltProc2[/Process\nunfiltered/] --> Split2[/Split\nmodalities/] --> ProcGEX2 & ProcADT2
  ProcGEX1[/Process GEX\nprofile/] & ProcGEX2[/Process GEX\nprofile/] --> ConcatGEX[/Concatenate\nprofiles/] --> ProcGEX[/Process GEX\nprofiles/]
  ProcADT1[/Process ADT\nprofile/] & ProcADT2[/Process ADT\nprofile/] --> ConcatADT[/Concatenate\nprofiles/] --> ProcADT[/Process ADT\nprofiles/]
  ProcGEX & ProcADT --> Merge[/Merge\nmodalities/] --> Integration[/Integration/]

Loading

Unfiltered processing includes:

• Ambient RNA correction
• Cell calling

[rcannood]

Option 5 seems like the preferred option here.