feat(parser): add output references to for gen flows

[chrismclarke]

PR Checklist

• PR title descriptive (can be used in release notes)

Description

Following on from #2440, this PR reverts the change so that data_pipe and generator flows are again stored within the output jsons (and still copied to main app), but also adds an updated _output_flows property to each that creates a reference to the generated outputs for use in debugging

Review Notes

See example pre/post changes output on debug repo created via pr
https://github.com/IDEMSInternational/app-debug-content/pull/92/files

Dev Notes

There's a few extra changes made to the code just to make the functions more consistent between data_pipe and generator parsers. Now I've done that I can clearly see about 90% of the code is identical between them (which makes sense given they both primarily function to take a bunch of flow inputs and some parameters and produce flow outputs). I had thought to try to unify the two, but possibly overkill as the duplicate code should still be reasonably easy to maintain in both for now.

In the future maybe we could merge both under some sort of transformer flow type or similar, or maybe it will be clear enough that data_pipes focus on transforming data_list types and generators on anything else.

Git Issues

Closes #

Screenshots/Videos

If useful, provide screenshot or capture to highlight main changes

[esmeetewinkel]

I inspected the two debug content sync PRs and all looks great, thanks.

[jfmcquade]

Looks good.

There's a few extra changes made to the code just to make the functions more consistent between data_pipe and generator parsers. [...] In the future maybe we could merge both under some sort of transformer flow type or similar, or maybe it will be clear enough that data_pipes focus on transforming data_list types and generators on anything else.

I suppose the question is really whether that would make sense from an authoring perspective – if conceptually it would make sense to unite them then I think that could be good. What do you think @esmeetewinkel? I do agree that for now the distinction makes sense as one operates on data_listss and the other on other flow types.

[esmeetewinkel]

What do you think @esmeetewinkel? I do agree that for now the distinction makes sense

To me generators and data pipes are quite different things, although I appreciate they might grow more similar as we extend functionality

... as one operates on data_lists and the other on other flow types.

I'm confused by this statement - generators can also operate on data lists. To me the distinction is

• generators iteratively (looping over a data list) create content of any flow type, whereas
• data pipes modify data lists

[chrismclarke]

To me generators and data pipes are quite different things, although I appreciate they might grow more similar as we extend functionality

... as one operates on data_lists and the other on other flow types.

I'm confused by this statement - generators can also operate on data lists. To me the distinction is

• generators iteratively (looping over a data list) create content of any flow type, whereas
• data pipes modify data lists

You're right, that doesn't summarise very well. Data pipes also loop over a data_list, just like a generator, but only output to data_lists (they create new outputs instead of modifying the input, so behave in the same way as generators for reading inputs and writing outputs).

But yeah, in terms of the code the duplication is to both read in the source data and output the generated data, then both have a 1-line call to process the data - but then the specific processors are totally different (array map, filter, sort operations vs find/replace). So I guess if we ever wanted to combine we could always add a new pipe operation to handle replacement of templated variables, but also happy to keep separate