Format Market Share Target and SDA articles in consistent template

[jdhoffa]

See examples of the articles here:
https://rmi-pacta.github.io/r2dii.analysis/articles/target-market-share.html
https://rmi-pacta.github.io/r2dii.analysis/articles/target-sda.html

We (Nick and I) have been thinking that it may be a good idea to define our own "template" for "RMI/ PACTA metrics".
This template could include sections around:

• Audience
• Description/ Formulae
• Expected outcome
• Impactful actions: e.g. "Use in a dialogue with a company"
• Use in combination with other metrics
• etc.

This would help focus and align our metrics, and give us a common starting point towards developing new metrics.

Obviously the scope of this is broader than this particular package, but I am thinking it may be good to prototype it here as Nick and I work out the kinks.

AB#9900

[jdhoffa]

I have had a chance to brainstorm a metric template here:
On metrics

I have sent this directly to Nicholas Dodd as I think he would be most interested in it(and is not on GitHub), however I imagine this may also be an interesting thing for:
@cjyetman @MonikaFu @jacobvjk

These are super rough notes at this stage! Curious if any of you folks have done anything similar or thought about this at all :-)

[jdhoffa]

Relates to #320

[MonikaFu]

Sounds like a very good idea to me! Maybe we could also link it to the plots we use for specific metrics where applicable. (I also proposed that in the loop document, not sure where the discussion is now.

[jdhoffa]

Definitely. I wonder also if it may be preferable to have it in a top-level wrapper package (e.g. pactaverse) or something like that, so it can reference any plot or dataset necessary?

I was thinking it's something that could be prototyped here, but may not live here at the end of the day

[MonikaFu]

Makes sense about the top level. Would we then move all documentation to the top level? Or would we maintain both?

Also (on the other hand), I am using r2dii.data and r2dii.analysis in the documentation for r2dii.plot so maybe vice versa is not such a bad idea. When you render the documentation, it would expose (some) breaking changes to r2dii.plot.

[jdhoffa]

My gut feeling is that having crisscrossed dependencies management can get quite confusing to manage.
e.g. managing APIs in the direction:
r2dii.data-> r2dii.match -> r2dii.analysis -> r2dii.plot

is quite simple. If anything earlier in the chain makes a "breaking API change" it's their responsibility to communicate it well, and everyone downstream to adapt.

When there is a bidirectional arrow on any of those, it gets very confusing to know who's responsibility it is to manage what.

Same as if dplyr makes a breaking API change, they notify us and we change things. But dplyr doesn't check a prior if changes they make are going to break r2dii.analysis

[jacobvjk]

Tell me if it is beyond the scope of the discussion, but in general I think we dearly need a well maintained public place where we have our full methodological documentation that we can cite/link to on reports. We have consistently been rewording the same pieces of text a slight bit to avoid "plagiarising" something that we came up with ourselves, because there was nothing available that we could properly cite. I don't know if everything needs to be in the same place. E.g. modelling of market share / SDA may be better documented in GitHub, whereas the scenario reference document may not. But I think this is something to consider, especially when trying to avoid documenting the same thing in multiple places.

[jdhoffa]

@jacobvjk I see this issue as a very clear first step to achieving this, yes.

My goal is that each pacta.* package has vignettes documenting the "methodology" achieved in that package, which would publicly render in a pkgdown site.

My further goal is that pactaverse would be able to render ALL methodology in it's top-level wrapper pkgdown site.

Shall we discuss this perhaps in a Tech Review?

[jacobvjk]

sounds like a good topic to me. Maybe not even controversial, but still good to have a short group discussion about it

[MonikaFu]

My gut feeling is that having crisscrossed dependencies management can get quite confusing to manage. e.g. managing APIs in the direction: r2dii.data-> r2dii.match -> r2dii.analysis -> r2dii.plot

is quite simple. If anything earlier in the chain makes a "breaking API change" it's their responsibility to communicate it well, and everyone downstream to adapt.

When there is a bidirectional arrow on any of those, it gets very confusing to know who's responsibility it is to manage what.

Same as if dplyr makes a breaking API change, they notify us and we change things. But dplyr doesn't check a prior if changes they make are going to break r2dii.analysis

I understand and in principle, I agree. However, with r2dii suite they are very tightly bound and I think that it might be useful to see the metrics visualized right next to the mathematical explanation of how they work. If that were the case then the need for upstream dependency management would be a side product.

[AlexAxthelm]

However, with r2dii suite they are very tightly bound and I think that it might be useful to see the metrics visualized right next to the mathematical explanation of how they work.

I think that's exactly for Suggests dependencies, rather than Imports (soft vs hard deps) We can put the downstream packages (like .plot) as Suggests dependencies, which means that we avoid a circular dependency problem, but can still use it while rendering documentation. Then the cocs for the methodology can live next to the methodology

[jdhoffa]

Understood, that the technical solution is to use Suggests, but my recommendation would be to avoid that circular inter-dependence altogether. As mentioned, I don't think it should be r2dii.analysis should be concerned with testing the API of r2dii.plot.

If the goal is to have demo visuals and data living under the same roof, I think that place would be an umbrella package (e.g. https://github.com/RMI-PACTA/pactaverse)

That package could in principle be responsible for building CI/CD across all of our use-cases (and in doing so, create demonstrative walkthroughs of how to use the various functions to create different visuals)

We can chat more in the Tech Review

[jacobvjk]

Another idea for combining everything into its full glory would be to create cook books for different use cases (which we are planning on doing in any case). I think this would probably not go into the level of detail that we would want to have in the individual repo documentation, but I think it is a great place for clarifying how a given metric is generated, visualised, and interpreted. There will probably be overlaps with the metric template, but it gives us space to avoid cramming all possible information into one document.

[jdhoffa]

Update comment, but yeah that's exactly what I had in mind really.
pactaverse could contain many many different vignettes, that walk through lots of different potential use-cases using demo data.

It could simultaneously be a demonstrative library of all of our metrics and packages AND an automated CI/CD system

@AlexAxthelm I imagine we could build an Action that ensures pactaverse builds all vignettes on any push to main across any of our packages. That could be a good "universal" check of the API consistency across all packages.

[AlexAxthelm]

If the goal is to have demo visuals and data living under the same roof, I think that place would be an umbrella package (e.g. https://github.com/RMI-PACTA/pactaverse)

I think this can lead to complications, since you would need to coordinate PRs across repos. It would be better to have 1 pr that has code and docs changes together, and depend on optional packages for generating docs where they are implemented. So methodology docs would likve in .analysis, plots about visualizations in .plot, etc.)

If you look at the tidyverse vignettes (https://github.com/tidyverse/tidyverse/tree/main/vignettes), they are generally about the "big picture" of using the whole ecosystem together.

https://r-pkgs.org/dependencies-in-practice.html#sec-dependencies-in-suggests-in-examples-and-vignettes

[jdhoffa]

^ that's exactly what I am talking about :-)