Document our design decisions/docs in this repo

See this thread: https://dhis2.slack.com/archives/CBM8LNEQM/p1592406496111500

It would be good to clearly document the design decisions for the components in this repo. That way maintainers and users know the intention of the components. It allows us to clearly establish what can be considered a bug, and how a component is meant to be used.

[amcgee]

Quoting from the slack discussion:

I would like to take a step back for a moment. I think the real deficiency here is in documentation - how am I, a first-time user, supposed to know the intended behavior of a menu and its various children?

I think if we had simple, plain-english documentation of what each component is and how it’s supposed to be used, then the features would be obvious (even the less-intuitive ones) and we could have informed discussions about missing or suspect features, and could confirm at least that there weren’t bugs in the implementation of those features.

A lot of context is lost when we exclusively rely on single-component Storybook stories and (somewhat) jsdoc API specs. A few more examples: what is a Box and why would I use it? How do I use various Form components together? And many more.

I think interactive documentation would be ideal for this - you could clearly see the result of certain JSX patterns - but even writing a Menu entry in markdown at ui.dhis2.nu (covering the various Menu flavors and their valid children) with screenshots would be incredibly useful. It would serve as a kind of “living record” of the macro design decisions we’ve made and how they affect library consumers in real-life scenarios. This could be implemented in Storybook, of course, but I separately worry about relying too heavily on that tool.

There's overlap with the design system. It seems practical to colocate that kind of documentation with the ui codebase. Should we integrate the design system with the ui codebase somehow? Maybe move over docs once they've been implemented? Or move over the entire design system to this repo? Have it function as documentation as well?

[varl]

Storybook makes a lot of sense from a development (automated testing is included in the term) perspective, where it is easy to build a component, and its features in isolation in addition to writing unit and e2e tests for those parts.

Storybook is no good at being our end-user documentation, which is why we remove the "notes" plugin and stripped down Storybook to the essentials in the first place.

Right now we have *.stories.testing.js for testing and *.stories.js for demos. I would like to:

• drop the concept of "demos" and merge the testing/demo stories into just *.stories.js and focus storybook as a tool for developers of components.
• merge the dhis2/design-system into the dhis2/ui docsite.
• replace the static images in the dhis2/design-system with live components in the docsite.

There's overlap with the design system. It seems practical to colocate that kind of documentation with the ui codebase. Should we integrate the design system with the ui codebase somehow? Maybe move over docs once they've been implemented? Or move over the entire design system to this repo? Have it function as documentation as well?

Before I went on parental leave @cooper-joe and I discussed this, and we were both in favor of archiving the dhis2/design-system repo and integrating it into dhis2/ui. It went on the backburner because we needed to get ui@5 out first.

[cooper-joe]

I've been thinking more about how to make this ui / design-system documentation as effective as possible. I also would like to use this oppurtunity to make the design system documentation more visible and encourage developers to reference this documentation when using the components.

I think there are a number of requirements we can state up front:

• Navigating to a component in the documentation should provide clear sections/tabs for:
  • Design system usage reference,
  • Interactive code examples / recipes for different common use cases,
  • API reference,
• The design system reference for each component should contain live demos of those components, not static screenshots,
• The documentation should provide guidance for composing bespoke components from atoms/molecules for individual use-cases.
• The documentation should be a trustworthy part of the DHIS2 core ecosystem, including DHIS2 branding and consistency with our other developer-facing materials.

Design System Reference

To encourage consumers of ui to use the design system usage reference information, I think that should be the 'start tab' for each component.

The reference text would include interactive, live components to illustrate the reference texts.

A current design system reference document has a structure for each page (usage, composition, types, options, examples). This structure may need rethinking in a new documentation setup.

The current design system has some non-component pages: 'Using the design system', 'Content & Communication', 'Layout...', 'Typography' etc. It may be that these sections need a different treatment, but ideally they will live alongside the component references.

Interactive code examples

Examples of each type/variant of the component with accompanying code. Using storybook style 'controls' would allow exploration of all the types/options without needing to create many stories.

API reference

Useful to have here, but should not be the only place to find the API. (Issue raised by @varl in an offline discussion: needing to load up a resource-heavy storybook just to reference the API is not great. Should be a lightweight API-reference-only in another location.)

---

I am open to using any kind of platform, storybook or others. I would ideally like to move forward quickly on this to encourage the use of the design system reference and components together.

[Mohammer5]

I like the direction this is taking! In terms of organizing, I really like the structure of the jest docs.

We're using one of the newer storybook version which supports mdx. We could use the storybook with mdx and it's folder mechanism.

@Mohammer5: We're using one of the newer storybook version which supports mdx. We could use the storybook with mdx and it's folder mechanism.

I think that if I understand @varl correctly, he wants to move away from using storybook for user facing documentation. I would be in favor of that as well. From a maintenance perspective, storybook has been a lot of work to keep up to date. It's the one thing in our lib that breaks a lot during upgrades, and I've personally found it the hardest to fix as well.

@varl: drop the concept of "demos" and merge the testing/demo stories into just *.stories.js and focus storybook as a tool for developers of components

Maybe we should elaborate on this in a separate issue? It seems like there is still a lot of room for interpretation. How do we imagine this workflow to work, etc.

---

I think it makes sense to split what has been mentioned so far in documentation requirements & documentation structure. I've tried to distill what has been mentioned so far, and added some items myself as well:

documentation requirements, the documentation should:

• Be a trustworthy part of the DHIS2 core ecosystem, including DHIS2 branding and consistency with our other developer-facing materials.
• Inform users on the purpose of the library and each component (design-system, scope of lib and components)
• Inform users how to use our components on a technical level (api docs)
• Help more experienced users to get started quickly (succinct getting started section)
• Show users how to compose our components with recipes for common usecases
• Allow users to experiment with live code examples (non storybook)
• Encourage users to engage with the development of the library (where to open issues and PRs, contributing docs)

As for the documentation structure:

1. Introduction and getting started section
2. Purpose of the lib
3. Design-system
   a. Using the design system
   b. Content & Communication
   c. Layout
   d. Typography
4. Component sections
   a. Purpose of the component
   b. API reference
   c. Live, interactive demos (non storybook)
5. Recipes for common use cases
6. Contributing

Let me know what you think, if anything is missing, incorrect, etc.

[Mohammer5]

1. Introduction and getting started section
2. Purpose of the lib

I'd mention purpose of the library in the introduction, other than that 👍 🎉

[cooper-joe]

Great write-up @ismay.

I had some clarifications/additional comments, which led me to some conceptual questions about how we want to present this library.

In requirements:

Inform users on the purpose of the library and each component (design-system, scope of lib and components)

I'd expand on what we mean by purpose here. I'd like to cover not only the purpose of the component, but it's guidelines for usage, when it should not be used, how to use effectively, and so on.

In structure:

3. Design-system

Just to note that this section could possibly expand. I'd like to include 'patterns' for common actions, such as 'how to present the user with edit mode' or 'how to display a step by step form'. These patterns do not fall into the component sections, nor do they belong in 'code recipes' that are included below. I think the most appropriate place would be as a subsection of this design system.

However, I'm wondering if it is wise to create this difference between the design system and the library...

---

Design System, or ui?

I have some concerns about the way we are communicating that the design system and ui are separate entities. This may make sense for us as the developers of these systems, ui is in fact an implementation of the design system, but I think it may be simpler for consumers of this library to see it as a single entity.

This would mean that ui, rather than 'design system', has a set of guidelines for use. The structure of the ui documentation would then be as follows:

1. Introduction, getting started
2. Purpose of this library
3. Guidelines for designing and building DHIS2 applications
   a. intro
   b. content and communication
   c. layout
   d. forms
   e. color
   f. typography
   g. iconography
4. Components
   a. Purpose, usage guidelines, options and types per component
   b. API reference
   c. Live interactive demos
5. Patterns/Recipes (code and design suggestions?) for common use cases
6. Contributing
7. Resources
   a. download design files, etc.

The 'design system' would cease to exist as a separate entity (😭). I think this is a reliable way to make the whole library feel cohesive, otherwise it will still be too easy to view the design system guidelines as extraneous to the components and DHIS2 applications themselves.

Let me know what you all think of this concept.

[varl]

@Mohammer5: We're using one of the newer storybook version which supports mdx. We could use the storybook with mdx and it's folder mechanism.

@ismay: I think that if I understand @varl correctly, he wants to move away from using storybook for user facing documentation. I would be in favor of that as well. From a maintenance perspective, storybook has been a lot of work to keep up to date. It's the one thing in our lib that breaks a lot during upgrades, and I've personally found it the hardest to fix as well.

First and foremost I am in favor of having a holistic approach to the docs. We've tried out some plugins in the past that were pretty bad.

I didn't know about MDX when I made the comment, so it is something to consider. It does look like what I had in mind from a presentation perspective.

I was thinking of going the other direction and integrating components and user docs into the "other" docs (ui.dhis2.nu) but maybe the other direction is better? To me it's an open question.

@cooper-joe That does tie the documentation together nicely. Your suggestions sound very good to me. One thing that it does make me wonder is whether we should consider the entire ecosystem of our frontend libs in this planning process. Since the documentation topic 'designing and building dhis2 apps' will likely also involve libs of ours that live outside of UI.

Should we draft a separate plan for docs that are dhis2-wide (frontend), so that we can see which would make sense on a UI level, and which should span our entire ecosystem? Otherwise I can imagine we could run into discussions about whether what's currently documented in the UI docs actually concerns the entire dhis2 library ecosystem, or vice-versa.

@varl I see. I don't have any strong preferences either. I ran into issues with storybook, but I can't say it's much more than me being slightly annoyed with it. I'm fine with either. Maybe it's possible to finalize the intended doc structure and requirements irrespective of tech., and then see what seems like the best technical solution to address those?

[varl]

@varl I see. I don't have any strong preferences either. I ran into issues with storybook, but I can't say it's much more than me being slightly annoyed with it. I'm fine with either. Maybe it's possible to finalize the intended doc structure and requirements irrespective of tech., and then see what seems like the best technical solution to address those?

I agree. Figuring out the doc structure that we want should drive what tech we use, not the other way around.

I've tried to integrate using raw React components into Docsify (our other doc framework), and while it is doable, it also has some annoyances. I used to think they were less than Storybook -- but with MDX maybe it works better?

One thing I do think is a must have is using the actual components everywhere in the documentation over e.g. using images of a Button.

[cooper-joe]

one thing that it does make me wonder is whether we should consider the entire ecosystem of our frontend libs in this planning process. Since the documentation topic 'designing and building dhis2 apps' will likely also involve libs of ours that live outside of UI.

Good point @ismay. The content inside 'Designing and building DHIS2 applications' could be more suited to a high level overview. This section, for example, is a very brief introduction to the user research/planning/design phase. This doesn't necessarily belong in the ui library. Pages like color and layout could be relevant in a higher level overview and a focused ui documentation, so there is some overlap. Generally speaking, pulling out the high level conceptual documentation and presenting this somewhere in the general dev documentation would make more sense.

I agree that drafting a plan for DHIS2-wide frontend documentation system would be a good idea, though I do want to ensure we do not spend too much time trying to plan the perfect system. The documentation will evolve over time (as it is doing right now), so I think identifying the missing pieces now and getting those patched up is my priority. The highest priority missing piece, from my perspective, is the missing link between ui documentation and "design-system" usage guidelines. A more cohesive front-end documentation system is definitely the end goal, just to say that I don't want the pursuit of the perfect system to lead us away from (what I consider) the pressing issue.

@cooper-joe Yeah I agree, that sounds pragmatic. We could always move documentation around afterwards. So we could just concern ourselves with the ui docs for the moment. And then after that, once we've finished the ui docs, discuss the docs on a higher level. Should we do that? So I guess then we only need to agree on the requirements and structure as outlined above? I'm ok with going ahead with what we've specified so far.

[amcgee]

Hi all - I haven't read this entire thread yet, it's quite hefty, but want to add a few thoughts off the bat:

1. I think it's critical that we have a singular and unified documentation system for all our developer tools
2. We've fragmented this discussion significantly - see for instance a proposal to add MDX and components-in-docs support to our docsite tool and a discussion about priorities and roadmap for our entire documentation ecosystem, also Consider consolidating with or bundling Storybook cli-utils-docsite#59, Our storybook demos aren't interactive ui#136, Explore adding support for an interactive documentation / code gen ui#135
3. Storybook has some performance downsides we should consider

I would need a lot of convincing to standardize on Storybook as our tool for ALL frontend libs, guides, tutorials, etc. and so I am strongly in favor of a solution that allows us to embed executable code samples and component demos in other places, and deprecating storybook (for documentation)

[varl]

Thanks for the links here, @amcgee. I see that I have missed and/or forgotten about a lot of prior context. For example, I forgot that MDX is standalone and not a Storybook specific thing. 🤦