ConfAsDocs: Idea for new sphinx extension

[danwos]

Problem

In some bigger projects it is often the case, that a tool-configuration needs to be reused in different tools as well.
For instance the configured need-types shall be documented in a sphinx project, they shall be part of an IDE specific config to allow auto-completion during writing and they shall be presented in a diagram as need-model.

Additionally configs may need to be shared, inherited or archived.

The problem is, that each "representation/usage" need common but also additional kind of information:

• A sphinx conf.py needs name, value and maybe a short comment
• A documentation needs name, value and a longer description. Maybe combined with a feature for discussion/comments.
• An IDE-config needs the specific value and maybe the combination with other configs
• A diagram needs value and an additional relationship

So from the technical part an overall config would be needed, which is then separated into tool-specific configs.

Example:

But replacing a config file with an even more complex config file sounds not like a good solution.

Idea

Instead of creating a new, additional "super-config", we could also use one of the needed "configs/output formats" to store all information.
And the only format, which allows to have an extensible format to define additional options is rst, or better Sphinx together with Sphinx-Needs. Therefor I call this concept ConfAsDocs

It may be used like this:

Audience

Who may use it:

• Projects in bigger companies with a strong need to document configs (maybe because of Standards to use, e.g. ISO26262)
• Projects with bigger, decentralized teams to share knowledge
• Projects with a lot of config-variants and the need to document this
• Projects with tools, which config formats do not allow comments
• Projects with a strong deployment automation and the need for a standardized config handling
• Projects with a strong review-based process and the need to perform diffs and have history data (if rst is put into git and co.)

Who should not use it (overkill):

• Single-person projects
• Small projects with a small team, where the communication is working.
• Projects using other solutions to handle and document config
• Project without CI and/or CD (configs are generated and deployed by hand).

Spec (rough)

in my understanding we have two kind of main information/objects:

• config: The name of the configuration parameters, which are need/supported by a tool
• values: The value for a config parameter, which shall be set for a specific tool instance/project.

Therefore the rst code may look like:

.. config:: needs_types
   :tool: sphinx_needs
   :type: list
   
    The option allows the setup of own need types like bugs, user_stories and more.
    
    By default it is set to:  [...]


.. needs_types:: dict(directive="req", title="Requirement", prefix="R_", color="#BFD8D2", style="node")
   :tool: sphinx_needs
   :instance: my_project  

    Documents our requirements.
    Please set always the following options:
    
    * status
    * author

.. needs_types:: dict(directive="need", title="Need", prefix="N_", color="#9856a5", style="node")
   :tool: sphinx_needs
   :instance: my_project  
[...]

So the defined config parameters are available as directives, to easily add a value.
This would need to run a sphinx-build action twice, if a new config parameter got added.
Reason: 1. Run: Detect all config parameters. 2. Run: Register config-params as directives.

The config-parameter set is defined only once per tool, so for Open-Source projects they may be shared and and get imported, if no project specific documentation is needed for them.

Values types to be supported:

• Strings
• Numbers
• Lists
• Dicts

I'm not yet 100% sure what is the best way to define and manipulate lists and dicts.
In the example you see that for each entry in a list a single directive-call is made.
So in this case it is really easy to document each single element of a list.

There may be options to override a complete list to define it completely by one directive.
Have to make some tests :) (Feedback welcome!)

Simplification

As you may have recognized, each config must be assigned to a tool.
And each config-value must be assigned to a tool and an instance.

As in most cases most configs and config-values for a tool/instance are written down on the same page, there are some simplification planned.

The above example can be written also like this:

.. conf_tool:: sphinx_needs
.. conf_instance:: my_project

.. config:: needs_types
   :type: list
   
    The option allows the setup of own need types like bugs, user_stories and more.
    
    By default it is set to:  [...]


.. needs_types:: dict(directive="req", title="Requirement", prefix="R_", color="#BFD8D2", style="node")

    Documents our requirements.
    Please set always the following options:
    
    * status
    * author

.. needs_types:: dict(directive="need", title="Need", prefix="N_", color="#9856a5", style="node")

[...]

.. conf_tool:: another_tool
   .. but same instance
   
.. another_config:: another-value

[...]

conf_tool and conf_instance allow to activate/set the related values for each following config and config-value.
They get used until another conf_tool or conf_instance is used.

Output

The output shall not use the typical layout/style of sphinx-needs.
Instead new layouts and styles shall be configured, to allow a more seamless integration into the documentation flow.

Other magics / ideas

• The needed sphinx-needs id shall be generated automatically.
  Rule: instance.tool.config, e.g. my-project.sphinx.html-theme
• Links between config needs and value needs shall be set automatically.
• There may be need-types for tool and instance (may be decided and configured by user on its own with already available sphinx-needs power)

That's all folks. Any feedback?

[twodrops]

@danwos Thanks a lot for all the great ideas you wrote above.

Firstly, some comments on my current requirements

In my project we need a central "needs model" to generate the following:

1. need types, options and links in conf.py - for sphinx-needs
2. Visual Studio Code snippets - as templates in VS Code editor
3. Documentation of needs - for users to know what each need type with its options and link means
4. Visualization in PlantUML - for seeing the entire needs as a model to ensure consistency, find duplication etc. (currently 10 to 12 people are creating need elements and we could lose the overview easily)
5. Linter configurations - for rstcheck to consider the directives created by sphinx-needs

Additional to this, it would be nice to have a "needs model" to model needs as a structure which is closer to the domain. This means if we have 10 need types and 100 need options, from the domain point of view those options might be valid for some of the need-types only. sphinx-needs currently supports only a flat list of options. It would be nice if the model can show this, but in the generated conf.py it would be removed and (if possible) need-warnings created to check that the options-usage is according to the defined model. The last part was your idea @danwos :)