Add ability to search and jump-to docs-page anchors

[kroeder]

Is your feature request related to a problem? Please describe.

The current search index seems to contain only navigation items from the manager sidebar.

In v6, all <Canvas><Story /></Canvas> of a docs-page created a sidebar navigation item.

Given a navigation structure in v6

Table
- Sorting
- Filtering
- API

You were able to enter "Table sorting" in the search-bar and directly jump to the "Sorting" story in the docs-page (+ it called scrollIntoView())

In v7, MDX and stories are strictly separated. MDX files no longer create sub-navigation items. If user-land users search for "Table sorting", they now jump to the Canvas version instead of the docs-page version with additional context information, code snippets, API explanations etc.

If you built a design-system with Storybook, the consequence is to build storybook with --docs. Now, only detailed docs-pages are rendered in the navigation structure. The same example above is now just:

Table

This leaves us with only trade-off solutions:

• Either you build Storybook without --docs and users now can search with more details for a certain story but they ultimately end up in a Canvas and not the actual documentation
• Or you build Storybook with --docs but now the search index shrinks to almost not usable if you are used to the previous behavior in v6. Users now need to first search for a certain component, e.g. "Table", then read the whole table-of-contents until they find what they were looking for.
• Or you create an extra MDX file for everything and split documentation, that originally belonged together, in multiple files but now you can search for it. Users then need to jump between manager navigation items instead of one docs-page where they can scroll through

The goal of the discussion is to find a proper way to improve the search UX.

Describe the solution you'd like

Variant A: Render sub-navigation-items as in v6

Render docs-pages as in v6 and use the sub-navigation items as anchors

// Version without `--docs`
Table
- Docs
  - Sorting
  - Filtering
  - API 
- Sorting
- Filtering
- API

// Version with `--docs`
Table
- Sorting
- Filtering
- API

Advantages:

• Search index would automatically be extended by the new navigation items
• Behaves like v6, users do not need to learn a new UX pattern

Disadvantages:

• Mimics behavior of the table-of-contents feature and adds additional UI/UX complexity.
  • What happens if you enter a docs-page via manager sidebar and then click on another docs-page toc item? Does it also change the active item in the manager?
  • Do you even need the toc-feature or is it just a duplicate, then?
  • What about deeply nested navigation items ##, ###, ####?
• Duplicate entries if built without --docs

Variant B: Extend the search-index with the docs-page headline anchors

Make the search index extendable.

E.g. the MDX parser could fetch additional context information from MDX files. E.g., collect all #, ##, ### and add them to the search-index.

Advantages:

• The parsed headlines can directly be used as anchors to jump from the search-bar directly into the docs-page + scrollIntoView() the clicked search item
• Same as for the table-of-contents, an additional property could be used to control the search-index extension

toc: { 
  headingSelector: 'h1, h2, h3',
  searchIndexSelector: 'h1, h2' // <-- new
}

Disadvantages:

• (tbd; I don't know the parser and the downside of this approach)

Describe alternatives you've considered

No response

Are you able to assist to bring the feature to reality?

yes, I can

Additional context

No response