Search: more robust symbol detection

[jtibshirani]

Zoekt currently uses ctags to detect symbols in documents and determine their types. Symbol information has an important role in ranking -- we boost files containing symbol matches highly in the search results.

We've traced several complaints about surprising ranking to incorrect ctags output:

• https://github.com/sourcegraph/sourcegraph/issues/45800
• ctags unexpected response for some C/C++ files zoekt#448
• https://news.ycombinator.com/item?id=34694621 (we didn't think drawPoint was a symbol)

Also here's an example where we mistake comments for symbols:

It would be great to switch to a more robust way of detecting symbols. Maybe we could integrate more closely with code intel and take advantage of our work on SCIP and tree-sitter? This issue is meant to document + brainstorm the problem.

/cc @sourcegraph/search-core

[jtibshirani]

Closing in favor of https://github.com/sourcegraph/sourcegraph/issues/52467.