GSoC 2024- Scaladoc Search in Protosearch #241
Conversation
Fix `scaladoc` sbt project build
Merge changes from 'scaladoc-plugin'
There was a problem hiding this comment.
Thanks @golden-skipper03 for all your hard work on this!
I think we're getting very close to merge ready. I've left a handful of comments and change requests here, but they're mostly minor things. The bulk of your work here is done and working nicely :)
| worker.onmessage = function(e) { | ||
| app.innerHTML = e.data | ||
| console.log(e.data) |
There was a problem hiding this comment.
| console.log(e.data) |
We only needed this during testing.
| .sortBy(_._1) | ||
| .reverse |
There was a problem hiding this comment.
| .sortBy(_._1) | |
| .reverse | |
| .sortBy(-_._1) |
If we sort by a different condition we can avoid reversing the list.
|
|
||
| object ScaladocIndexer { | ||
|
|
||
| def createScaladocIndex(scaladocInfoList: List[ScaladocInfo]): MultiIndex = { |
There was a problem hiding this comment.
Instead of providing this helper function, let's have ScaladocIndexer just expose the analyzer and the IndexBuilder.
Something like:
object ScaladocIndexer {
val analyzer = ...
val indexBuilder = ...
}There was a problem hiding this comment.
When you do this, you'll have to update the buildIndex line in ProtosearchScaladocPlugin.scala to something like:
val buildIndex = scaladocInfos.compile.toList.map(ScaladocIndexer.indexBuilder.fromList)The motivation for doing this is that IndexBuilder will eventually have different construction methods so we don't have to use an intermediate List like we are now.
|
|
||
| package pink.cozydev.protosearch.scaladoc | ||
|
|
||
| object ScaladocSearcher { |
There was a problem hiding this comment.
I think we can remove this file completely as it is likely to only be used in your unit tests.
You can just copy the searchScaladoc function into your test suite if necessary.
| mod.toString | ||
| } | ||
|
|
||
| val hyperlinks = urlRegex.findAllMatchIn(rawComment).map(_.matched).toList |
There was a problem hiding this comment.
What's the value that extracting hyperlinks in this way adds?
I am unsure that a user will ever want to search for scaladocs that contain a certain hyperlink. Maybe I am missing something, but that strikes me as not a particularly common/useful flow.
Additionally this will only find fully formed URLs, not all links from one scaladoc to another.
For example, note the relative link to Stream.map here: https://github.com/typelevel/fs2/blob/ff9b4d19e0040855a35e00560eb3cc0f35c7c2c7/core/shared/src/main/scala/fs2/Stream.scala#L145
This would not appear in hyperlinks as it's currently defined.
| val optionalParams = paramss.flatMap { case Member.ParamClauseGroup(_, params) => | ||
| params.flatMap { case clause: Member.ParamClause => | ||
| clause.values.collect { | ||
| case param: Term.Param if param.default.isDefined => | ||
| val commentDescription: String = paramsComm | ||
| .find(_.startsWith(param.name.value)) | ||
| .getOrElse(param.name.value) | ||
| val declaredType: String = param.decltpe.map(_.toString).getOrElse("Unknown Type") | ||
|
|
||
| s"$commentDescription: $declaredType" | ||
| } | ||
| } | ||
| } | ||
|
|
||
| val implicitParams = paramss.flatMap { case Member.ParamClauseGroup(_, params) => | ||
| params.flatMap { case clause: Member.ParamClause => | ||
| clause.values.collect { | ||
| case param: Term.Param if param.mods.exists(_.is[Mod.Implicit]) => | ||
| val commentDescription: String = paramsComm | ||
| .find(_.startsWith(param.name.value)) | ||
| .getOrElse(param.name.value) | ||
| val declaredType: String = param.decltpe.map(_.toString).getOrElse("Unknown Type") | ||
|
|
||
| s"$commentDescription: $declaredType" | ||
| } | ||
| } | ||
| } | ||
|
|
||
| val allParams = typeParams ++ params ++ optionalParams ++ implicitParams |
There was a problem hiding this comment.
Now that we're combining all the various param types, do we need to separate them out in the first place?
Instead of having optionalParams and implicitParams could we modify params to capture all types by removing the guard clause on the match?
e.g. going from
case param: Term.Param
if param.default.isEmpty && !param.mods.exists(_.is[Mod.Implicit])
to just
case param: Term.Param
| } | ||
| } | ||
|
|
||
| """ |
There was a problem hiding this comment.
This test input has a lot of code but very little scaladoc that we're actually testing. Could we perhaps shrink it down?
This pull request introduces key features to integrate Scaladoc search into Protosearch, making it easier to search through documentation contained within Scala source files.
Key changes-
Visible changes-
Instructions to follow-
projectName/createScaladocIndexsite\target\docs\site\searchdirectorypython -m http.serverlocalhost:XXXX/search/search.html?index="indexName"&type=scaladoc