Rework documentation with JVM based site generator and mdoc

[ekrich]

The idea here is to make it as easy as working on the project using sbt to build the site - can have better integration than the current site which is hard to get configured.

Some Notes:

1. Ornate looks like it is not being maintained - it is really cool that it supports Math, diagrams, and other goodies. (szieger) is doing Scala at Databricks.
2. Pamflet (eed3si9n) is used for sbt so that is ok and cross Scala 2.12/2.13/3.0.
3. Laika is maintained by one person regularly and has 2.12/2.13/3.0 support.
4. Paradox (lightbend) is used by akka and other but looks like it could use some TLC.
5. Mdoc with Docusaurus (not JVM) - https://scalameta.org/mdoc/

Would be combined with mdoc for checked code.

Paradox with mdoc* - https://tpolecat.github.io/doobie/
Pamflet - https://www.scala-sbt.org/1.x/docs/
Docusaurus with mdoc sites - https://scalameta.org/mdoc/docs/docusaurus.html

Other references:
https://twitter.com/posco/status/1297224648988401664

• Maybe the best choice.

[keynmol]

A couple months back I did a quick experiment and converted RSTs and the site to markdown and docusaurus: https://new-scala-native-website.surge.sh/

Usage of mdoc will be complicated by the fact that SN will need a special modifier, and a couple of hacks (adding dependency on the SN libs) to make it compile-only. Execution of mdoc snippets will be difficult as you need to link, execute, embed output etc.

I'm all for a face lift for the site (my vote is for docusaurus because it integrates better with Algolia for search) and making it easier to build.

Really really like the look of ScalaCenter's AoC site: https://scalacenter.github.io/scala-advent-of-code/

[ekrich]

My heard that Akka also uses Algolia for search as well - https://akka.io/docs/

I was hoping that no dependencies would need to be installed so it would be easy for contributors, that Is why I was thinking of a Scala/Java/sbt solution.

[nafg]

For completeness, there's always the option of doing your own thing with scalatags or scalatex. (Or Twirl...) I wonder if mdoc exposes its core functionality as a function that you can pass a block of code with modifiers and get back the result.

Laika looks like the most hackable and "hookable" (i.e. if you need something it doesn't provide, easiest to add on top of it rather than having to wait for someone to merge a PR or create a kludgy workaround).

My experience with Paradox has been slightly disappointing.

But really, I could give better advice if you could someone make a list of requirements or a wishlist of what you need to be able to do in the site

[nafg]

One issue with Paradox is that it uses Parboiled which on newer JVMs has Jigsaw issues, so you have to add --add-opens java.base/java.lang=ALL-UNNAMED. (See e.g. lightbend/paradox#491)

But if you add that to your .jvmopts then Java 8 will refuse to start because of "Unrecognized option."

So now running CI across JVM versions is tricky. (The simplest solution is to build the docs under Java 8.)

[nafg]

Here's another major one I hit: lightbend/paradox#458

[nafg]

Here's another major one I hit: lightbend/paradox#458

I "solved" it... https://github.com/slick/slick/blob/dbb46ac190c7fc10bf39350f2ebe1fe2239fac4d/project/Docs.scala#L116-L128

[nafg]

Now I'm finding paradox to be super slow

[ekrich]

@nafg Sorry to hear that. Can we find someone willing to maintain Ornate?

[nafg]

Now I'm finding paradox to be super slow

Sorry this was user error lightbend/paradox#505

[SethTisue]

Java 8 will refuse to start because of "Unrecognized option."

can you -XX:+IgnoreUnrecognizedVMOptions your way out of that, perhaps?

or could the build simply (haha, yes I know, "simply") require JDK 11+, or even require 17+? it's getting to be time to consign Java 8 to the dustbin of history

[ekrich]

consign Java 8 to the dustbin of history

🎉

[i10416]

I was hoping that no dependencies would need to be installed so it would be easy for contributors,

Hello, I found scaladoc in dotty has SSG.
https://github.com/lampepfl/dotty/projects/7

I think it helps keep build/maintenance simple though It seems we need to wait for it to be stable for a while.

[ekrich]

Said by @keynmol

A couple months back I did a quick experiment and converted RSTs and the site to markdown and docusaurus: https://new-scala-native-website.surge.sh/

Usage of mdoc will be complicated by the fact that SN will need a special modifier, and a couple of hacks (adding dependency on the SN libs) to make it compile-only. Execution of mdoc snippets will be difficult as you need to link, execute, embed output etc.

I'm all for a face lift for the site (my vote is for docusaurus because it integrates better with Algolia for search) and making it easier to build.

Really really like the look of ScalaCenter's AoC site: https://scalacenter.github.io/scala-advent-of-code/

I think if the setup is really easy and it changes us to markdown then I think this is a win - I am not sure if rst is a great tech or just what was available. Do you know what Scala.js uses?

I think we should do something for 0.5.0. Just need some consensus. The single maintainer ones list above are very cool but now they are unmaintained so a mainstream approach would be a win.

[ekrich]

Maybe we could consider the new scaladoc? https://www.tooling-talks.com/episode-15

[ekrich]

We could use this to combine the separate API docs together? https://github.com/sbt/sbt-unidoc
This is the example site - https://dotty.epfl.ch/