Autogenerate Java docs (II)

.github/workflows/pages.yml

@@ -0,0 +1,34 @@
+name: "CI Docs"
+
+on:
+  push:
+    branches:
+    - latest
+    tags:       
+    - '**'
+jobs:
+  release:
+    name: Build
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout
+      uses: actions/checkout@v2
+      with:
+        persist-credentials: false
+        fetch-depth: 0
+    - name: Set up Python
+      uses: actions/setup-python@v1
+      with:
+        python-version: 3.7
+    - name: Set up Doxygen
+      run: sudo apt-get install doxygen
+    - name: Build Sphinx docs
+      run: |
+        export PATH=$PATH:~/.local/bin
+        cd docs
+        make multiversion
+    - name: Deploy
+      run : ./docs/_utils/deploy.sh
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        LATEST_VERSION: latest

.gitignore

@@ -4,8 +4,10 @@ testing/
 .settings
 .classpath
 .project
-doc
-docs
+docs/_build
+docs/_source
+html
+latex
 notes
 .DS_Store
 

README-dev.rst

@@ -0,0 +1,36 @@
+Building the Docs
+=================
+
+*Note*: The docs build instructions have been tested with Sphinx 2.4.4 and Fedora 32.
+
+To build and preview the docs locally, you will need to install the following software:
+
+- `Git <https://git-scm.com/book/en/v2/Getting-Started-Installing-Git>`_
+- `Python 3.7 <https://www.python.org/downloads/>`_
+- `pip <https://pip.pypa.io/en/stable/installing/>`_
+- `doxygen <https://www.tutorialspoint.com/how-to-install-doxygen-on-ubuntu/>`_
+
+Run the following command to build the docs.
+
+.. code:: console
+
+    cd docs
+    make preview
+
+Once the command completes processing, open http://127.0.0.1:5500/ with your preferred browser.
+
+Building multiple documentation versions
+========================================
+
+Build Sphinx docs for all the versions defined in ``docs/conf.py``.
+
+The multiverson command does not build doxygen docs.
+
+.. code:: console
+
+    cd docs
+    make multiversion
+
+Then, open ``docs/_build/dirhtml/<version>/index.html`` with your preferred browser.
+
+**NOTE:** If you only can see docs generated for the master branch, try to run ``git fetch --tags`` to download the latest tags from remote.

README.md

@@ -1,3 +1,6 @@
+```eval_rst
+:orphan:
+```
 # Datastax Java Driver for Apache Cassandra®
 
 *If you're reading this on github.com, please note that this is the readme

changelog/README.md

@@ -5,25 +5,25 @@
   3.x versions get published.
 -->
 
-## 3.10.1
+### 3.10.1
 
 - [bug] JAVA-2857: Fix NPE when built statements without parameters are logged at TRACE level.
 - [bug] JAVA-2843: Successfully parse DSE table schema in OSS driver.
 
-## 3.10.0
+### 3.10.0
 
 - [improvement] JAVA-2676: Don't reschedule flusher after empty runs
 - [new feature] JAVA-2772: Support new protocol v5 message format
 
-## 3.9.0
+### 3.9.0
 
 - [bug] JAVA-2627: Avoid logging error message including stack trace in request handler.
 - [new feature] JAVA-2706: Add now_in_seconds to protocol v5 query messages.
 - [improvement] JAVA-2730: Add support for Cassandra® 4.0 table options
 - [improvement] JAVA-2702: Transient Replication Support for Cassandra® 4.0
 
 
-## 3.8.0
+### 3.8.0
 
 - [new feature] JAVA-2356: Support for DataStax Cloud API.
 - [improvement] JAVA-2483: Allow to provide secure bundle via URL.
@@ -48,10 +48,9 @@
 - [bug] JAVA-2174: Metadata.needsQuote should accept empty strings.
 - [bug] JAVA-2193: Fix flaky tests in WarningsTest.
 
-
 ### 3.7.0
 
-- [improvement] JAVA-2025: Include exception message in Abstract\*Codec.accepts(null).
+- [improvement] JAVA-2025: Include exception message in Abstract*Codec.accepts(null).
 - [improvement] JAVA-1980: Use covariant return types in RemoteEndpointAwareJdkSSLOptions.Builder methods.
 - [documentation] JAVA-2062: Document frozen collection preference with Mapper.
 - [bug] JAVA-2071: Fix NPE in ArrayBackedRow.toString().
@@ -80,13 +79,13 @@
 - [bug] JAVA-1924: StatementWrapper setters should return the wrapping statement.
 - [new feature] JAVA-1532: Add Codec support for Java 8's LocalDateTime and ZoneId.
 - [improvement] JAVA-1786: Use Google code formatter.
-- [bug] JAVA-1871: Change LOCAL\_SERIAL.isDCLocal() to return true.
+- [bug] JAVA-1871: Change LOCAL_SERIAL.isDCLocal() to return true.
 - [documentation] JAVA-1902: Clarify unavailable & request error in DefaultRetryPolicy javadoc.
 - [new feature] JAVA-1903: Add WhiteListPolicy.ofHosts.
 - [bug] JAVA-1928: Fix GuavaCompatibility for Guava 26.
 - [bug] JAVA-1935: Add null check in QueryConsistencyException.getHost.
 - [improvement] JAVA-1771: Send driver name and version in STARTUP message.
-- [improvement] JAVA-1388: Add dynamic port discovery for system.peers\_v2.
+- [improvement] JAVA-1388: Add dynamic port discovery for system.peers_v2.
 - [documentation] JAVA-1810: Note which setters are not propagated to PreparedStatement.
 - [bug] JAVA-1944: Surface Read and WriteFailureException to RetryPolicy.
 - [bug] JAVA-1211: Fix NPE in cluster close when cluster init fails.
@@ -661,7 +660,7 @@ Merged from 2.2 branch:
 - [bug] JAVA-817: TimeCodec does not format values correctly.
 - [bug] JAVA-818: TypeCodec.getDataTypeFor() does not handle LocalDate instances.
 - [improvement] JAVA-836: Make ResultSet#fetchMoreResult return a
-  ListenableFuture<ResultSet>.
+  ``ListenableFuture<ResultSet>``.
 - [improvement] JAVA-843: Disable frozen checks in mapper.
 - [improvement] JAVA-721: Allow user to register custom type codecs.
 - [improvement] JAVA-722: Support custom type codecs in mapper.
@@ -683,7 +682,7 @@ Merged from 2.2 branch:
 - [bug] JAVA-817: TimeCodec does not format values correctly.
 - [bug] JAVA-818: TypeCodec.getDataTypeFor() does not handle LocalDate instances.
 - [improvement] JAVA-836: Make ResultSet#fetchMoreResult return a
-  ListenableFuture<ResultSet>.
+  ``ListenableFuture<ResultSet>``.
 - [improvement] JAVA-843: Disable frozen checks in mapper.
 - [improvement] JAVA-721: Allow user to register custom type codecs.
 - [improvement] JAVA-722: Support custom type codecs in mapper.
@@ -984,7 +983,7 @@ Merged from 2.0 branch:
 - [bug] JAVA-595: Fix cluster name mismatch check at startup.
 - [bug] JAVA-620: Fix guava dependency when using OSGI.
 - [bug] JAVA-678: Fix handling of DROP events when ks name is case-sensitive.
-- [improvement] JAVA-631: Use List<?> instead of List<Object> in QueryBuilder API.
+- [improvement] JAVA-631: Use ``List<?>`` instead of ``List<Object>`` in QueryBuilder API.
 - [improvement] JAVA-654: Exclude Netty POM from META-INF in shaded JAR.
 - [bug] JAVA-655: Quote single quotes contained in table comments in asCQLQuery method.
 - [bug] JAVA-684: Empty TokenRange returned in a one token cluster.
@@ -1235,7 +1234,7 @@ Merged from 2.0.10_fixes branch:
 - [bug] JAVA-595: Fix cluster name mismatch check at startup.
 - [bug] JAVA-620: Fix guava dependency when using OSGI.
 - [bug] JAVA-678: Fix handling of DROP events when ks name is case-sensitive.
-- [improvement] JAVA-631: Use List<?> instead of List<Object> in QueryBuilder API.
+- [improvement] JAVA-631: Use ``List<?>`` instead of ``List<Object>`` in QueryBuilder API.
 - [improvement] JAVA-654: Exclude Netty POM from META-INF in shaded JAR.
 - [bug] JAVA-655: Quote single quotes contained in table comments in asCQLQuery method.
 - [bug] JAVA-684: Empty TokenRange returned in a one token cluster.

docs/Makefile

@@ -0,0 +1,78 @@
+# You can set these variables from the command line.
+POETRY        = $(HOME)/.poetry/bin/poetry
+SPHINXOPTS    =
+SPHINXBUILD   = $(POETRY) run sphinx-build
+PAPER         =
+BUILDDIR      = _build
+SOURCE_DIR    = _source
+
+# Internal variables.
+PAPEROPT_a4     = -D latex_paper_size=a4
+PAPEROPT_letter = -D latex_paper_size=letter
+ALLSPHINXOPTS   = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) $(SOURCE_DIR)
+# the i18n builder cannot share the environment and doctrees with the others
+I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+
+.PHONY: all
+all: dirhtml
+
+.PHONY: pristine
+pristine: clean
+	git clean -dfX
+
+.PHONY: setup
+setup:
+	./_utils/setup.sh
+	cp -Tr source $(SOURCE_DIR)
+	cd $(SOURCE_DIR) && find -name README.md -execdir mv '{}' index.md ';'
+.PHONY: clean
+clean:
+	rm -rf $(BUILDDIR)/*
+	rm -rf $(SOURCE_DIR)/*
+
+.PHONY: preview
+preview: setup
+	cd .. && ./docs/_utils/doxygen.sh
+	$(POETRY) run sphinx-autobuild -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml --port 5500
+
+.PHONY: dirhtml
+dirhtml: setup
+	$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
+	@echo
+	@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
+
+.PHONY: singlehtml
+singlehtml: setup
+	$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
+	@echo
+	@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
+
+.PHONY: epub
+epub: 	setup
+	$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
+	@echo
+	@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
+
+.PHONY: epub3
+epub3:	setup
+	$(SPHINXBUILD) -b epub3 $(ALLSPHINXOPTS) $(BUILDDIR)/epub3
+	@echo
+	@echo "Build finished. The epub3 file is in $(BUILDDIR)/epub3."
+
+.PHONY: dummy
+dummy: 	setup
+	$(SPHINXBUILD) -b dummy $(ALLSPHINXOPTS) $(BUILDDIR)/dummy
+	@echo
+	@echo "Build finished. Dummy builder generates no files."
+
+.PHONY: linkcheck
+linkcheck: setup
+	$(SPHINXBUILD) -b linkcheck $(SOURCE_DIR) $(BUILDDIR)/linkcheck
+
+.PHONY: multiversion
+multiversion: setup
+	@mkdir -p $(HOME)/.cache/pypoetry/virtualenvs
+	$(POETRY) run ./_utils/multiversion.sh
+	@echo
+	@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
+

docs/_utils/deploy.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+
+# Clone repo
+git clone "https://x-access-token:${GITHUB_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" --branch gh-pages --single-branch gh-pages
+cp -r docs/_build/dirhtml/* gh-pages
+# Redirections
+./docs/_utils/redirect.sh > gh-pages/index.html
+# Deploy
+cd gh-pages
+touch .nojekyll
+git config --local user.email "[email protected]"
+git config --local user.name "GitHub Action"
+git add .
+git commit -m "Publish docs" || true
+git push origin gh-pages --force

docs/_utils/doxygen.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+
+OUTPUT_DIR="docs/_build/dirhtml/api"
+if [[ "$SPHINX_MULTIVERSION_OUTPUTDIR" != "" ]]; then
+    OUTPUT_DIR="$SPHINX_MULTIVERSION_OUTPUTDIR/api"
+    echo "HTML_OUTPUT = $OUTPUT_DIR" >> doxyfile
+fi
+mkdir -p "$OUTPUT_DIR"
+doxygen doxyfile

docs/_utils/multiversion.sh

@@ -0,0 +1,5 @@
+#! /bin/bash	
+
+cd .. && sphinx-multiversion docs/source docs/_build/dirhtml \
+    --pre-build './docs/_utils/doxygen.sh' \
+    --pre-build "find docs/source -name README.md -execdir mv '{}' index.md ';'"

docs/_utils/redirect.sh

@@ -0,0 +1,13 @@
+#!/bin/bash
+
+cat <<- _EOF_
+<!DOCTYPE html>
+<html>
+  <head>
+    <title>Redirecting to Driver</title>
+    <meta charset="utf-8">
+    <meta http-equiv="refresh" content="0; URL=./${LATEST_VERSION}/index.html">
+    <link rel="canonical" href="./${LATEST_VERSION}/index.html">
+  </head>
+</html>
+_EOF_

docs/_utils/redirections.yaml

@@ -0,0 +1 @@
+api: /api/classes.html

docs/_utils/setup.sh

@@ -0,0 +1,10 @@
+#! /bin/bash
+
+if pwd | egrep -q '\s'; then
+	echo "Working directory name contains one or more spaces."
+	exit 1
+fi
+
+which python3 || { echo "Failed to find python3. Try installing Python for your operative system: https://www.python.org/downloads/" && exit 1; }
+which poetry || curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/1.1.3/get-poetry.py | python3 - && source ${HOME}/.poetry/env
+poetry install