Sphinx parallel errors

[aaronmarkham]

Ran into an issue when using nbconvert master as a dep in our Sphinx build pipeline.

• git+https://github.com/jupyter/nbconvert@master
• nbsphinx>=0.4.2

Lots of different errors on different files but it was always something to do with Sphinx parallel and reducing -j to 1 would get around the error.
Eventually we figured out if we switch from master to the latest release it fixes the issue.
Tracked the issue here:
apache/mxnet#16624

So, it would seem there's some sort of issue in master at this point that warrants investigation.

This was the environment:
https://github.com/apache/incubator-mxnet/blob/master/docs/python_docs/environment.yml

[ChaiBapchya]

might want to point to environment.yml of particular commit (since it would get updated after
apache/mxnet#16630

[MSeal]

Well, we merged out 6.x branch to master after we released 5.6.1 -- which reworks how templates are organized -- so I'm not surprised you'd see nbsphinx.NotebookError: TemplateNotFound in tutorials/deploy/export/onnx.ipynb: rst.tpl. You probably shouldn't be pulling a master commit into a build process -- I'd keep to released versions.

I only see an error for template not found. Where there any logs showing other errors that weren't posted? I'd expect -j 1 to also fail if it can't find a template.

[aaronmarkham]

I had to use master due to an issue from pre-5.5, then just forgot to switch it back after the release.

Here were a couple more.
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwebsite/detail/PR-16615/7/pipeline
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwebsite/detail/PR-16619/3/pipeline

When I was testing it, if I removed a tutorial that it errored on, it would just error on a different one.

[MSeal]

As @ChaiBapchya suggested next time you probably want to pin to a commit for unreleased fixes next time.

But, yes based on those failure message I think this expected as there'll need to be an update on sphinx to be compatible with nbconvert 6.0 when it releases. I'll create an issue there so they have some heads up for the changes.

[MSeal]

I opened an issue on sphinx-doc/sphinx#6773 and am going to close the one here since this is a future concern and the other thread can focus on future proofing the libraries.

[mgeier]

AFAICT, this has nothing to do with parallel builds.

The error (and it is an error from nbsphinx, not from Sphinx itself) occurs because the template nbconvert/templates/rst.tpl has been renamed to share/jupyter/nbconvert/templates/rst/index.rst.j2 in #1056, which simply is a breaking change.

This has nothing to do with Sphinx templates, it's just about nbconvert templates..