Improve error message for initialization failures with troubleshootin… #2637

AoifeHughes · 2025-07-29T22:17:22Z

…g link

github-actions · 2025-07-29T22:19:43Z

Turing.jl documentation for PR #2637 is available at:
https://TuringLang.github.io/Turing.jl/previews/PR2637/

codecov · 2025-07-29T22:22:44Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.88%. Comparing base (97bcf89) to head (a178d70).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2637   +/-   ##
=======================================
  Coverage   84.88%   84.88%           
=======================================
  Files          22       22           
  Lines        1475     1475           
=======================================
  Hits         1252     1252           
  Misses        223      223

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

mhauru

Thanks Aoife, I left a few suggestions for the code.

For the test failures, none of them seem related to this PR, although one of them I would like to understand. Anything that has pre in the name is running on the upcoming release candidate version of Julia, v1.12-rc1. They may keep failing, and we should pay attention because we'll eventually have to fix those failures, but they are not a blocker for merging since v1.12 isn't out yet. The Inference tests running on v1 (e.g. latest stable release of Julia) seem to fail because coveralls is bugging out, hopefully will be fixed be rerunning. The one that troubles me is Inference tests running on min, i.e. the oldest version of Julia we claim to support, because that one segfaults. I've seen a couple of segfaults in tests over the last few weeks that have been fixed be rerunning, I'm starting to suspect that we have some nasty indeterministic test somewhere, probably to do with one of the AD backends.

Let's see what happens with the tests if we just rerun them. Regardless, they seem clearly unrelated to this PR.

mhauru · 2025-07-30T09:10:25Z

src/mcmc/hmc.jl

@@ -168,7 +168,7 @@ function find_initial_params(

    # if we failed to find valid initial parameters, error
    return error(
-        "failed to find valid initial parameters in $(max_attempts) tries. This may indicate an error with the model or AD backend; please open an issue at https://github.com/TuringLang/Turing.jl/issues",
+        "failed to find valid initial parameters in $(max_attempts) tries. This may indicate an error with the model or AD backend. See https://turinglang.org/docs/usage/troubleshooting/#initial-parameters for common causes and solutions. If the issue persists, please open an issue at https://github.com/TuringLang/Turing.jl/issues",


Maybe the sentence "This may indicate an error with the model or AD backend" can go? The message is quite long, and I think the link gives a more nuanced explanation of what might be wrong.

mhauru · 2025-07-30T09:11:36Z

test/mcmc/hmc.jl

+        err = nothing
+        try
+            sample(failing_model(), NUTS(), 10; progress=false)
+        catch e
+            err = e
+        end
+
+        @test err isa ErrorException
+        @test contains(
+            string(err),
+            "https://turinglang.org/docs/usage/troubleshooting/#initial-parameters",
+        )
+    end


This works, but check out Test.@test_throws, which can do this for for you.

mhauru · 2025-07-30T09:14:35Z

test/mcmc/hmc.jl

+            μ ~ Uniform(-π, π)
+            κ ~ InverseGamma(2, 3)
+            return x ~ VonMises(μ, κ)


This model might get fixed at some point, independently of us (the issue is in AD packages/Distributions.jl). Would something like any super simple model but with @addlogprob!!(-Inf) thrown in at the end trigger the same error more reliably? @addlogprob!! is a thing you can call within a model to add any number you want to the log likelihood, and I think @addlogprob!!(-Inf) should make all samples be rejected. @addlogprob!!(NaN) might work too.

line 208 through 215 already contains the same test.

with @addlogprob! -Inf

…t information and enhance clarity

coveralls · 2025-08-06T11:19:49Z

Pull Request Test Coverage Report for Build 16832126241

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0 changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 84.997%

Totals
Change from base Build 16734461208:	0.0%
Covered Lines:	1252
Relevant Lines:	1473

💛 - Coveralls

mhauru

Looks good. One final nit about a simplification to the tests.

mhauru · 2025-08-06T12:56:23Z

test/mcmc/hmc.jl

+        # Verify the error message contains the troubleshooting link
+        err = try
+            sample(failing_model(), NUTS(), 10; progress=false)
+        catch e
+            e
+        end
+
+        @test contains(
+            string(err),
+            "https://turinglang.org/docs/usage/troubleshooting/#initial-parameters",
+        )


This, too, can in fact be done with @test_throws, since its first argument can be a string that must be contained in the error message. E.g.

julia> f() = throw(ErrorException("this is an error message that contains the phrase 'this must be in the error'")) f (generic function with 1 method) julia> @test_throws "this must be in the error" f() Test Passed Message: "this is an error message that contains the phrase 'this must be in the error'"

I'm not sure if you can check for both the substring and the exception type in one call to @test_throws. There may be a way, or you may need to call @test_throws twice.

yebai · 2025-08-07T10:10:26Z

src/mcmc/hmc.jl

@@ -168,7 +168,7 @@ function find_initial_params(

    # if we failed to find valid initial parameters, error
    return error(
-        "failed to find valid initial parameters in $(max_attempts) tries. This may indicate an error with the model or AD backend; please open an issue at https://github.com/TuringLang/Turing.jl/issues",
+        "failed to find valid initial parameters in $(max_attempts) tries. See https://turinglang.org/docs/usage/troubleshooting/#initial-parameters for common causes and solutions. If the issue persists, please open an issue at https://github.com/TuringLang/Turing.jl/issues",


Can we set up a more robust URI for this link, e.g. via https://turinglang.org/docs/uri/initial-parameters? This URI redirects to the corresponding docs link, and would never change if we restructure the docs.

Quarto supports this via redirects: https://quarto.org/docs/websites/website-navigation.html#redirects

Cc @shravanngoswamii

Sure, let me add a redirects template in docs repo so we can easily create permanent redirects for docs.

TuringLang/docs#632

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Improve error message for initialization failures with troubleshootin…

67efd02

…g link

github-actions bot assigned AoifeHughes Jul 29, 2025

mhauru requested changes Jul 30, 2025

View reviewed changes

AoifeHughes added 2 commits August 6, 2025 11:56

Refactor error message for initialization failures to remove redundan…

9c33116

…t information and enhance clarity

fc

85c7f74

Merge branch 'main' into ah-dists

fa97490

mhauru requested changes Aug 6, 2025

View reviewed changes

simplier

a0fe160

yebai reviewed Aug 7, 2025

View reviewed changes

shravanngoswamii mentioned this pull request Aug 7, 2025

Added url redirects TuringLang/docs#632

Open

Update test/mcmc/hmc.jl

a178d70

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve error message for initialization failures with troubleshootin… #2637

Improve error message for initialization failures with troubleshootin… #2637

Uh oh!

AoifeHughes commented Jul 29, 2025

Uh oh!

github-actions bot commented Jul 29, 2025

Uh oh!

codecov bot commented Jul 29, 2025 •

edited

Loading

Uh oh!

mhauru left a comment

Uh oh!

mhauru Jul 30, 2025 •

edited

Loading

Uh oh!

mhauru Jul 30, 2025

Uh oh!

mhauru Jul 30, 2025

Uh oh!

penelopeysm Jul 30, 2025

Uh oh!

penelopeysm Jul 30, 2025

Uh oh!

coveralls commented Aug 6, 2025 •

edited

Loading

Uh oh!

mhauru left a comment

Uh oh!

mhauru Aug 6, 2025

Uh oh!

yebai Aug 7, 2025

Uh oh!

shravanngoswamii Aug 7, 2025 •

edited

Loading

Uh oh!

shravanngoswamii Aug 7, 2025

Uh oh!

Uh oh!

Improve error message for initialization failures with troubleshootin… #2637

Are you sure you want to change the base?

Improve error message for initialization failures with troubleshootin… #2637

Uh oh!

Conversation

AoifeHughes commented Jul 29, 2025

Uh oh!

github-actions bot commented Jul 29, 2025

Uh oh!

codecov bot commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mhauru left a comment

Choose a reason for hiding this comment

Uh oh!

mhauru Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mhauru Jul 30, 2025

Choose a reason for hiding this comment

Uh oh!

mhauru Jul 30, 2025

Choose a reason for hiding this comment

Uh oh!

penelopeysm Jul 30, 2025

Choose a reason for hiding this comment

Uh oh!

penelopeysm Jul 30, 2025

Choose a reason for hiding this comment

Uh oh!

coveralls commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 16832126241

Warning: This coverage report may be inaccurate.

Details

💛 - Coveralls

Uh oh!

mhauru left a comment

Choose a reason for hiding this comment

Uh oh!

mhauru Aug 6, 2025

Choose a reason for hiding this comment

Uh oh!

yebai Aug 7, 2025

Choose a reason for hiding this comment

Uh oh!

shravanngoswamii Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shravanngoswamii Aug 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Jul 29, 2025 •

edited

Loading

mhauru Jul 30, 2025 •

edited

Loading

coveralls commented Aug 6, 2025 •

edited

Loading

shravanngoswamii Aug 7, 2025 •

edited

Loading