StableRNG in Turing is not yielding reproducible output

[sourish-cmi]

Hi All:

In the following code, we are using StableRNGs. However, the results are not reproducing. Can anybody help us? So that we can reproduce the results.

using Turing, StableRNGs

@model function example(x)
    m ~ Normal(0, 1)
    x ~ Normal(m, sqrt(1))
end

res = sample(StableRNG(123), example(1), NUTS(), 1000)

First Run

julia> res = sample(StableRNG(123), example(1), NUTS(), 1000)
┌ Info: Found initial step size
└   ϵ = 1.6
Chains MCMC chain (1000×13×1 Array{Float64, 3}):

Iterations        = 501:1:1500
Number of chains  = 1
Samples per chain = 1000
Wall duration     = 0.54 seconds
Compute duration  = 0.54 seconds
parameters        = m
internals         = lp, n_steps, is_accept, acceptance_rate, log_density, hamiltonian_energy, hamiltonian_energy_error, max_hamiltonian_energy_error, tree_depth, numerical_error, step_size, nom_step_size

Summary Statistics
  parameters      mean       std   naive_se      mcse        ess      rhat   ess_per_sec 
      Symbol   Float64   Float64    Float64   Float64    Float64   Float64       Float64 

           m    0.4905    0.6835     0.0216    0.0314   490.8002    1.0054      903.8677

Quantiles
  parameters      2.5%     25.0%     50.0%     75.0%     97.5% 
      Symbol   Float64   Float64   Float64   Float64   Float64 

           m   -0.8832    0.0403    0.4865    0.9456    1.7660

Second Run

julia> res = sample(StableRNG(123), example(1), NUTS(), 1000)
┌ Info: Found initial step size
└   ϵ = 3.2
Chains MCMC chain (1000×13×1 Array{Float64, 3}):

Iterations        = 501:1:1500
Number of chains  = 1
Samples per chain = 1000
Wall duration     = 0.55 seconds
Compute duration  = 0.55 seconds
parameters        = m
internals         = lp, n_steps, is_accept, acceptance_rate, log_density, hamiltonian_energy, hamiltonian_energy_error, max_hamiltonian_energy_error, tree_depth, numerical_error, step_size, nom_step_size

Summary Statistics
  parameters      mean       std   naive_se      mcse        ess      rhat   ess_per_sec 
      Symbol   Float64   Float64    Float64   Float64    Float64   Float64       Float64 

           m    0.5167    0.7029     0.0222    0.0301   533.6340    0.9999      975.5648

Quantiles
  parameters      2.5%     25.0%     50.0%     75.0%     97.5% 
      Symbol   Float64   Float64   Float64   Float64   Float64 

           m   -0.8081    0.0530    0.5059    0.9911    1.8123

Can you help us that what mistakes are we making?

[torfjelde]

Probably related: TuringLang/AdvancedHMC.jl#306

[sourish-cmi]

Probably related: TuringLang/AdvancedHMC.jl#306

Yes - providing step-size fixes with the issue for now. It is better to use user-provided rng - instead of global rng. Anyway, thanks for taking up the issue so quickly <3

Closing this issue - since you guys already have taken the necessary steps. Thanks again.