GPUArrays seeding does not generate reproducible values past a certain array size

[christiangnrd]

Tested on main. Seems to be a Metal-specific issue as the test passes with JLArrays.

using Metal, GPUArrays, Random, Test
begin
    AT = MtlArray
    a = AT(zeros(Float32, 1000,1000))
    b = AT(zeros(Float32, 1000,1000))

    rng = GPUArrays.default_rng(AT)

    Random.seed!(rng, 1234)
    s1 = copy(rng.state)
    rand!(rng, a)
    s2 = copy(rng.state)

    Random.seed!(rng, 1234)
    s3 = copy(rng.state)
    rand!(rng, b)
    s4 = copy(rng.state)

    @show s1 != s2
    @show s3 != s4
    @show s1 == s3
    @show s2 == s4

    @show collect(a) == collect(b)
end

[christiangnrd]

Ok this also affects CUDA.jl. Should I also submit this to CUDA.jl or is this a GPUArrays issue?

[maleadt]

If it's because of the GPUArrays RNG it is probably better to move it there.