Gradient of pdf of beta distribution errors. Missing rule?

[mattiasvillani]

Not sure if this is a bug, or a missing feature yet to be implemented.
I am on Julia 1.10.0-beta2+0.x64.linux.gnu
within the following environment:

Status ~/Dropbox/Study/Julia/AutoDiff/Project.toml
[6e4b80f9] BenchmarkTools v1.3.2
⌃ [31c24e10] Distributions v0.25.45
[f6369f11] ForwardDiff v0.10.36
[5087d2c0] IncGammaBeta v0.1.0
⌅ [276daf66] SpecialFunctions v0.8.0

Gradient of pdf for Beta distribution (wrt to the parameters of the distribution) works in ForwardDiff.jl, but not in Diffractor.jl:
gradient(ForwardDiffBackend(), θ -> pdf(Beta(θ[1],θ[2]), 0.4), [0.3, 0.8]) # works gradient(DiffractorForwardBackend(), θ -> pdf(Beta(θ[1],θ[2]), 0.4), [0.3, 0.8]) # errors
The error message is given at the end of this text.

For the normal pdf, both packages work however, so the suggestion in the error message of a missing rule (for the Beta function?) is probably correct:
gradient(ForwardDiffBackend(), θ -> pdf(Normal(θ[1],θ[2]), 0.4), [0.3, 0.8]) # works gradient(DiffractorForwardBackend(), θ -> pdf(Normal(θ[1],θ[2]), 0.4), [0.3, 0.8]) # works

---

ERROR MESSAGE from the command that errored above:

ERROR: Attempted to AD a foreigncall. Missing rule?
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:35
[2] betapdf
@ Diffractor ~/.julia/packages/StatsFuns/P7Mci/src/rmath.jl:70 [inlined]
[3] (::Diffractor.∂☆recurse{1})(::ZeroBundle{1, typeof(StatsFuns.RFunctions.betapdf)}, ::Diffractor.TangentBundle{1, Float64, Diffractor.TaylorTangent{…}}, ::Diffractor.TangentBundle{1, Float64, Diffractor.TaylorTangent{…}}, ::ZeroBundle{1, Float64})
@ Diffractor ~/.julia/packages/Diffractor/vzpvE/src/stage1/recurse_fwd.jl:0
[4] (::Diffractor.∂☆internal{1})(::ZeroBundle{1, typeof(StatsFuns.RFunctions.betapdf)}, ::Vararg{Diffractor.AbstractTangentBundle{1}})
@ Diffractor ~/.julia/packages/Diffractor/vzpvE/src/stage1/forward.jl:121
[5] (::Diffractor.∂☆{1})(::ZeroBundle{1, typeof(StatsFuns.RFunctions.betapdf)}, ::Vararg{Diffractor.AbstractTangentBundle{1}})
@ Diffractor ~/.julia/packages/Diffractor/vzpvE/src/stage1/forward.jl:160
[6] pdf
@ Diffractor ~/.julia/packages/Distributions/9Albf/src/univariates.jl:619 [inlined]
[7] (::Diffractor.∂☆recurse{1})(::ZeroBundle{1, typeof(pdf)}, ::Diffractor.CompositeBundle{1, Beta{Float64}, Tuple{Diffractor.TangentBundle{…}, Diffractor.TangentBundle{…}}}, ::ZeroBundle{1, Float64})
@ Diffractor ~/.julia/packages/Diffractor/vzpvE/src/stage1/recurse_fwd.jl:0
[8] (::Diffractor.∂☆internal{1})(::ZeroBundle{1, typeof(pdf)}, ::Vararg{Diffractor.AbstractTangentBundle{1}})
@ Diffractor ~/.julia/packages/Diffractor/vzpvE/src/stage1/forward.jl:121
[9] (::Diffractor.∂☆{1})(::ZeroBundle{1, typeof(pdf)}, ::Vararg{Diffractor.AbstractTangentBundle{1}})
@ Diffractor ~/.julia/packages/Diffractor/vzpvE/src/stage1/forward.jl:160
[10] #49
@ Diffractor ./REPL[51]:1 [inlined]
[11] (::Diffractor.∂☆recurse{1})(::ZeroBundle{1, var"#49#50"}, ::Diffractor.TangentBundle{1, Vector{Float64}, Diffractor.TaylorTangent{Tuple{Vector{Float64}}}})
@ Diffractor ~/.julia/packages/Diffractor/vzpvE/src/stage1/recurse_fwd.jl:0
[12] (::Diffractor.∂☆internal{1})(::ZeroBundle{1, var"#49#50"}, ::Vararg{Diffractor.AbstractTangentBundle{1}})
@ Diffractor ~/.julia/packages/Diffractor/vzpvE/src/stage1/forward.jl:121
[13] (::Diffractor.∂☆{1})(::ZeroBundle{1, var"#49#50"}, ::Vararg{Diffractor.AbstractTangentBundle{1}})
@ Diffractor ~/.julia/packages/Diffractor/vzpvE/src/stage1/forward.jl:160
[14] (::Diffractor.var"#pushforward#359"{var"#49#50", Tuple{Vector{Float64}}})(vs::Tuple{Vector{Float64}})
@ Diffractor ~/.julia/packages/Diffractor/vzpvE/src/AbstractDifferentiation.jl:13
[15] (::Diffractor.var"#358#362"{Diffractor.var"#pushforward#359"{var"#49#50", Tuple{Vector{Float64}}}})(cols::Vector{Float64})
@ Diffractor ~/.julia/packages/AbstractDifferentiation/eEkWP/src/AbstractDifferentiation.jl:526
[16] mapslices(f::Diffractor.var"#358#362"{Diffractor.var"#pushforward#359"{var"#49#50", Tuple{Vector{Float64}}}}, A::Matrix{Float64}; dims::Int64)
@ Base ./abstractarray.jl:3179
[17] jacobian(b::DiffractorForwardBackend, f::Function, args::Vector{Float64})
@ Diffractor ~/.julia/packages/AbstractDifferentiation/eEkWP/src/AbstractDifferentiation.jl:524
[18] gradient(ab::DiffractorForwardBackend, f::Function, xs::Vector{Float64})
@ AbstractDifferentiation ~/.julia/packages/AbstractDifferentiation/eEkWP/src/AbstractDifferentiation.jl:43
[19] top-level scope
@ REPL[51]:1
Some type information was truncated. Use show(err) to see complete types.

[oxinabox]

yes. So this means we hit a ccall to libRMath
we can't AD into compiled external code.
So StatsFuns.jl would need to add a ChainRulesCore.frule (or a @scalar_rule) for StatsFuns.RFunctions.betapdfStatsFuns does have [bunch of these](https://github.com/JuliaStats/StatsFuns.jl/blob/master/ext/StatsFunsChainRulesCoreExt.jl) but not one forbetapdf`

I suggest openning an issue or PR on StatsFuns.jl. we can't fix it here.

[mattiasvillani]

Ok, thanks. Will not open an issue, however, since they already have rule for the logpdf, which is what one typically uses. I came to the pdf case via the case that I am really interested, AD for the cdf of a beta distribution.