Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DenseMetric and Component arrays (solve #344) #345

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

DenseMetric and Component arrays (solve #344) #345

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
dbb5d16
correct length check for phase point
erathorn Aug 3, 2023
18dc7f0
fix typo
erathorn Aug 3, 2023
449a589
DenseMetric and Comoponent Arrays
erathorn Aug 3, 2023
7c58524
Update test/hamiltonian.jl
erathorn Aug 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions src/hamiltonian.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,16 +42,19 @@ end
∂H∂r(h::Hamiltonian{<:UnitEuclideanMetric,<:GaussianKinetic}, r::AbstractVecOrMat) = copy(r)
∂H∂r(h::Hamiltonian{<:DiagEuclideanMetric,<:GaussianKinetic}, r::AbstractVecOrMat) =
torfjelde marked this conversation as resolved.
Show resolved Hide resolved
h.metric.M⁻¹ .* r
∂H∂r(h::Hamiltonian{<:DenseEuclideanMetric,<:GaussianKinetic}, r::AbstractVecOrMat) =
h.metric.M⁻¹ * r
function ∂H∂r(h::Hamiltonian{<:DenseEuclideanMetric,<:GaussianKinetic}, r::AbstractVecOrMat)
out = similar(r) # Make sure the output of this function is of the same type as r
mul!(out, h.metric.M⁻¹, r)
out
end
Comment on lines +45 to +49
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit uncertain about this change as it "complicates" code to mainly just stay compatible with ComponentArrays.jl, and thus I'd be more in favour of just making it an extension instead, I think 😕 Then in the extension, we just overload whatever we need to be compatible.

Also, will this code break if, say, h.metric.M⁻¹ has eltype Float64 but r has eltype Float32, rather than just promoting, as is current behavior?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

p1 = ComponentArray(m=one(Float32), s = one(Float32))
r = similar(p1)
M = diagm(randn(Float64, 2))
mul!(r, M, p1)

This works on my machine, and returns r as a component array of eltype Float32 as expected.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AHMC supports vectorised sampling, when passing arguments in a suitable type. In this case, r::AbstractVecOrMat could be a single momentum realization or a vector of momentum realizations. Therefore, the new code needs to be able to handle the vectorized sampling mode for the tests to pass.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the silence. Thank you for the suggestion, it totally makes sense to me. However, I looked into this a bit more and am honestly slightly lost. The call to the rand function, which fails in the tests only works in the test case. Calling this function in a plain Julia session fails for me (on the main branch). A brute force solution, which dispatches on r::AbtractVecOrMat{AbstractVecOrMat}, does unfortunately not do the trick either.


struct PhasePoint{T<:AbstractVecOrMat{<:AbstractFloat},V<:DualValue}
θ::T # Position variables / model parameters.
r::T # Momentum variables
ℓπ::V # Cached neg potential energy for the current θ.
ℓκ::V # Cached neg kinect energy for the current r.
function PhasePoint(θ::T, r::T, ℓπ::V, ℓκ::V) where {T,V}
@argcheck length(θ) == length(r) == length(ℓπ.gradient) == length(ℓπ.gradient)
@argcheck length(θ) == length(r) == length(ℓπ.gradient) == length(ℓκ.gradient)
if any(isfinite.((θ, r, ℓπ, ℓκ)) .== false)
# @warn "The current proposal will be rejected due to numerical error(s)." isfinite.((θ, r, ℓπ, ℓκ))
# NOTE eltype has to be inlined to avoid type stability issue; see #267
Expand Down
35 changes: 34 additions & 1 deletion test/hamiltonian.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
using ReTest, AdvancedHMC
using AdvancedHMC: GaussianKinetic, DualValue, PhasePoint
using LinearAlgebra: dot, diagm
using ComponentArrays

@testset "Hamiltonian" begin
f = x -> dot(x, x)
Expand Down Expand Up @@ -38,7 +39,7 @@ end
end
end

@testset "Metric" begin
@testset "Metric Base Array" begin
n_tests = 10

for T in [Float32, Float64]
Expand All @@ -49,18 +50,50 @@ end
h = Hamiltonian(UnitEuclideanMetric(T, D), ℓπ, ∂ℓπ∂θ)
@test -AdvancedHMC.neg_energy(h, r_init, θ_init) == sum(abs2, r_init) / 2
@test AdvancedHMC.∂H∂r(h, r_init) == r_init
@test typeof(AdvancedHMC.∂H∂r(h, r_init)) == typeof(r_init)

M⁻¹ = ones(T, D) + abs.(randn(T, D))
h = Hamiltonian(DiagEuclideanMetric(M⁻¹), ℓπ, ∂ℓπ∂θ)
@test -AdvancedHMC.neg_energy(h, r_init, θ_init) ≈
r_init' * diagm(0 => M⁻¹) * r_init / 2
@test AdvancedHMC.∂H∂r(h, r_init) == M⁻¹ .* r_init
@test typeof(AdvancedHMC.∂H∂r(h, r_init)) == typeof(r_init)

m = randn(T, D, D)
M⁻¹ = m' * m
h = Hamiltonian(DenseEuclideanMetric(M⁻¹), ℓπ, ∂ℓπ∂θ)
@test -AdvancedHMC.neg_energy(h, r_init, θ_init) ≈ r_init' * M⁻¹ * r_init / 2
@test AdvancedHMC.∂H∂r(h, r_init) == M⁻¹ * r_init
@test typeof(AdvancedHMC.∂H∂r(h, r_init)) == typeof(r_init)
end
end
end

@testset "Metric ComponentArrays" begin
n_tests = 10
for T in [Float32, Float64]
for _ = 1:n_tests
θ_init = ComponentArray(a = randn(T, D), b = randn(T, D))
r_init = ComponentArray(a = randn(T, D), b = randn(T, D))

h = Hamiltonian(UnitEuclideanMetric(T, 2*D), ℓπ, ∂ℓπ∂θ)
@test -AdvancedHMC.neg_energy(h, r_init, θ_init) == sum(abs2, r_init) / 2
@test AdvancedHMC.∂H∂r(h, r_init) == r_init
@test typeof(AdvancedHMC.∂H∂r(h, r_init)) == typeof(r_init)

M⁻¹ = ones(T, 2*D) + abs.(randn(T, 2*D))
h = Hamiltonian(DiagEuclideanMetric(M⁻¹), ℓπ, ∂ℓπ∂θ)
@test -AdvancedHMC.neg_energy(h, r_init, θ_init) ≈
r_init' * diagm(0 => M⁻¹) * r_init / 2
@test AdvancedHMC.∂H∂r(h, r_init) == M⁻¹ .* r_init
@test typeof(AdvancedHMC.∂H∂r(h, r_init)) == typeof(r_init)

m = randn(T, 2*D, 2*D)
M⁻¹ = m' * m
h = Hamiltonian(DenseEuclideanMetric(M⁻¹), ℓπ, ∂ℓπ∂θ)
@test -AdvancedHMC.neg_energy(h, r_init, θ_init) ≈ r_init' * M⁻¹ * r_init / 2
@test all(AdvancedHMC.∂H∂r(h, r_init) .== M⁻¹ * r_init)
@test typeof(AdvancedHMC.∂H∂r(h, r_init)) == typeof(r_init)
end
end
end