Improvements to model performance by reducing allocations

[ConnectedSystems]

A swathe of changes to the simulations to improve runtime performance, principally by reducing the volume of allocations.

Includes purpose-specific weighted sum function for use when determining DHW tolerance thresholds.

Before:

After:

Error between approaches:

using Revise, Infiltrator

using ADRIA

using Statistics
using Serialization


RME_DOM = "C:/Users/tiwanaga/development/RME/rme_ml_2024_01_08"
gbr_dom = ADRIA.load_domain(RMEDomain, RME_DOM, "45")

global debug_c = 1
scen = ADRIA.param_table(gbr_dom)
rs = ADRIA.run_model(gbr_dom, scen[1, :]);

cover = dropdims(sum(rs.raw, dims=2), dims=2)
# serialize("cover_optimized.dat", cover)
# serialize("cover_unoptimized.dat", cover)

Error calculation was determined by serializing raw results to disk and then comparing the absolute maximum difference:

# Comparison

cover_opt = deserialize("cover_optimized.dat")
cover_unopt = deserialize("cover_unoptimized.dat")

maximum(abs.(cover_opt .- cover_unopt))

There are further performance optimizations potentially possible within CoralBlox and elsewhere in ADRIA, but seems to be mostly around determining strategies to avoid the GC.

We could even see if it is possible to make key functions completely allocation free and turning off the GC for those...

Update:

After some more tweaks

[Zapiano]

Just two questions

[Zapiano]

Is there any advantage of using view instead of the macro @view here? (I'm asking because the macro seems easier to read to me)

[ConnectedSystems]

Now I'm not sure which is better between view() and @view() (from what I understand the difference should be minimal?) but they both beat @views, at least when slicing a single matrix (size: 79x3806)

view()	@view()	@views

https://www.juliabloggers.com/the-view-and-views-macros-are-you-sure-you-know-how-they-work/

https://discourse.julialang.org/t/difference-between-view-and-view/40798

[Zapiano]

Would it make a relevant difference to cache this?

[ConnectedSystems]

The tricky thing is the number of sources can change.

But maybe there is a way of preallocating and then resizing. Maybe it will avoid excessive triggering of the GC... let me have a think.

[ConnectedSystems]

Couldn't think of a way to cache the weights matrix because the number of source locations change as I said, and making the cache matrix dynamic resize involved a few more intermediate steps which would increase overall allocations (at least the way I was thinking of it).

I did play with the idea that we can cache the result outside the function given the number of source locations are constant, but this is only true in our current case where we use mean connectivity. If/when we move to variable/dynamic connectivity data, then we'd need this implementation anyway.

I did think of some other tweaks which helped a tiny bit though: