Using compiler flags / performance annotations to speed-up Oceananigans

[tomchor]

Has anyone experimented with using compiler flags or performance annotations with Oceananigans?

Reading about some of these flags, it seems to me that some of them might be useful for our purposes (for example ignoring subnormals and compensated arithmetic), but I haven't heard anything about them here. Even math-mode=fast (or its (safer?) annotation version @fastmath) sounds kinda useful.

The trade-off is a decrease (often major decrease) in accuracy. A great reference to the dangers of these optimizations can be found in Simon Byrne's notes.

Reading some pages online (including questions and answers in discourse, etc.), it seems the consensus is that these shouldn't be used in applications where accuracy is paramount (like if you're calculating a physical constant), but can be helpful in situations when accuracy isn't specially important (like machine learning).

Thinking about the chaotic nature of the Navier-Stokes equations, it feels that these optimizations might be useful for Oceananigans. (I should say I don't know if the same conclusions apply for both CPU and GPU here...)

Has anyone played around with that? Or has some insight to share?

[glwagner]

I believe that some of these annotations are enforced by KernelAbstractions during code transformation, so we should check over there to see what's being done.

Definitely agree that optimizations like "fast math" are ok for CFD!