SIMD support in the vector library

[Magalame]

Hi!

I was wondering if there were any plan to add support for SIMD instructions to vector now that GHC supports them even without llvm?

Thanks to @Drezil and @klapaucius I'm aware of this paper and this work, but the branch is 6 years old and so probably doesn't reflect the current direction of vector.

If there's any need for help, I'd be more than happy to contribute!

[Magalame]

Came from lehins/massiv#80

[chessai]

as far as i am aware, that patch you link to doesn't actually work as intended. @bgamari and @AndreasPK would be able to say why, if so.

[andrewthad]

Here's the commit that reverted NCG SIMD support: https://gitlab.haskell.org/ghc/ghc/commit/db948daea6c01c073f8d09a79fa5adda279fbf0c

[Magalame]

Thanks @andrewthad for the link! Do we know when it'll be added back?

[chessai]

@Magalame i know the two individuals that plan on implementing it/supervising its implementation are Ben Gamari and Andreas Klebinger.

[bgamari]

@AndreasPK has indeed expressed interest in picking it up. In short, the commit as written has an overly-simplistic view of register allocation and breaks in some cases.

[cartazio]

More broadly : the way fusion based optimization as realize in vector vs the way simd performance engineering work mean they will seldom happen together. Broadly : even once ghc has better simd support, the only way to get robust performance out of it will be via hand coded routines. The way vector fusion works means any auto simd inclusion in vector will not actually be a compelling case for simd ... I’m open to being wrong, but the optimization architecture of vector and the patterns for General simd performance are very different. Plus there’s some very precise constraints on when simd supports what datatypes. I will say this: maintaining a hypothetical future simd vector as a quality default would require orders of magnitude more contributors being comfortable improving and contributing to the internals of vector design / fusion architecture etc wise. Barring that , even when ghc gets good simd, there won’t be enough contributor bandwidth to maintain that massive portability and performance liability. I’m oversimplifying this massively but I do agree it’s be an interesting direction. Just one that’s actually incredibly subtle and challenging to do well. Let alone maintain one support!

…

On Tue, Sep 24, 2019 at 5:23 PM Ben Gamari ***@***.***> wrote: @AndreasPK <https://github.com/AndreasPK> has indeed expressed interest in picking it up. In short, the commit as written has an overly-simplistic view of register allocation and breaks in some cases. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#251?email_source=notifications&email_token=AAABBQSENCLE7OATH354AVLQLKAMDA5CNFSM4H7ESAZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7P3H2A#issuecomment-534754280>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAABBQVK3VHTNSBHROJN3E3QLKAMDANCNFSM4H7ESAZQ> .

[Magalame]

@bgamari @AndreasPK thanks a lot for the work you're putting into this. Do you have a general idea of when it would be available?

[Magalame]

Thanks @cartazio for the details, I thought the MultisC type would have ensured a relatively seamless integration?

[cartazio]

it worked as a technology demonstrator, but simd as it currently exists in the forseeable future is incredible monomorphic, only really for vectors on Float/Double/Int32/Int64/Word32/Word64, so it would only pay off for a pretty spare/small fraction of user code.

It is possible that a future version of vector could allow simd integration to the fusion framework, but given engineering priorities for the future of vector as they currently exist, unless we can jointly improve the current O2 compilation performance AND fusion quality first, its not going to happen ever. Adding another rewrite rule layer to the current setup seems like piling architecture debt.

that aside, not enough compiler support in the GHC layer actually exists to robustly support simd across all supported targets. And since vector is so widely used, that might be a blocker to including it in core vector as a library until conditional compilation / distinct code paths per microarchtecture is sanely supported at the GHC layer.

if you want to experiment rebasing the multiC change set onto vector master or friends, so we can look at it in an up to date way, i'm all ears. We just genuinely dont have the time/resources/bandwidth though. And i'm skeptical that its the right architecture going forward

[Magalame]

I see, thanks so much for taking the time to answer!

[cartazio]

I appreciate your interest! It’s just a tricky area and evaluating user benefit / impact is super challenging.

…

On Sat, Oct 12, 2019 at 8:59 AM Magalame ***@***.***> wrote: I see, thanks so much for taking the time to answer! — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#251?email_source=notifications&email_token=AAABBQVGWSIRFSQ3QO67NMLQOHC4FA5CNFSM4H7ESAZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBB64KQ#issuecomment-541322794>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAABBQXOJPFUXTSDUVN2UYTQOHC4FANCNFSM4H7ESAZQ> .

[GeorgeCo]

I assume that if you compile with llvm you will get SIMD instructions when applicable. Is that false?

[cartazio]

If you disable fusion and write imperative code that can be loopified, simd might happen. But streams are kinda at odds at least the simpler loads and stores that I think some autovectorization can see easily. But I haven’t tried out stuff with ghc and modern llvm so I could be wrong.

…

On Fri, Jun 23, 2023 at 8:03 AM GeorgeCo ***@***.***> wrote: I assume that if you compile with llvm you will get SIMD instructions when applicable. Is that false? — Reply to this email directly, view it on GitHub <#251 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAABBQQHTZUY3WDVDQTRLTDXMWAYXANCNFSM4H7ESAZQ> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[Drezil]

Depends on the operations you do.
vector is basically an implementation of the "recycle your arrays"-paper. It has streaming-capabilities AND block/buffer ones. What gets done is ruled by compiler optimisations and if the generated llvm looks code looks optimizable for libllvm than those things get turned into simd operations. But it is hit and miss - as far as I remember.
It has been years when I last looked into it .. so it may have changed in the meantime.

[cartazio]

I’ve not seen that paper before, is there a good link? But yeah, you’re absolutely right that it will sometimes fire. But also that unless you’re mentally modelling the generated core that’s passed to llvm you’re not gonna have anything reliable with the current stuff

…

On Sat, Jun 24, 2023 at 9:39 AM Drezil ***@***.***> wrote: Depends on the operations you do. vector is basically an implementation of the "recycle your arrays"-paper. It has streaming-capabilities AND block/buffer ones. What gets done is ruled by compiler optimisations and if the generated llvm looks code looks optimizable for libllvm than those things get turned into simd operations. But it is hit and miss - as far as I remember. It has been years when I last looked into it .. so it may have changed in the meantime. — Reply to this email directly, view it on GitHub <#251 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAABBQSVHTWWNLNHZ4Y2BJDXM3UXJANCNFSM4H7ESAZQ> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>