CPU Feature Detection 2

[martindevans]

The new CUDA autodetection and NativeLibraryConfig lost one major feature of the previous loading system that it replaced - automatic detection of AVX levels. Reintroduced that by probing support in the NativeLibraryConfig constructor.

I also added in support for AVX512 even on dotnet versions that don't support it! Autodetection won't detect AVX512 except on dotnet 8.0, but you can manually specify it yourself by calling WithAvx(AvxLevel.Avx512).

A few other minor changes to the NativeLibraryConfig system while I was there:

• Renamed NativeLibraryConfig.Default to NativeLibraryConfig.Instance. I don't think Default makes sense since it's mutable (so it's not the default after you've changed it)!
• using Lazy<T> to initialize it automatically, just a little cleaner than doing it with a lock but basically the same thing.
• Some spelling and grammar fixes in docs.

Regression

I noticed that there's probably a (minor, ish) regression in the current 0.8.0 release for CPU only inference. Previously the default libllama.dll required AVX2 support, now the system uses no SIMD at all for the default version. That makes sense, we get wider support for older platforms. However 0.8.0 didn't ship any of the more advanced binaries for CPU, so this is probably a big slowdown for pure CPU inference!

@AsakusaRinne can we aim to get a 0.8.1 patch out next weekend with all of the various SIMD binaries for Windows and Linux? I can get started on a PR for that tomorrow if so :)

[AsakusaRinne]

@AsakusaRinne can we aim to get a 0.8.1 patch out next weekend with all of the various SIMD binaries for Windows and Linux? I can get started on a PR for that tomorrow if so :)

Yes, I think it's a good idea. 😄

For the regression you mentioned, I think there's probably no regression compared to 0.7.0 because the libllama in NuGet package is still the one with avx2. However, yes, it will be a problem for users with netstandard2.0 if we split libraries into different avx levels.