From 9003f9b183327df80fda97aa82dfc8054e1d3dce Mon Sep 17 00:00:00 2001 From: Pascal Ernster Date: Sun, 5 Nov 2023 10:35:24 +0100 Subject: [PATCH] Fix AVX detection The old/faulty code would try to use AVX/AVX2 if either the SSE bit or the AVX bit were set in XCR0, but did not check if both bits were set. In most cases, this still worked, but on some machines, enabling linux kernel mitigations for the "gather data sampling" vulnerability results in only the SSE bit but not the AVX bit being set, thus resulting in an illegal instruction and crashing the application. Fix this by checking that both bits are set. Fixes: 4bbb590bd047e7d877bf9cc9218ee525ceca1a9b ("Proper check of CPU's AVX2 feature support (with MSVC support)") Signed-off-by: Pascal Ernster --- lib/codec_choose.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/codec_choose.c b/lib/codec_choose.c index ae4e5b5a..abef3f2a 100644 --- a/lib/codec_choose.c +++ b/lib/codec_choose.c @@ -208,7 +208,7 @@ codec_choose_x86 (struct codec *codec) if (ecx & bit_XSAVE_XRSTORE) { uint64_t xcr_mask; xcr_mask = _xgetbv(_XCR_XFEATURE_ENABLED_MASK); - if (xcr_mask & _XCR_XMM_AND_YMM_STATE_ENABLED_BY_OS) { + if ((xcr_mask & _XCR_XMM_AND_YMM_STATE_ENABLED_BY_OS) == _XCR_XMM_AND_YMM_STATE_ENABLED_BY_OS) { // check multiple bits at once #if HAVE_AVX512 if (max_level >= 7) { __cpuid_count(7, 0, eax, ebx, ecx, edx);