SIGILL on some XEON CPU caused by prebuilt bin in Docker container

[heldrida]

Description

The Ursa process finishes with exit code 132 (interrupted by signal 4: SIGILL) on some XEON CPUs.

A Docker image is provided, prebuilt on an x86_64 DigitalOcean Intel vCPU machine, which is not supported by some Intel processors that VPS providers might supply.

A few examples of CPU are:

• Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz

Here are a few examples of VPS provider's client VPS where this was observed:

• https://cloudcone.com
• https://pq.hosting

As a temporary solution, the user is asked to build the image instead of accepting the recommended "latest" or "nightly". This means that cargo build enables or disables some CPU features by considering the native CPU in which the build is launched.

Environment

• We're providing a prebuilt Docker image, cargo built by a GitHub workflow
• On the prebuilt binary, run as a Docker container
• The CI runs Ubuntu's latest

Demo

As discussed and observed in: https://discord.com/channels/965698989464887386/1047823650541277194/1090335532669735053

Another user:
https://discord.com/channels/965698989464887386/1047823650541277194/1090966733130580008

For clarity, this has nothing to do with Docker. Here we see a binary built natively DO Intel Ubuntu latest x86_64, attempt run on the XEON E5 VPS:

Notes

Debugging details

Starting program: /usr/local/bin/ursa
warning: Error disabling address space randomization: Operation not permitted
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fb8b1615700 (LWP 627)]
[New Thread 0x7fb8b1414700 (LWP 628)]
[New Thread 0x7fb8b120d700 (LWP 629)]
[New Thread 0x7fb8b1009700 (LWP 630)]
  2023-03-30T18:47:59.665085Z  INFO ursa::config: Loading config from: "/root/.ursa/config.toml"
    at crates/ursa/src/config.rs:36

  2023-03-30T18:47:59.679817Z  INFO ursa::ursa::identity: Loaded identity `default` (12D3KooWSUuPHmNLj3z9TxjdHjHpEdb1Pr49cWbogdC1M6ZjH2r1)
    at crates/ursa/src/ursa/identity.rs:155

  2023-03-30T18:47:59.680409Z  INFO ursa_consensus::keys: Creating '"/root/.ursa/keystore/consensus/primary.key"' because it doesn't exists.
    at crates/ursa-consensus/src/keys.rs:43


Thread 1 "ursa" received signal SIGILL, Illegal instruction.
0x000056316b498a2f in ?? ()

Asssembly

multi-thre Thread 0x7fce3fdb63 In:               L??   PC: 0x5573c33eea2f
┌────────────────────────────────────────────────────────────────────────┐
│  >0x5573c33eea2f      mulx   %r14,%rax,%r11                            │
│   0x5573c33eea34      call   0x5573c33eeae0                            │
│   0x5573c33eea39      mov    0x8(%rsp),%r15                            │
│   0x5573c33eea3e      mov    0x10(%rsp),%r14                           │
│   0x5573c33eea43      mov    0x18(%rsp),%r13                           │
│   0x5573c33eea48      mov    0x20(%rsp),%r12                           │
│   0x5573c33eea4d      mov    0x28(%rsp),%rbx                           │
│   0x5573c33eea52      mov    0x30(%rsp),%rbp                           │
│   0x5573c33eea57      lea    0x38(%rsp),%rsp                           │
│   0x5573c33eea5c      repz ret                                         │
│   0x5573c33eea5e      xchg   %ax,%ax                                   │
│   0x5573c33eea60      endbr64                                          │
│   0x5573c33eea64      push   %rbp                                      │
│   0x5573c33eea65      push   %rbx                                      │
│   0x5573c33eea66      push   %r12                                      │
│   0x5573c33eea68      push   %r13                                      │
│   0x5573c33eea6a      push   %r14                                      │
│   0x5573c33eea6c      push   %r15                                      │
│   0x5573c33eea6e      sub    $0x8,%rsp                                 │
│   0x5573c33eea72      mov    %rsi,%rbx                                 │
│   0x5573c33eea75      mov    %rcx,%r8                                  │
│   0x5573c33eea78      mov    %rdx,%rcx                                 │
│   0x5573c33eea7b      mov    (%rsi),%rdx                               │
│   0x5573c33eea7e      mov    0x8(%rsi),%r15                            │
└────────────────────────────────────────────────────────────────────────┘
multi-thre Thread 0x7fce3fdb63 In:               L??   PC: 0x5573c33eea2f
(gdb)

A mulx described here https://docs.oracle.com/cd/E37838_01/html/E61064/gnyak.html seems to be a CPU feature not available on older intel CPU such like the XEON E5 family above.

Obs: A cargo build with target x86_64-unknown-linux-gnu was attempted, but that seems to be used by default when no --target is specified. This was tested in a Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz VPS shared by a community member " LinKong| Sui Global" (on Discord).

Checklist

• I have ensured that my version is up-to-date
• I have ensured that my issue is reproducible
• I have ensured that my issue is not a duplicate

[heldrida]

Another CPU where this happens, as reported by a different user.

🤖 CPU details
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 6
model name	: Common KVM processor
stepping	: 1
microcode	: 0x1
cpu MHz		: 2594.122
cache size	: 16384 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni cx16 x2apic hypervisor lahf_lm cpuid_fault pti
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips	: 5188.24
clflush size	: 64
cache_alignment	: 128
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 6
model name	: Common KVM processor
stepping	: 1
microcode	: 0x1
cpu MHz		: 2594.122
cache size	: 16384 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni cx16 x2apic hypervisor lahf_lm cpuid_fault pti
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips	: 5188.24
clflush size	: 64
cache_alignment	: 128
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 15
model		: 6
model name	: Common KVM processor
stepping	: 1
microcode	: 0x1
cpu MHz		: 2594.122
cache size	: 16384 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni cx16 x2apic hypervisor lahf_lm cpuid_fault pti
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips	: 5188.24
clflush size	: 64
cache_alignment	: 128
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 15
model		: 6
model name	: Common KVM processor
stepping	: 1
microcode	: 0x1
cpu MHz		: 2594.122
cache size	: 16384 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 3
initial apicid	: 3
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni cx16 x2apic hypervisor lahf_lm cpuid_fault pti
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips	: 5188.24
clflush size	: 64
cache_alignment	: 128
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 15
model		: 6
model name	: Common KVM processor
stepping	: 1
microcode	: 0x1
cpu MHz		: 2594.122
cache size	: 16384 KB
physical id	: 0
siblings	: 8
core id		: 4
cpu cores	: 8
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni cx16 x2apic hypervisor lahf_lm cpuid_fault pti
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips	: 5188.24
clflush size	: 64
cache_alignment	: 128
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 15
model		: 6
model name	: Common KVM processor
stepping	: 1
microcode	: 0x1
cpu MHz		: 2594.122
cache size	: 16384 KB
physical id	: 0
siblings	: 8
core id		: 5
cpu cores	: 8
apicid		: 5
initial apicid	: 5
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni cx16 x2apic hypervisor lahf_lm cpuid_fault pti
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips	: 5188.24
clflush size	: 64
cache_alignment	: 128
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 15
model		: 6
model name	: Common KVM processor
stepping	: 1
microcode	: 0x1
cpu MHz		: 2594.122
cache size	: 16384 KB
physical id	: 0
siblings	: 8
core id		: 6
cpu cores	: 8
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni cx16 x2apic hypervisor lahf_lm cpuid_fault pti
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips	: 5188.24
clflush size	: 64
cache_alignment	: 128
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 15
model		: 6
model name	: Common KVM processor
stepping	: 1
microcode	: 0x1
cpu MHz		: 2594.122
cache size	: 16384 KB
physical id	: 0
siblings	: 8
core id		: 7
cpu cores	: 8
apicid		: 7
initial apicid	: 7
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni cx16 x2apic hypervisor lahf_lm cpuid_fault pti
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips	: 5188.24
clflush size	: 64
cache_alignment	: 128
address sizes	: 40 bits physical, 48 bits virtual
power management:

[heldrida]

Another user report

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 62
model name	: Intel(R) Xeon(R) CPU E5-2403 v2 @ 1.80GHz
stepping	: 4
microcode	: 0x42e
cpu MHz		: 1799.979
cache size	: 10240 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm arat pln pts md_clear flush_l1d
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips	: 3599.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 62
model name	: Intel(R) Xeon(R) CPU E5-2403 v2 @ 1.80GHz
stepping	: 4
microcode	: 0x42e
cpu MHz		: 1799.970
cache size	: 10240 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm arat pln pts md_clear flush_l1d
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips	: 3599.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 62
model name	: Intel(R) Xeon(R) CPU E5-2403 v2 @ 1.80GHz
stepping	: 4
microcode	: 0x42e
cpu MHz		: 1799.975
cache size	: 10240 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm arat pln pts md_clear flush_l1d
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips	: 3599.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 62
model name	: Intel(R) Xeon(R) CPU E5-2403 v2 @ 1.80GHz
stepping	: 4
microcode	: 0x42e
cpu MHz		: 1799.972
cache size	: 10240 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm arat pln pts md_clear flush_l1d
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips	: 3599.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management: