Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add split lock #140

Merged
merged 3 commits into from
Nov 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions splitlock/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
*.o
sl_test
12 changes: 12 additions & 0 deletions splitlock/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# SPDX-License-Identifier: GPL-2.0-only
# Copyright (c) 2022 Intel Corporation.

BIN := sl_test

all: $(BIN)

sl_test: sl_test.c
gcc $^ -o $@

clean:
rm -rf $(BIN) *.o
29 changes: 29 additions & 0 deletions splitlock/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# SPLIT LOCK

## Description
A split lock is any atomic operation whose operand crosses two cache lines.
Since the operand spans two cache lines and the operation must be atomic,
the system locks the bus while the CPU accesses the two cache lines.

A bus lock is acquired through either split locked access to writeback (WB)
memory or any locked access to non-WB memory. This is typically thousands of
cycles slower than an atomic operation within a cache line. It also disrupts
performance on other cores and brings the whole system to its knees.

## Usage
make
./sl_test
1. Linux kernel driver will export the split_lock_detect flag to /proc/cpuinfo
if hardware supports this feature.
2. Test if the locked instruction will trigger #AC. If no split_lock_detect is
not set by default or it is split_lock_detect=warn, #AC will be triggered.
The boot parameter could also be set as fatal or ratelimit(ratelimit:1 max
ratelimit:1000). If split_lock_detect=fatal, split lock test will output
"Caught SIGBUS/#AC" in console.
For more, please refer to handle_bus_lock in arch/x86/kernel/cpu/intel.c.
Examples of exception in dmesg:
x86/split lock detection: #AC: sl_test/4354 took a split_lock trap at address: 0x401231
x86/split lock detection: #DB: sl_test/5137 took a bus_lock trap at address: 0x4011f5

## Expected result
All test results should show pass, no fail.
121 changes: 121 additions & 0 deletions splitlock/sl_test.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
// SPDX-License-Identifier: GPL-2.0-only
// Copyright (c) 2022 Intel Corporation.

/*
* Split lock test app to check if the locked instruction will trigger #AC
*/

#define _GNU_SOURCE
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <stdlib.h>
#include <signal.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <time.h>
#include <sys/time.h>
#include <stdio.h>

void catch_sigbus(int sig)
{
printf("Caught SIGBUS/#AC due to split locked access\n");

exit(-1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confirm the exit code -1 or 0 ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

catch_sigbus is a void function, it is to handle user space SIGBUS which is sent by kernel. For test purpose, print out a message is enough. To mark it is abnormal, use non zero is OK, whatever it is 1 or -1.

}

/*
* Atomically add 1 to *iptr using "lock addl" instruction.
* Since *iptr crosses two cache lines, #AC is generated by the split lock.
*/
void do_split_locked_inst(int *iptr)
{
/*
* The distance between iptr and next cache line is 3 bytes.
* Operand size in "addl" is 4 bytes. So iptr will span two cache
* lines. "lock addl" instruction will trigger #AC in hardware
* and kernel either delivers SIGBUS to this process or re-execute
* the instruction depending on
* /sys/kernel/debug/x86/split_lock/user_mode setting.
*/
asm volatile ("lock addl $1, %0\n\t"
: "=m" (*iptr));
}

/*
* Test SIGBUS delivered after a lock instruction generates #AC for split lock
* operand *iptr.
*/
void test_sigbus(int *iptr)
{
pid_t pid;

pid = fork();
if (pid)
return;

/*
* The locked instruction will trigger #AC and kernel will deliver
* SIGBUS to this process. The SIGBUS handler in this process will
* verify that the signal is delivered and the process is killed then.
*/
do_split_locked_inst(iptr);
}

int main(int argc, char **argv)
{
int *iptr;
char *cptr;
char *line = NULL;
int ret = 0;
size_t len;
FILE *fp;

fp = fopen("/proc/cpuinfo", "r");
if (!fp)
return 1;

while (getline(&line, &len, fp) != -1) {
if (!strncmp(line, "flags", 5)) {
if (strstr(line, "split_lock_detect"))
ret = 1;
break;
}
}

free(line);
fclose(fp);
if (!ret) {
printf("split_lock_dect: [FAIL]\n");
return !ret;
}

signal(SIGBUS, catch_sigbus);

/*
* Enable Alignment Checking on x86_64.
* This will generate alignment check on not only split lock but also
* on any misalignment.
* Turn on this for reference only.
*/
/* __asm__("pushf\norl $0x40000,(%rsp)\npopf"); */

/* aligned_alloc() provides 64-byte aligned memory */
cptr = (char *)aligned_alloc(64, 128);

/*
* Increment the pointer by 61, making it 3 bytes away from the next
* cache line and 4-byte *iptr across two cache line.
*/
iptr = (int *)(cptr + 61);

test_sigbus(iptr);

free(cptr);

printf("Split Lock test - exits normally\n");

return 0;
}
70 changes: 70 additions & 0 deletions splitlock/split_lock.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
#!/bin/bash
# SPDX-License-Identifier: GPL-2.0-only
# Copyright (c) 2023 Intel Corporation
# Description: Test script for split lock of CPU

cd "$(dirname "$0")" 2>/dev/null && source ../.env

: "${CASE_NAME:="check_cpu_info"}"

usage() {
cat <<__EOF
usage: ./${0##*/} [-t TESTCASE_ID] [-H]
-t TEST CASE ID
-H show this
__EOF
}

# Check this feature is enabled/supported
# Return: 0 for true, otherwise false
check_cpu_info() {
local cpu_func="split_lock_detect"
grep -q "$cpu_func" /proc/cpuinfo || block_test "CPU not support:$cpu_func"
test_print_trc "/proc/cpuinfo contain '$cpu_func'"
}

# Call split lock test and check dmesg if #AC is triggered
split_lock_ac() {
local ac_dmesg_start=$(dmesg | grep "x86/split lock detection: #" | wc -l)
sl_test
local ac_dmesg_end=$(dmesg | grep "x86/split lock detection: #" | wc -l)
grep -q "split_lock_detect=fatal" /proc/cmdline && block_test "If set is fatal won't trigger #AC"
if [[ $ac_dmesg_end -gt $ac_dmesg_start ]];then
test_print_trc "split_lock_dect: [PASS]"
else
die "split_lock_dect: [FAIL]"
fi
}

split_lock_test() {
case $TEST_SCENARIO in
sl_on_default)
check_cpu_info
;;
check_ac_dmesg)
split_lock_ac
;;
esac
return 0
}

while getopts :t:H arg; do
case $arg in
t)
TEST_SCENARIO=$OPTARG
;;
H)
usage && exit 0
;;
\?)
usage
die "Invalid Option -$OPTARG"
;;
:)
usage
die "Option -$OPTARG requires an argument."
;;
esac
done

split_lock_test
5 changes: 5 additions & 0 deletions splitlock/tests
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# This file collects the split lock feature tests on
# Intel® Architecture-based platforms.

split_lock.sh -t sl_on_default
split_lock.sh -t check_ac_dmesg