syscall(linux): forkAndExecInChild

[luoliwoshang]

result

before use exec.command in linux
got

panic("todo: syscall.forkAndExecInChild")

now

root@be00d9b1c2c9:~/llgo/_demo/commandrun# llgo run .
len: 62
data: total 4
-rw-r--r-- 1 root root 258 Jan 16 10:52 commandrun.go

in the ci , use the exec.command to call nm tool,it can work normally.goplus/llcppg#160

resolution

• The libc wrapper's clone function cannot create processes with an empty child stack. Therefore, we need to directly use syscall to invoke the system's clone function instead of using libc's clone implementation.

• On Linux x86, when creating a process using clone with CLONE_VM flag (similar to Go's approach), it sets up memory sharing between parent and child processes. This can lead to unexpected parent process termination. Go addresses this on Linux x86 through assembly optimizations, allowing parent and child processes to share memory while ensuring that executing exec doesn't clear the shared stack, thus enabling normal execution.

question

• in ci, the demo will fail,but in my local docker env , can run success https://github.com/goplus/llgo/actions/runs/12826009988/job/35765102523?pr=959

other question(resolved)

in linux, go use clone to fork a child process,current is temp use fork to create a child process
src/syscall/exec_linux.go

	locked = true
	if clone3 != nil {
		pid, err1 = rawVforkSyscall(_SYS_clone3, uintptr(unsafe.Pointer(clone3)), unsafe.Sizeof(*clone3))
	} else {
		flags |= uintptr(SIGCHLD)
		if runtime.GOARCH == "s390x" {
			// On Linux/s390, the first two arguments of clone(2) are swapped.
			pid, err1 = rawVforkSyscall(SYS_CLONE, 0, flags)
		} else {
			pid, err1 = rawVforkSyscall(SYS_CLONE, flags, 0)
		}
	}

in above code,only pass two param to clone, but in the signature of c,have three params

int clone (int (*__fn) (void *__arg), void *__child_stack, int __flags, void *__arg, ...) ;

and if we only pass the flags & arg through call c.clone

err1 = Errno(os.Clone(nil, 0, c.Int(flags), 0))
fmt.Println(err1)
fmt.Println(Errno(os.Errno()))

// llgo:type C
type fn func(__arg c.Pointer) c.Int

//go:linkname Clone C.clone
func Clone(__fn fn, __childStack c.Pointer, __flags c.Int, __arg uintptr, __llgo_va_list ...any) c.Int

will got

Unknown error -1
Invalid argument

its error output like use clone without child_stack , got Invalid argument

int main() {
    const int STACK_SIZE = 1024 * 1024;
    void *stack = malloc(STACK_SIZE);
    if (!stack) {
        perror("malloc");
        return 1;
    }

    int flags = SIGCHLD;
    // int pid = clone(child_function, stack + STACK_SIZE, flags, NULL);  work normally
    int pid = clone(child_function, NULL, flags, NULL); // got clone: Invalid argument
    if (pid == -1) {
        perror("clone");
        free(stack);
        return 1;
    }

    printf("Parent process PID: %d, Child PID: %d\n", getpid(), pid);
    free(stack);
    return 0;
}

in the under asm code，it have some operate

// func rawVforkSyscall(trap, a1, a2 uintptr) (r1, err uintptr)
TEXT ·rawVforkSyscall(SB),NOSPLIT|NOFRAME,$0-40
	MOVQ	a1+8(FP), DI
	MOVQ	a2+16(FP), SI
	MOVQ	$0, DX
	MOVQ	$0, R10
	MOVQ	$0, R8
	MOVQ	$0, R9
	MOVQ	trap+0(FP), AX	// syscall entry
	POPQ	R12 // preserve return address
	SYSCALL
	PUSHQ	R12
	CMPQ	AX, $0xfffffffffffff001
	JLS	ok2
	MOVQ	$-1, r1+24(FP)
	NEGQ	AX
	MOVQ	AX, err+32(FP)
	RET
ok2:
	MOVQ	AX, r1+24(FP)
	MOVQ	$0, err+32(FP)

so in current resolution, use the syscall to call the clone instead the libc wrapper clone;
from #959 (comment) 's resolution

//go:linkname Syscall C.syscall
func Syscall(sysno c.Long, __llgo_va_list ...any) c.Long

ret := os.Syscall(syscall.SYS_CLONE, flags, 0)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.61%. Comparing base (e016e92) to head (1b536bf).
Report is 8 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #959      +/-   ##
==========================================
+ Coverage   81.58%   81.61%   +0.03%     
==========================================
  Files          53       53              
  Lines        9050     9067      +17     
==========================================
+ Hits         7383     7400      +17     
  Misses       1516     1516              
  Partials      151      151              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[MeteorsLiu]

The wrapper is accurate, the reason why the function is different from Go's is that libc wraps Clone in its way.

Raw Syscall may like(x86-64):

 long clone(unsigned long flags, void *stack,
                      int *parent_tid, int *child_tid,
                      unsigned long tls);

In libc, it may like:

 int clone(int (*fn)(void *_Nullable), void *stack, int flags,
                 void *_Nullable arg, ...  /* pid_t *_Nullable parent_tid,
                                              void *_Nullable tls,
                                              pid_t *_Nullable child_tid */ );

Acutally, the stack can be nullable, however, libc does a force verification for stack:

Code from gblic

ENTRY (__clone)
	cmp	r0, 0		/* @fn can't be NULL.  */
	and	r1,r1,-4	/* @child_stack be 4 bytes aligned per ABI.  */
	cmp.ne	r1, 0		/* @child_stack can't be NULL.  */
	bz	L (__sys_err)

In libc, stack cannot be nullable because of calling the fn.

When you don't need to call fn, the stack param can be zero.

[luoliwoshang]

In libc, stack cannot be nullable because of calling the fn.

When you don't need to call fn, the stack param can be zero.

Good investgation! We may use the c.syscall to call the clone function from core like go vForkSyscall , to avoid libc's wrapper check.

[luoliwoshang]

when excute clone with log output

println("Attempting clone with flags:", flags)
			ret := os.Syscall(syscall.SYS_CLONE, flags, 0)
			if ret >= 0 {
				if ret == 0 {
					println("clone succeeded, now in child")
				} else {
					println("clone succeeded now in parent")
				}
				pid = uintptr(ret)
				err1 = Errno(0)
			} else {
				pid = 0
				err1 = Errno(os.Errno())
				println("clone failed", err1)
			}

in my docker's linux will got clone succeeded now in parent & clone succeeded, now in child

VSCODE_IPC_HOOK_CLI=/tmp/vscode-ipc-07733c65-1a88-47ec-87d2-10a6579cd33d.sock
_=/root/go/bin/llgo
Attempting clone with flags: 16657
clone succeeded, now in child
Pass 1: look for fd[i] < i and move those up above len(fd)
Pass 2: dup fd[i] down onto i.
clone succeeded now in parent
len: 11
data: hello llgo

but in ci ,only got clone succeeded, now in child
https://github.com/goplus/llgo/actions/runs/12859660677/job/35850417459?pr=959

_=/home/runner/go/bin/llgo
OLDPWD=/home/runner/work/llgo/llgo
Attempting clone with flags: 16657
clone succeeded, now in child
Pass 1: look for fd[i] < i and move those up above len(fd)
Pass 2: dup fd[i] down onto i.
os.Execve(argv0, argv, envv) 2

[luoliwoshang]

when set the clone flag only use CLONE_VFORK , it can run normally

https://github.com/goplus/llgo/actions/runs/12862918395/job/35858617503?pr=959

The Go origin implement is:

flags |= CLONE_VFORK | CLONE_VM

llgo show use CLONE_VFORK only to clone normally

flags |= CLONE_VFORK

https://www.man7.org/linux/man-pages/man2/clone.2.html

In contrast to the glibc wrapper, the raw clone() system call
accepts NULL as a stack argument (and clone3() likewise allows
cl_args.stack to be NULL). In this case, the child uses a
duplicate of the parent's stack. (Copy-on-write semantics ensure
that the child gets separate copies of stack pages when either
process modifies the stack.) In this case, for correct
operation, the CLONE_VM option should not be specified. (If the
child shares the parent's memory because of the use of the
CLONE_VM flag, then no copy-on-write duplication occurs and chaos
is likely to result.)