Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot mount overlay: Invalid argument #242

Closed
sergei opened this issue Jan 16, 2017 · 12 comments
Closed

Cannot mount overlay: Invalid argument #242

sergei opened this issue Jan 16, 2017 · 12 comments
Assignees
Labels
Bug Darling emulation behaves unexpectedly Container The emulation container is configured incorrectly Discussion Relating to Darling Project strategy

Comments

@sergei
Copy link

sergei commented Jan 16, 2017

Getting the following error:

$ darling shell 
Cannot mount overlay: Invalid argument

OS Ubuntu 16.04 running under Virtual Box

$ uname -srvmpio
Linux 4.4.0-59-generic #80-Ubuntu SMP Fri Jan 6 17:47:47 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
$ strace darling shell ls
execve("/usr/local/bin/darling", ["darling", "shell", "ls"], [/* 17 vars */]) = 0
brk(NULL)                               = 0x2405000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb423545000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/usr/local/lib//darling/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/local/lib//darling/tls/x86_64", 0x7ffd9882af60) = -1 ENOENT (No such file or directory)
open("/usr/local/lib//darling/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/local/lib//darling/tls", 0x7ffd9882af60) = -1 ENOENT (No such file or directory)
open("/usr/local/lib//darling/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/local/lib//darling/x86_64", 0x7ffd9882af60) = -1 ENOENT (No such file or directory)
open("/usr/local/lib//darling/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/local/lib//darling", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=110778, ...}) = 0
mmap(NULL, 110778, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fb423529000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\t\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1864888, ...}) = 0
mmap(NULL, 3967488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fb422f59000
mprotect(0x7fb423119000, 2093056, PROT_NONE) = 0
mmap(0x7fb423318000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1bf000) = 0x7fb423318000
mmap(0x7fb42331e000, 14848, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fb42331e000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb423528000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb423527000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb423526000
arch_prctl(ARCH_SET_FS, 0x7fb423527700) = 0
mprotect(0x7fb423318000, 16384, PROT_READ) = 0
mprotect(0x604000, 4096, PROT_READ)     = 0
mprotect(0x7fb423547000, 4096, PROT_READ) = 0
munmap(0x7fb423529000, 110778)          = 0
geteuid()                               = 0
brk(NULL)                               = 0x2405000
brk(0x2426000)                          = 0x2426000
open("/proc/modules", O_RDONLY)         = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(3, "overlay 49152 0 - Live 0xfffffff"..., 1024) = 1024
close(3)                                = 0
getuid()                                = 0
getgid()                                = 0
stat("/home/sergei/.darling", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/home/sergei/.darling", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/home/sergei/.darling/.init.pid", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=4, ...}) = 0
read(3, "2351", 4096)                   = 4
read(3, "", 4096)                       = 0
close(3)                                = 0
kill(2351, SIG_0)                       = -1 ESRCH (No such process)
unlink("/home/sergei/.darling/.init.pid") = 0
stat("/home/sergei/.darling.workdir", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
pipe([3, 4])                            = 0
unshare(CLONE_NEWPID)                   = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fb4235279d0) = 2359
close(4)                                = 0
read(3, Cannot mount overlay: Invalid argument
"", 1)                          = 0
close(3)                                = 0
setresuid(-1, 0, -1)                    = 0
setresgid(-1, 0, -1)                    = 0
open("/home/sergei/.darling/.init.pid", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
setresuid(-1, 0, -1)                    = 0
setresgid(-1, 0, -1)                    = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
write(3, "2359", 4)                     = 4
close(3)                                = 0
getcwd("/home/sergei", 4096)      = 19
open("/proc/2359/ns/pid", O_RDONLY)     = 3
setns(3, CLONE_NEWPID)                  = 0
close(3)                                = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fb4235279d0) = 2360
setresuid(-1, 0, -1)                    = 0
wait4(2360, [{WIFSIGNALED(s) && WTERMSIG(s) == SIGKILL}], 0, NULL) = 2360
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=2360, si_uid=0, si_status=SIGKILL, si_utime=0, si_stime=0} ---
exit_group(9)                           = ?
+++ exited with 9 +++
@ahyattdev ahyattdev added Bug Darling emulation behaves unexpectedly Container The emulation container is configured incorrectly labels Jan 16, 2017
@bugaevc
Copy link
Member

bugaevc commented Jan 17, 2017

Please attach the output of sudo strace -f -u $USER darling shell (-f is for trace children, because the mounts are set up by prelaunchd).

@bugaevc bugaevc self-assigned this Jan 17, 2017
@sergei
Copy link
Author

sergei commented Jan 17, 2017

Here you go:

$ sudo strace -f -u $USER darling shell 
execve("/usr/local/bin/darling", ["darling", "shell"], [/* 17 vars */]) = 0
brk(NULL)                               = 0x1937000
fcntl(0, F_GETFD)                       = 0
fcntl(1, F_GETFD)                       = 0
fcntl(2, F_GETFD)                       = 0
access("/etc/suid-debug", F_OK)         = -1 ENOENT (No such file or directory)
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3f72050000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/usr/local/lib//darling/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/local/lib//darling/tls/x86_64", 0x7ffcd26be730) = -1 ENOENT (No such file or directory)
open("/usr/local/lib//darling/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/local/lib//darling/tls", 0x7ffcd26be730) = -1 ENOENT (No such file or directory)
open("/usr/local/lib//darling/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/local/lib//darling/x86_64", 0x7ffcd26be730) = -1 ENOENT (No such file or directory)
open("/usr/local/lib//darling/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/usr/local/lib//darling", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=110778, ...}) = 0
mmap(NULL, 110778, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f3f72034000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\t\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1864888, ...}) = 0
mmap(NULL, 3967488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3f71a64000
mprotect(0x7f3f71c24000, 2093056, PROT_NONE) = 0
mmap(0x7f3f71e23000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1bf000) = 0x7f3f71e23000
mmap(0x7f3f71e29000, 14848, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3f71e29000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3f72033000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3f72032000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3f72031000
arch_prctl(ARCH_SET_FS, 0x7f3f72032700) = 0
mprotect(0x7f3f71e23000, 16384, PROT_READ) = 0
mprotect(0x604000, 4096, PROT_READ)     = 0
mprotect(0x7f3f72052000, 4096, PROT_READ) = 0
munmap(0x7f3f72034000, 110778)          = 0
geteuid()                               = 0
brk(NULL)                               = 0x1937000
brk(0x1958000)                          = 0x1958000
open("/proc/modules", O_RDONLY)         = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(3, "overlay 49152 0 - Live 0x0000000"..., 1024) = 1024
close(3)                                = 0
getuid()                                = 1000
getgid()                                = 1000
stat("/home/sergei/.darling", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/home/sergei/.darling", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/home/sergei/.darling/.init.pid", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0664, st_size=4, ...}) = 0
read(3, "2540", 4096)                   = 4
read(3, "", 4096)                       = 0
close(3)                                = 0
kill(2540, SIG_0)                       = -1 ESRCH (No such process)
unlink("/home/sergei/.darling/.init.pid") = 0
stat("/home/sergei/.darling.workdir", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
pipe([3, 4])                            = 0
unshare(CLONE_NEWPID)                   = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f3f720329d0) = 7708
close(4)                                = 0
read(3, strace: Process 7708 attached
 
[pid  7708] close(3)                    = 0
[pid  7708] unshare(CLONE_NEWNS)        = 0
[pid  7708] mount(NULL, "/", NULL, MS_REC|MS_SLAVE, NULL) = 0
[pid  7708] mount("overlay", "/home/sergei/.darling", "overlay", 0, "lowerdir=/usr/local/libexec/darl"...) = -1 EINVAL (Invalid argument)
[pid  7708] write(2, "Cannot mount overlay: Invalid ar"..., 39Cannot mount overlay: Invalid argument
) = 39
[pid  7708] exit_group(1)               = ?
[pid  7707] <... read resumed> "", 1)   = 0
[pid  7707] close(3)                    = 0
[pid  7707] setresuid(-1, 1000, -1)     = 0
[pid  7707] setresgid(-1, 1000, -1)     = 0
[pid  7707] open("/home/sergei/.darling/.init.pid", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
[pid  7707] setresuid(-1, 0, -1)        = 0
[pid  7707] setresgid(-1, 0, -1)        = 0
[pid  7707] fstat(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
[pid  7707] write(3, "7708", 4)         = 4
[pid  7707] close(3)                    = 0
[pid  7707] getcwd("/home/sergei", 4096) = 19
[pid  7707] open("/proc/7708/ns/pid", O_RDONLY) = 3
[pid  7707] setns(3, CLONE_NEWPID)      = 0
[pid  7707] close(3)                    = 0
[pid  7707] clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f3f720329d0) = 7709
[pid  7707] setresuid(-1, 1000, -1)     = 0
[pid  7707] wait4(7709, strace: Exit of unknown pid 7709 ignored
[{WIFSIGNALED(s) && WTERMSIG(s) == SIGKILL}], 0, NULL) = 7709
[pid  7707] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=7709, si_uid=1000, si_status=SIGKILL, si_utime=0, si_stime=0} ---
[pid  7707] exit_group(9)               = ?
[pid  7707] +++ exited with 9 +++
+++ exited with 1 +++

@bugaevc
Copy link
Member

bugaevc commented Jan 18, 2017

I cannot reproduce this on the same setup (Ubuntu 16.04 running under Virtual Box; clean installation of both Ubuntu and Darling).

If you had a version of Darling installed before the introduction of the Darling container, you may need to remove all leftover files (in /usr/local/libexec/darling , /usr/local/lib), remove your prefix dir, and then reinstall Darling and let it recreate the prefix. I'm not sure this is the cause of your problem though.

Are you able to perform the same mount normally, ex. from shell?

sudo mount overlay -t overlay /home/sergei/.darling -o lowerdir=/usr/local/libexec/darling,upperdir=/home/sergei/.darling,workdir=/home/sergei/.darling.workdir

@sergei
Copy link
Author

sergei commented Jan 19, 2017

Thanks, must be something wrong with my file system.

mount: wrong fs type, bad option, bad superblock on overlay,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.
sergei@sergei-VirtualBox:~$ dmesg | tail
[ 1019.876159] overlayfs: filesystem on '/home/sergei/.darling' not supported as upperdir

I wonder why wouldn't it be supported, it's just ext4

/dev/sda1 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)

Anyway doesn't appear to be the darling problem, so I would close the issue.

@sergei sergei closed this as completed Jan 19, 2017
@ahyattdev
Copy link
Member

ahyattdev commented Jan 19, 2017

Another cause of this error on Ubuntu is having an encrypted home folder.

@bugaevc
Copy link
Member

bugaevc commented Jan 19, 2017

Do you have an encrypted home folder by any chance? <..> overlayfs doesn't support them.

I'm assuming this is because it needs d_type, which "only some filesystems (among them: Btrfs, ext2, ext3, and ext4) have full support for".

We don't have a lot of options if we want this to work — either to patch ecryptfs and get it upstreamed, or to abandon overlayfs to use mergerfs, or even roll our own darlingfs (that would solve some other problems too).

@bugaevc
Copy link
Member

bugaevc commented Jan 19, 2017

You should still be able to use prefixes under e.g. /tmp, but they won't obviously persist across reboots.

The next step is to update README saying we don't support "encrypted home folder" at the moment.

bugaevc added a commit that referenced this issue Jan 19, 2017
Copied from the website and updated. Also mention that we don't support encrypted home dir (#242)
@LubosD
Copy link
Member

LubosD commented Jan 19, 2017

I was thinking about developing "icasefs" just to solve the case sensitivity problem, but I guess things are more complicated than that.

"Fun" fact: it is impossible to unmount fuse mounts (e.g. mounted DMGs or mergerfs) if you mount them under an NFS4 directory, because fusermount is suid root and root cannot access other users' NFS4 mounts.

@bugaevc
Copy link
Member

bugaevc commented Jan 19, 2017

I was thinking about developing "icasefs" just to solve the case sensitivity problem, but I guess things are more complicated than that.

I was thinking case-insensitivity, /Volumes (#220) and top-level folder ownership faking. And then maybe Mac-like /dev.

By the way, we should bind-mount, not symlink, things like /Users and /dev

@bugaevc bugaevc added the Discussion Relating to Darling Project strategy label Jan 19, 2017
@bugaevc
Copy link
Member

bugaevc commented Jan 19, 2017

"Fun" fact: it is impossible to unmount fuse mounts (e.g. mounted DMGs or mergerfs) if you mount them under an NFS4 directory, because fusermount is suid root and root cannot access other users' NFS4 mounts.

Same for sshfs, IIRC. Really stupid.

@sergei
Copy link
Author

sergei commented Jan 19, 2017

Yes indeed. I removed the encryption and it runs just fine.
Since people are seemingly paying attention to this thread do you know if darling would run in the Docker container? Has anyone tried it? That essentially what I need: to run the OSX command line binary in Docker container.

@bugaevc bugaevc mentioned this issue Feb 21, 2018
@prhbrt
Copy link

prhbrt commented Jun 2, 2023

You should still be able to use prefixes under e.g. /tmp, but they won't obviously persist across reboots.

The next step is to update README saying we don't support "encrypted home folder" at the moment.

If an encrypted home directory is detectable, maybe also mention it in the error message?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Darling emulation behaves unexpectedly Container The emulation container is configured incorrectly Discussion Relating to Darling Project strategy
Projects
None yet
Development

No branches or pull requests

5 participants