Should we add support stow directory is symbolic link.

[cuonglm]

Should we support the issue raised at Unix & Linux Stack Exchange: "symlink - Can GNU Stow use a stow directory that is a symbolic link?" ?

It seems that former Stow developers had this in mind, base on the comment in canon_path

EDIT: Clarification from @aspiers:

• This issue is about the Stow directory being a symlink.
• Support package trees containing absolute symlinks #51 is about package trees containing absolute symlinks.
• Create symlinks with absolute path names #3 is about the symlinks which Stow manages in the target directory being absolute symlinks.

[cuonglm]

@aspiers

[aspiers]

I've already seen this, but currently too busy to investigate, sorry. Hopefully I'll have some free time soon.

[liloman]

Hi,

I resolved this issue this way:

Before

stow -vS htop 
WARNING! stowing htop would cause conflicts:
  * existing target is not owned by stow: .config
All operations aborted.

I figure out that the problem was that the target dir (~/.config in my case) was a symlink. So you need to specify the realpath in your script.

After

#get real target dir without symlinks
r=$(realpath $HOME/.config) && target=${r%/*} || echo "Error!!"
stow -vS htop -t $target
LINK: .config/htop/htoprc => ../../../../../home/charly/dotfiles/htop/.config/htop/htoprc 

Cheers

[aspiers]

Suggestions on how to fix stow to handle this natively are welcome. I haven't had time to think about it.

[cuonglm]

@aspiers After finding stow code base, I see getcwd() always be used. Then we maybe face with complex problem, where both source and dest are symlink.

The change also break previous package which was stowed.

[liloman]

I'll fix it if I knew perl. :)

Anyway I ended up getting rid off that "wrong" symbolic link making a "right" bind mount.

[dmarcoux]

It would be great to have this feature.

Stowing a file which is an absolute symbolic link should be less complex, right? I'm wondering as in the raised issue on StackOverflow, it is about a directory which is an absolute symbolic link. Right now, it is impossible to stow files which are absolute symbolic links. It happens here: https://github.com/aspiers/stow/blob/master/lib/Stow.pm.in

[aspiers]

Please can somebody give details of a real world example where this feature would be desirable? I understand the request, I just don't understand why there'd be a burning need for it.

@dmarcoux Pretty much all the logic happens in Stow.pm.in :-)

[dmarcoux]

Personally, I would like to stow some symlinks which are symlinks to /dev/null. For example, I symlinked .mysql_history to /dev/null as I don't want to keep MySQL's history.

For the logic in Stow.pm.in, I didn't realize at the time that almost everything happens there... The file is HUGE! I would love to help on this, but I don't know Perl and to be totally honest, I'm not interested in learning Perl.

[aspiers]

Ah, /dev/null is an interesting use case - thanks! Is that the history for the mysql CLI client? Why would that file exist within your Stow package directory?

Yes Stow.pm.in is huge, and yes that sucks ;-) I inherited the codebase, and whilst I did a lot of refactoring early on, splitting it into more manageable chunks is not something I've had time for :-/

I totally understand your lack of interest in Perl. IMHO it's basically a dead language - it was amazingly innovative in the 1990s and early 2000s, and it allowed me to do many things well and have a good career, but more recently Ruby and Python have far surpassed it. If someone sent a pull request rewriting Stow in Ruby or Python I would welcome it with open arms ... but I doubt that will happen, and unfortunately I don't have time to do it myself currently :-/

[dmarcoux]

Exactly, .mysql_history keeps the history of the MySQL CLI client. I do have a few of these symlinks to /dev/null. By having these symlinks in my Stow package directory, I can easily stow all of them without having to do it manually and it's easier to have a big picture of what is symlinked.

As for rewriting Stow to something like Ruby or Python, this would be great. Would you consider only Ruby and Python as a language to rewrite Stow? Perhaps, Go would be a good choice. It is great for CLI applications. I wonder if someone like me without any Perl experience could rewrite Stow in something else... It cannot be that complex.

[cuonglm]

@aspiers @dmarcoux I am not sure the problem with file symlink is cover in this issue.

The issue here is handling the case the stowed directory is symlink.

[aspiers]

Exactly, .mysql_history keeps the history of the MySQL CLI client. I do have a few of these symlinks to /dev/null.

Why more than one? Multiple versions of the client software?

By having these symlinks in my Stow package directory, I can easily stow all of them without having to do it manually and it's easier to have a big picture of what is symlinked.

I'm slightly confused by this. Presumably .mysql_history isn't included in the distribution? So what difference does it make whether you set up those symlinks within the Stow package directory, or directly within ~ ? The latter would have the advantage of being required only once, rather than once per client version (assuming you have multiple versions, but it sounds like I'm missing some key information here).

That said, I think @Gnouc is right that this is a separate issue.

As for rewriting Stow to something like Ruby or Python, this would be great. Would you consider only Ruby and Python as a language to rewrite Stow?

Personally, yes, because I don't currently have enough spare time to learn another language well. (I wish I did!)

Perhaps, Go would be a good choice. It is great for CLI applications.

Sorry, I can't agree with that. Go seems far too low-level for this, and in fact for CLI applications in general, unless they require significant performance and/or parallelism. I think it would simply make the codebase needlessly verbose. Stow only deals with the filesystem, and is not multi-threaded or performance-critical.

I wonder if someone like me without any Perl experience could rewrite Stow in something else...

Sure, why not?

It cannot be that complex.

It really isn't :-) And the existing test suite is comprehensive enough to help ensure a rewrite would go smoothly.

[dmarcoux]

I only have one .mysql_history symlink to /dev/null. I have the same for .less_history and .bash_history for example.

For the language choice, I'm totally fine with Ruby. It's the language I know the most too. Your points are completely valid. Go is indeed maybe a bit too low-level for this. It is nice to distribute the binary only though.

[aspiers]

On 13 June 2016 at 11:20, Dany Marcoux [email protected] wrote:

I only have one .mysql_history symlink to /dev/null.

So what is the advantage of putting this symlink within the Stow package
directory, rather than directly in $HOME? I'm still a bit confused.

[dmarcoux]

My Stow package directory is basically my dotfiles, so I could easily stow the home directory and avoid having a specific workflow for these symlinks, just like anything else in my dotfiles. All advantages of using Stow over manual copy-paste is what I would have too with the possibility to symlink files being symlinks (or directories for this specific GitHub issue).

[aspiers]

Ohhh, OK - now I get it! Thanks :-) This is one of the ways I use Stow too, so I entirely understand. To clarify, are you saying that stowing a link to /dev/null doesn't currently work? Or that it stows it as an ugly relative link?

[dmarcoux]

It doesn't work. Here's the output of stow home in my dotfiles directory (the Stow package directory):

WARNING! stowing home would cause conflicts:
  * source is an absolute symlink dotfiles/home/.bash_history => /dev/null
  * source is an absolute symlink dotfiles/home/.lesshst => /dev/null
  * source is an absolute symlink dotfiles/home/.mysql_history => /dev/null
All operations aborted.

In my $HOME directory, I don't have any of these files which I'm trying to symlink with Stow. I would expect the command stow home to stow these files in $HOME just like any file.

[aspiers]

Thanks, that info helps a lot. I agree this should work (although the target should link back to the source which then links to /dev/null, rather than the target linking directly to /dev/null).

[CristianCantoro]

Here's my use case for this feature.
I don't use full-disk encryption on my laptop at the moment (I have used it in the past and I plan to use it in the future again, but I want to set it up properly so it's not a hassle), but I like to keep my ~/.gnupg folder on a removable drive which is encrypted. I also have a collection of dotfiles that includes a gpg.conf in a gpg package.

So this is my current situation:

~ $ ls -lahrt | grep gnupg               
lrwxrwxrwx  1 cristian cristian   46 gen 31 22:29 .gnupg -> /my/encrypted/drive/path/.gnupg

If I try to stow the package gpg, I get the following:

~/dotfiles $ stow gpg                                       
WARNING! stowing gpg would cause conflicts:
  * existing target is not owned by stow: .gnupg
All operations aborted.

whereas I would like that stow would just symlynk

~/.gnupg/gpg.conf -> ~/.dotfiles/clusterssh/.gnupg/gpg.conf

Note that if .gnupg was a regular directory I would get exactly that (within the .gnupg):

~/.gnupg $ ls -lh   
lrwxrwxrwx 1 cristian cristian 32 gen 31 22:25 gpg.conf -> ../.dotfiles/gpg/.gnupg/gpg.conf

The workaround at the moment is going within the package gpg:

$ ~/dotfiles/gpg $ stow -t ~/.gnupg .gnupg 

which works, but it is obviously less convenient.

[carlosnewmusic]

with the pages of manuals gives error also when they are located in /man/man1

[aspiers]

OK I've read through this issue more carefully now, and @cuonglm is right that we're confusing two totally separate issues:

@cuonglm commented on June 13, 2016 9:29 AM:

@aspiers @dmarcoux I am not sure the problem with file symlink is cover in this issue.

The issue here is handling the case the stowed directory is symlink.

The original issue reported (excellently described on Unix & Linux Stack Exchange) is that it's not possible for the Stow directory (i.e. the parent of the Stow package directories, see Terminology) to be a symlink. The example given has the Stow directory as a relative symlink. I guess some people might want it to be an absolute symlink instead, but I don't know because so far noone's provided a real world use case where this would actually be useful.

In contrast, @dmarcoux is asking for Stow to support stowing of package trees which contain absolute symlinks, i.e. where the end result would be a (relative) symlink in the target directory pointing to an absolute symlink in the package tree which points to something like /dev/null. This is totally different to the original request, so I've submitted #51 - please continue the conversation there.

@CristianCantoro your use case is different again to either of the above! IIUC, you want Stow to be less cautious and stow symlinks to normal files inside target directories which are symlinks not owned by Stow. In this case, your target directory is ~, and you want the .gnupg/gpg.conf file in the dotfiles package to be stowed inside ~/.gnupg even though ~/.gnupg is an absolute symlink not owned by Stow. Even if it was a relative symlink not owned by Stow, Stow would not be able to perform tree folding on it, so it cowardly quits. It might be feasible for it to proceed with folding disabled, but I'm not sure it's worth the effort. Have you considered just bind-mounting the .gnupg directory on your removable drive onto ~/.gnupg instead? That would eliminate the symlink and probably make it transparent to Stow. But if you really want this considered as a feature request, please can you file a separate issue? Thanks!

@carlosnewmusic Sorry I don't understand. Is this definitely related to the original report above regarding the Stow directory being a symlink? If not, please file a separate issue, thanks. But either way I definitely need more details.

[nbeaver]

As per request, I will describe an example use case where resolving the canonical path is inconvenient.

Symbolic links are sometimes used for compatibility. Debian even talks about this in official policy. Often the target is a single file, but sometimes it is a directory. I happen to have a few hundred on my system in /usr/share/doc/ alone:

$ find /usr/share/doc -xtype d -type l | wc -l
325

The default behavior of stow is fine as long as the symlink target doesn't get moved. But sometimes the desired targeted directory does get moved. For example, on Debian, the vim-runtime package installs files under /usr/share/vim/ in a directory that depends on the version, such as /usr/share/vim/vim64 for version 6.4. However, the package would also update a symlink at /usr/share/vim/vimcurrent that pointed to the current version. This means that a symlink pointing to, say

/usr/share/vim/vim64/doc/cmdline.txt

would break when the next release of Debian upgraded it to

/usr/share/vim/vim70/doc/cmdline.txt

but a symlink to

/usr/share/vim/vimcurrent/doc/cmdline.txt

would work in both versions.

Since stow uses the absolute canonical path of the stow directory, an invocation like

stow --dir=/usr/share/vim/vimcurrent --target=./my-vim-docs doc

would result in symbolic links like, e.g. this:

$ file cmdline.txt
cmdline.txt: symbolic link to ../../../../../usr/share/vim/vim64/doc/cmdline.txt

not like this:

$ file cmdline.txt
cmdline.txt: symbolic link to ../../../../../usr/share/vim/vimcurrent/doc/cmdline.txt

(The motivation for using stow on vimcurrent/docs is to be able to mingle my own vim notes alongside symlinks to the current documentation.) Note that the vimcurrent compatibility symlink is no longer present in current Debian distributions, though it may be in others like Arch Linux; I'm not sure. In any case, here's a script that gives the general idea for vim documentation:

#! /usr/bin/env bash
mkdir -p target
ln --symbolic /usr/share/vim/vim80 vimcurrent
stow --verbose --dir=./vimcurrent --target=./target pack
file target/dist

Output is:

LINK: dist => ../../../../../usr/share/vim/vim80/pack/dist
target/dist: symbolic link to ../../../../../usr/share/vim/vim80/pack/dist

Hypothetically, stow could have a flag called, say, --no-realpath, so the output would look something like this instead:

LINK: dist => ./vimcurrent/pack/dist
target/dist: symbolic link to ./vimcurrent/pack/dist

For other examples of compatibility symlinks that change with each version, here are two more I know of on my laptop:

$ file /usr/share/go
/usr/share/go: symbolic link to go-1.10
$ file /usr/share/mscore
/usr/share/mscore: symbolic link to mscore-2.1

To address the symlink-points-to-symlink case:

#! /usr/bin/env bash
mkdir -p target
mkdir -p mydir/package/
touch mydir/package/file
ln --symbolic mydir mylink
ln --symbolic mylink mylink2
namei mylink2

produces:

f: mylink2
 l mylink2 -> mylink
   l mylink -> mydir
     d mydir

and then:

$ stow --verbose --dir=./mylink2 --target=./target package
$ file target/file

produces:

LINK: file => ../mydir/package/file
target/file: symbolic link to ../mydir/package/file

whereas

$ stow --no-realpath --verbose --dir=./mylink2 --target=./target package
$ file target/file

would produce this:

LINK: file => ../mylink2/package/file
target/file: symbolic link to ../mylink2/package/file

So in the hypothetical --no-realpath behavior it would treat the stow directory as a regular directory. This feature would be applicable in a scenario where

1. the stow directory has to be a symlink, and

2. it is desirable to preserve that link in the generated symlinks.

While I don't consider the lack of this feature a great deficiency of stow, I hope that this example clarifies the potential usefulness of not always resolving canonical paths.

[carlosnewmusic]

OK I've read through this issue more carefully now, and @cuonglm is right that we're confusing two totally separate issues:

@cuonglm commented on June 13, 2016 9:29 AM:

@aspiers @dmarcoux I am not sure the problem with file symlink is cover in this issue.
The issue here is handling the case the stowed directory is symlink.

The original issue reported (excellently described on Unix & Linux Stack Exchange) is that it's not possible for the Stow directory (i.e. the parent of the Stow package directories, see Terminology) to be a symlink. The example given has the Stow directory as a relative symlink. I guess some people might want it to be an absolute symlink instead, but I don't know because so far noone's provided a real world use case where this would actually be useful.

In contrast, @dmarcoux is asking for Stow to support stowing of package trees which contain absolute symlinks, i.e. where the end result would be a (relative) symlink in the target directory pointing to an absolute symlink in the package tree which points to something like /dev/null. This is totally different to the original request, so I've submitted #51 - please continue the conversation there.

@CristianCantoro your use case is different again to either of the above! IIUC, you want Stow to be less cautious and stow symlinks to normal files inside target directories which are symlinks not owned by Stow. In this case, your target directory is ~, and you want the .gnupg/gpg.conf file in the dotfiles package to be stowed inside ~/.gnupg even though ~/.gnupg is an absolute symlink not owned by Stow. Even if it was a relative symlink not owned by Stow, Stow would not be able to perform tree folding on it, so it cowardly quits. It might be feasible for it to proceed with folding disabled, but I'm not sure it's worth the effort. Have you considered just bind-mounting the .gnupg directory on your removable drive onto ~/.gnupg instead? That would eliminate the symlink and probably make it transparent to Stow. But if you really want this considered as a feature request, please can you file a separate issue? Thanks!

@carlosnewmusic Sorry I don't understand. Is this definitely related to the original report above regarding the Stow directory being a symlink? If not, please file a separate issue, thanks. But either way I definitely need more details.

I think so, what data do you need?

[aspiers]

@nbeaver Many thanks for the extra info. I don't have time to process this right now, but I'll try to get to it soon.

[aspiers]

@carlosnewmusic commented on July 1, 2019 10:11 AM:

@carlosnewmusic Sorry I don't understand. Is this definitely related to the original report above regarding the Stow directory being a symlink? If not, please file a separate issue, thanks. But either way I definitely need more details.

I think so, what data do you need?

@carlosnewmusic I need to understand the concrete use case where you need the stow directory to be a symlink. Please give details of the exact paths of:

• the Stow directory (which must be a symlink, otherwise it's not relevant to this issue and you should submit a separate one)
• the path the Stow directory symlink points to (and whether it's absolute or relative)
• the target directory

and also:

• why you need the Stow directory to be a symlink
• the full stow command(s) you tried, and their full output (please use the -v option)

Thanks!

[soredake]

@aspiers i use https://github.com/graysky2/profile-sync-daemon and my .mozilla/firefox/profile folder i symlink to /run/user/1000/bausch-firefox-tor-main

[quiescer]

My use case is that I have a single git repository with dotfiles for a whole bunch of machines. I want to be able to check out that directory to a computer and then have ~/stow be a symlink to ~/dotfiles/$HOSTNAME.

I can of course achieve this by wrapping stow with a script (or a shell function) that invokes stow with the -t parameter, but it'd be nice if there were an easier way.