Test Failure TestCheckAndFixLongFilename

[edisj]

Expected behavior

test passed

Actual behavior

test failed

__________________________________________________ TestCheckAndFixLongFilename.test_symlink_file __________________________________________________
[gw1] win32 -- Python 3.8.3 C:\Users\16235\Anaconda3\envs\mda3develop\python.exe

self = <MDAnalysisTests.analysis.test_hole2.TestCheckAndFixLongFilename object at 0x000001C9AFCA5EB0>
tmpdir = local('C:\\Users\\16235\\AppData\\Local\\Temp\\pytest-of-16235\\pytest-0\\popen-gw1\\test_symlink_file0')

    @pytest.mark.skipif(os.name == 'nt' and sys.maxsize <= 2**32,
                        reason="OSError: symbolic link privilege not held")
    def test_symlink_file(self, tmpdir):
        long_name = 'a'*10 + self.filename

        with tmpdir.as_cwd():
>           fixed = check_and_fix_long_filename(long_name)

analysis\test_hole2.py:98:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

filename = 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.pdb', tmpdir = '.', max_length = 70, make_symlink = True

    def check_and_fix_long_filename(filename, tmpdir=os.path.curdir,
                                    max_length=70,
                                    make_symlink=True):
        """Return a file name suitable for HOLE.

        HOLE is limited to filenames <= ``max_length``. This method

        1. returns `filename` if HOLE can process it
        2. returns a relative path (see :func:`os.path.relpath`) if that shortens the
            path sufficiently
        3. creates a symlink to `filename` (:func:`os.symlink`) in a safe temporary
            directory and returns the path of the symlink.

        Parameters
        ----------
        filename : str
            file name to be processed
        tmpdir : str, optional
            By default the temporary directory is created inside the current
            directory in order to keep that path name short. This can be changed
            with the `tmpdir` keyword (e.g. one can use "/tmp"). The default is
            the current directory :data:`os.path.curdir`.

        Returns
        -------
        str
            path to the file that has a length less than
            ``max_length``

        Raises
        ------
        RuntimeError
            If none of the tricks for filename shortening worked. In this case,
            manually rename the file or recompile your version of HOLE.
        """

        if len(filename) <= max_length:
            return filename

        msg = ('HOLE will not read {} '
               'because it has more than {} characters.')
        logger.debug(msg.format(filename, max_length))

        # try a relative path
        new_name = os.path.relpath(filename)
        if len(new_name) <= max_length:
            msg = 'Using relative path: {} -> {}'
            logger.debug(msg.format(filename, new_name))
            return new_name

        if make_symlink:
            # shorten path by creating a symlink inside a safe temp dir
            _, ext = os.path.splitext(filename)
            dirname = tempfile.mkdtemp(dir=tmpdir)
            newname = os.path.join(dirname, os.path.basename(filename))
            if len(newname) > max_length:
                fd, newname = tempfile.mkstemp(suffix=ext, dir=dirname)
                os.close(fd)
                os.unlink(newname)

            if len(newname) > max_length:
                newname = os.path.relpath(newname)

            if len(newname) <= max_length:
>               os.symlink(filename, newname)
E               OSError: [WinError 1314] A required privilege is not held by the client: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.pdb' -> 'tmp3h6whzso\\tmpovk7hecc.pdb'

..\..\package\MDAnalysis\analysis\hole2\utils.py:142: OSError
__________________________________________________ TestCheckAndFixLongFilename.test_symlink_dir ___________________________________________________
[gw0] win32 -- Python 3.8.3 C:\Users\16235\Anaconda3\envs\mda3develop\python.exe

self = <MDAnalysisTests.analysis.test_hole2.TestCheckAndFixLongFilename object at 0x000001D2FF9FFE80>
tmpdir = local('C:\\Users\\16235\\AppData\\Local\\Temp\\pytest-of-16235\\pytest-0\\popen-gw0\\test_symlink_dir0')

    @pytest.mark.skipif(os.name == 'nt' and sys.maxsize <= 2**32,
                        reason="FileNotFoundError on Win 32-bit")
    def test_symlink_dir(self, tmpdir):
        dirname = 'really_'*20 +'long_name'
        short_name = self.filename[-20:]
        path = os.path.join(dirname, short_name)
        with tmpdir.as_cwd():
            os.makedirs(dirname)
            u = mda.Universe(PDB_HOLE)
            u.atoms.write(path)

>           fixed = check_and_fix_long_filename(path)

analysis\test_hole2.py:88:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

filename = 'really_really_really_really_really_really_really_really_really_really_really_really_really_really_really_really_really_really_really_really_long_name\\aaaaaaaaaaaaaaaa.pdb'
tmpdir = '.', max_length = 70, make_symlink = True

    def check_and_fix_long_filename(filename, tmpdir=os.path.curdir,
                                    max_length=70,
                                    make_symlink=True):
        """Return a file name suitable for HOLE.

        HOLE is limited to filenames <= ``max_length``. This method

        1. returns `filename` if HOLE can process it
        2. returns a relative path (see :func:`os.path.relpath`) if that shortens the
            path sufficiently
        3. creates a symlink to `filename` (:func:`os.symlink`) in a safe temporary
            directory and returns the path of the symlink.

        Parameters
        ----------
        filename : str
            file name to be processed
        tmpdir : str, optional
            By default the temporary directory is created inside the current
            directory in order to keep that path name short. This can be changed
            with the `tmpdir` keyword (e.g. one can use "/tmp"). The default is
            the current directory :data:`os.path.curdir`.

        Returns
        -------
        str
            path to the file that has a length less than
            ``max_length``

        Raises
        ------
        RuntimeError
            If none of the tricks for filename shortening worked. In this case,
            manually rename the file or recompile your version of HOLE.
        """

        if len(filename) <= max_length:
            return filename

        msg = ('HOLE will not read {} '
               'because it has more than {} characters.')
        logger.debug(msg.format(filename, max_length))

        # try a relative path
        new_name = os.path.relpath(filename)
        if len(new_name) <= max_length:
            msg = 'Using relative path: {} -> {}'
            logger.debug(msg.format(filename, new_name))
            return new_name

        if make_symlink:
            # shorten path by creating a symlink inside a safe temp dir
            _, ext = os.path.splitext(filename)
            dirname = tempfile.mkdtemp(dir=tmpdir)
            newname = os.path.join(dirname, os.path.basename(filename))
            if len(newname) > max_length:
                fd, newname = tempfile.mkstemp(suffix=ext, dir=dirname)
                os.close(fd)
                os.unlink(newname)

            if len(newname) > max_length:
                newname = os.path.relpath(newname)

            if len(newname) <= max_length:
>               os.symlink(filename, newname)
E               OSError: [WinError 1314] A required privilege is not held by the client: 'really_really_really_really_really_really_really_really_really_really_really_really_really_really_really_really_really_really_really_really_long_name\\aaaaaaaaaaaaaaaa.pdb' -> '.\\tmpg9t9ys2g\\aaaaaaaaaaaaaaaa.pdb'

..\..\package\MDAnalysis\analysis\hole2\utils.py:142: OSError

Code to reproduce the behavior

pytest --disable-pytest-warnings --numprocesses 4

Current version of MDAnalysis

• Which version are you using? current develop
• Which version of Python (python -V)? Python 3.8.3 (Python 3.8.3 | packaged by conda-forge | (default, Jun 1 2020, 16:59:10) [MSC v.1916 64 bit (AMD64)] on win32)
• Which operating system? Windows 10 Home 64-bit (MINGW64_NT-10.0-17763 MSI 3.0.7-338.x86_64 2019-11-21 23:07 UTC x86_64 Msys)

[lilyminium]

@pytest.mark.skipif(os.name == 'nt' and sys.maxsize <= 2**32,
                    reason="OSError: symbolic link privilege not held")

It makes sense that lack of symlinking privileges is not just a 32-bit thing; I guess we need a 64-bit build on Azure to catch this sort of stuff.

[orbeckst]

I didn't know that we had a test skipping because of lack of symlinking privileges.

Shouldn't the AppVeyor build have caught it? – I thought that one runs completely 64 bit everything?

cc @tylerjereddy

[tylerjereddy]

I don't think the failure happens on all Windows machines. There's probably some privilege issue with symlink on windows, so I don't think Appveyor is "missing" it, it just has a different config.

If I'm correct in my assumption that there's no HOLE port for Windows anyway, it seems like we could just broaden the skip to include 64-bit for these HOLE-related tests.

[tylerjereddy]

Or, if you want to be really thorough, I suppose you could check for the privilege issue/exception and xfail if it is present, otherwise run the test, or similar.