Skip to content

Recursive log dirs? #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ollenordesjo opened this issue Jul 12, 2021 · 3 comments
Open

Recursive log dirs? #16

ollenordesjo opened this issue Jul 12, 2021 · 3 comments

Comments

@ollenordesjo
Copy link

pip install runx installs v.0.0.11.

This breaks the log functionality somehow, and I'm seeing:

Copying codebase to ../logs/sweep/spectral-okapi_2021.07.12_15.16 ...
Running job sweep_spectral-okapi_2021.07.12_15.16
cd ../logs/sweep/spectral-okapi_2021.07.12_15.16/code; PYTHONPATH=  python testrunx.py --lr 0.01 --solver first --logdir ..//logs
/bin/sh: 1: cd: can't cd to ../logs/sweep/spectral-okapi_2021.07.12_15.16/code
python: can't open file 'testrunx.py': [Errno 2] No such file or directory

Copying codebase to ../logs/sweep/teal-trogon_2021.07.12_15.16 ...
Running job sweep_teal-trogon_2021.07.12_15.16
cd ../logs/sweep/teal-trogon_2021.07.12_15.16/code; PYTHONPATH=  python testrunx.py --lr 0.01 --solver second --logdir ..//logs
/bin/sh: 1: cd: can't cd to ../logs/sweep/teal-trogon_2021.07.12_15.16/code
python: can't open file 'testrunx.py': [Errno 2] No such file or directory

Copying codebase to ../logs/sweep/garrulous-wombat_2021.07.12_15.16 ...
Running job sweep_garrulous-wombat_2021.07.12_15.16
cd ../logs/sweep/garrulous-wombat_2021.07.12_15.16/code; PYTHONPATH=  python testrunx.py --lr 0.02 --solver first --logdir ..//logs
/bin/sh: 1: cd: can't cd to ../logs/sweep/garrulous-wombat_2021.07.12_15.16/code
python: can't open file 'testrunx.py': [Errno 2] No such file or directory

Copying codebase to ../logs/sweep/lavender-prawn_2021.07.12_15.16 ...
Running job sweep_lavender-prawn_2021.07.12_15.16
cd ../logs/sweep/lavender-prawn_2021.07.12_15.16/code; PYTHONPATH=  python testrunx.py --lr 0.02 --solver second --logdir ..//logs
/bin/sh: 1: cd: can't cd to ../logs/sweep/lavender-prawn_2021.07.12_15.16/code
python: can't open file 'testrunx.py': [Errno 2] No such file or directory

Looking at ../logs, I'm seeing recursive paths like this one:

../logs/sweep/logs/sweep/logs/sweep/logs/sweep/complex-mongrel_2021.07.12_15.06/submit_cmd.sh

Here is my python script (testrunx.py)

import sys
print(sys.argv)

And here is my sweep.yml:

CMD: 'python testrunx.py'

HPARAMS:
        lr: [0.01, 0.02]
        solver: ['first', 'second']
        logdir: '..//logs'

Installing version 0.0.5 fixes the problem.

@ajtao
Copy link
Collaborator

ajtao commented Jul 12, 2021

Hi, did you create a .runx file in which you define a LOGROOT? I would do so, and make the path absolute. Maybe try that.

@ollenordesjo
Copy link
Author

Yep, I tried to do so, and made the path absolute now as well. When running (on v.0.0.11), I'm getting something similar to this:

 python -m runx.runx sweep.yml -i
Copying codebase to /home/.../onordesjo/git/runxtest/sweep/economic-lyrebird_2021.07.13_13.17 ...
Traceback (most recent call last):
  File "/home/.../onordesjo/miniconda3/envs/sequences/lib/python3.8/runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/.../onordesjo/miniconda3/envs/sequences/lib/python3.8/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/.../onordesjo/miniconda3/envs/sequences/lib/python3.8/site-packages/runx/runx.py", line 402, in <module>
    main()
  File "/home/.../onordesjo/miniconda3/envs/sequences/lib/python3.8/site-packages/runx/runx.py", line 395, in main
    run_experiment(args.exp_yml)
  File "/home/.../onordesjo/miniconda3/envs/sequences/lib/python3.8/site-packages/runx/runx.py", line 390, in run_experiment
    run_yaml(experiment, runroot)
  File "/home/.../onordesjo/miniconda3/envs/sequences/lib/python3.8/site-packages/runx/runx.py", line 345, in run_yaml
    copy_code(logdir, runroot, code_ignore_patterns)
  File "/home/.../onordesjo/miniconda3/envs/sequences/lib/python3.8/site-packages/runx/runx.py", line 216, in copy_code
    copytree(runroot, tgt_code_dir, ignore=code_ignore_patterns)
  File "/home/.../onordesjo/miniconda3/envs/sequences/lib/python3.8/shutil.py", line 547, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
  File "/home/.../onordesjo/miniconda3/envs/sequences/lib/python3.8/shutil.py", line 504, in _copytree
    raise Error(errors)
shutil.Error: [('/home/.../onordesjo/git/runxtest/sweep/economic-lyrebird_2021.07.13_13.17/code/sweep/economic-lyrebird_2021.07.13_13.17/code/sweep/economic-lyrebird_2021.07.13_13.17/code/sweep/economic-lyrebird_2021.07.13_13.17/code/sweep/economic-lyrebird_2021.07.13_13.17/code/sweep/economic-lyrebird_2021.07.13_13.17/code/sweep/economic-lyrebird_2021.07.13_13.17/code/sweep/economic-lyrebird_2021.07.13_13.17/code/sweep/economic-lyrebird_2021.07.13_13.17/code/sweep/economic-lyrebird_2021.07.13_13.17/code/sweep/economic-lyrebird_2021.07.13_13.17...
...
economic-lyrebird_2021.07.13_13.17/code/sweep'")]

@ajtao
Copy link
Collaborator

ajtao commented Jul 13, 2021

It's not clear to me why there's ... in your LOGROOT path. Maybe you could show your .runx file?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants