heyhi version / FROZEN_SYM_BD issue

[bkj]

Hi --

I'm following the installation instructions in the README, and make test_fast fails with a bunch of errors like:

======================================================================
ERROR: testLoadSimple (test_conf.TestRootWithIncludesConf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ubuntu/projects/diplomacy_searchbot/heyhi/tests/test_conf.py", line 85, in testLoadSimple
    cfg = self._load(overrides=[])
  File "/home/ubuntu/projects/diplomacy_searchbot/heyhi/tests/test_conf.py", line 80, in _load
    task, meta_cfg = heyhi.conf.load_cfg(root_cfg, overrides)
AttributeError: module 'heyhi.conf' has no attribute 'load_cfg'

I guessed that maybe those calls should be changed from heyhi.conf.load_cfg to heyhi.conf.load_config -- though if I do that, I get errors like

======================================================================
ERROR: testLoadSimple (test_conf.TestRootWithIncludesAndRedefinesConf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ubuntu/projects/diplomacy_searchbot/heyhi/tests/test_conf.py", line 106, in testLoadSimple
    cfg = self._load(overrides=[])
  File "/home/ubuntu/projects/diplomacy_searchbot/heyhi/tests/test_conf.py", line 101, in _load
    task, meta_cfg = heyhi.conf.load_cfg(root_cfg, overrides)
  File "/home/ubuntu/projects/diplomacy_searchbot/heyhi/conf.py", line 389, in load_config
    return cfg.to_frozen()
  File "/home/ubuntu/projects/diplomacy_searchbot/conf/conf_pb2.py", line 63, in to_frozen
    value = maybe_to_dict(value)
  File "/home/ubuntu/projects/diplomacy_searchbot/conf/conf_pb2.py", line 47, in maybe_to_dict
    return msg.to_frozen()
  File "/home/ubuntu/projects/diplomacy_searchbot/conf/conf_pb2.py", line 63, in to_frozen
    value = maybe_to_dict(value)
  File "/home/ubuntu/projects/diplomacy_searchbot/conf/conf_pb2.py", line 47, in maybe_to_dict
    return msg.to_frozen()
  File "/home/ubuntu/projects/diplomacy_searchbot/conf/conf_pb2.py", line 83, in to_frozen
    return FROZEN_SYM_BD[descriptor.full_name](ret, self)
KeyError: 'fairdiplomacy.TestTask.SubMessage'
-------------------- >> begin captured logging << --------------------
root: INFO: Going to guess message type by trying all of them
root: INFO: Guessed type: <class 'conf.conf_pb2.MetaCfg'>
root: DEBUG: Constructing MetaCfg from /home/ubuntu/projects/diplomacy_searchbot/heyhi/tests/data/root_with_includes_redefined.prototxt with include overrides {} and scalar overrides {}
root: DEBUG: <class 'conf.conf_pb2.MetaCfg'> defaults {'sub': 'redefine_subscalar_22', 'sub2': 'redefine_subscalar_22'}
root: DEBUG: Constructing MetaCfg. Applying include: mount='test.sub' include=PosixPath('/home/ubuntu/projects/diplomacy_searchbot/heyhi/tests/data/redefine_subscalar_22.prototxt') subcfg=SubMessage
root: DEBUG: Constructing SubMessage from /home/ubuntu/projects/diplomacy_searchbot/heyhi/tests/data/redefine_subscalar_22.prototxt with include overrides {} and scalar overrides {}
root: DEBUG: <class 'conf.conf_pb2.SubMessage'> defaults {}
root: DEBUG: Constructing MetaCfg. Applying include: mount='test.sub2' include=PosixPath('/home/ubuntu/projects/diplomacy_searchbot/heyhi/tests/data/redefine_subscalar_22.prototxt') subcfg=SubMessage
root: DEBUG: Constructing SubMessage from /home/ubuntu/projects/diplomacy_searchbot/heyhi/tests/data/redefine_subscalar_22.prototxt with include overrides {} and scalar overrides {}
root: DEBUG: <class 'conf.conf_pb2.SubMessage'> defaults {}
--------------------- >> end captured logging << ---------------------

Any thoughts? Thanks!

[david-andrew]

I was able to get make test_fast working by doing the following:

1. it looks like the function heyhi.conf.load_cfg should be pointing to heyhi.conf.load_root_proto_message, so I added the following to heyhi/conf.py as a dumb wrapper to call the right function:

def load_cfg(*args, **kwargs):
    return load_root_proto_message(*args, **kwargs)

Alternatively you can replace all calls to heyhi.conf.load_cfg with heyhi.conf.load_root_proto_message in heyhi/tests/test_conf.py

2. There was one more error in unit_tests/test_resample_duplicate_disbands.py which was importing from the wrong location. I updated line 10

#from fairdiplomacy.agents.model_sampled_agent import resample_duplicate_disbands_inplace
from fairdiplomacy.agents.model_wrapper import resample_duplicate_disbands_inplace

[lightvector]

Sorry for the trouble, some last minute merging of different versions didn't go right. A fix should be pushed now to main for those errors and a few others.

[david-andrew]

Sorry for the trouble, some last minute merging of different versions didn't go right. A fix should be pushed now to main for those errors and a few others.

Hey, thanks for pushing the fix, though I think there might still be some issues

Running make test_integration fails it looks like when it's trying to save the result of a game because of a missing file/directory "squeue":

...
W0214 11:45:33 [run:54] Result: draw
I0214 11:45:33 [util:374] Saving result to /home/david/Programming/Jataware/facebook_diplomacy_searchbot/result.torch
Traceback (most recent call last):
  File "run.py", line 102, in <module>
    heyhi.parse_args_and_maybe_launch(main)
  File "/home/david/Programming/Jataware/facebook_diplomacy_searchbot/heyhi/run.py", line 114, in parse_args_and_maybe_launch
    maybe_launch(main, exp_root=get_exp_dir(PROJECT_NAME), overrides=overrides, **kwargs)
  File "/home/david/Programming/Jataware/facebook_diplomacy_searchbot/heyhi/run.py", line 167, in maybe_launch
    if exp_handle.is_done():
  File "/home/david/Programming/Jataware/facebook_diplomacy_searchbot/heyhi/util.py", line 333, in is_done
    return self.get_status() == Status.DONE
  File "/home/david/Programming/Jataware/facebook_diplomacy_searchbot/heyhi/util.py", line 326, in get_status
    if maybe_jobid is not None and maybe_jobid in get_all_runing_job_ids():
  File "/home/david/Programming/Jataware/facebook_diplomacy_searchbot/heyhi/util.py", line 261, in get_all_runing_job_ids
    _SLURM_CACHE["job_list"] = _get_all_runing_job_ids()
  File "/home/david/Programming/Jataware/facebook_diplomacy_searchbot/heyhi/util.py", line 253, in _get_all_runing_job_ids
    output = subprocess.check_output(cmd)
  File "/home/david/anaconda3/envs/fair_diplomacy/lib/python3.7/subprocess.py", line 411, in check_output
    **kwargs).stdout
  File "/home/david/anaconda3/envs/fair_diplomacy/lib/python3.7/subprocess.py", line 488, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/home/david/anaconda3/envs/fair_diplomacy/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/home/david/anaconda3/envs/fair_diplomacy/lib/python3.7/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'squeue': 'squeue'
E
======================================================================
ERROR: integration_test.test_compare_agents
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/david/anaconda3/envs/fair_diplomacy/lib/python3.7/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/david/Programming/Jataware/facebook_diplomacy_searchbot/integration_tests/integration_test.py", line 75, in test_compare_agents
    "max_turns=1",
  File "/home/david/Programming/Jataware/facebook_diplomacy_searchbot/integration_tests/heyhi_utils.py", line 40, in run_config
    return subprocess.check_call(cmd, env=env)
  File "/home/david/anaconda3/envs/fair_diplomacy/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python', 'run.py', '--cfg', '/home/david/Programming/Jataware/facebook_diplomacy_searchbot/conf/c01_ag_cmp/cmp.prototxt', '--mode', 'restart', '--force', 'I.agent_one=agents/model_sampled', 'I.agent_six=agents/model_sampled', 'agent_one.model_sampled.model_path=/tmp/dip_tests/c01_ag_cmp/model.pth', 'agent_six.model_sampled.model_path=/tmp/dip_tests/c01_ag_cmp/model.pth', 'max_turns=1']' returned non-zero exit status 1.
...

Additionally, when trying to run a game with a random agent, it looks like there's a point where RandomAgent is instantiated incorrectly:

python run.py --adhoc --cfg conf/c01_ag_cmp/cmp.prototxt I.agent_one=agents/searchbot.prototxt I.agent_six=agents/random.prototxt

Traceback (most recent call last):
  File "run.py", line 102, in <module>
    heyhi.parse_args_and_maybe_launch(main)
  File "/home/david/Programming/Jataware/facebook_diplomacy_searchbot/heyhi/run.py", line 114, in parse_args_and_maybe_launch
    maybe_launch(main, exp_root=get_exp_dir(PROJECT_NAME), overrides=overrides, **kwargs)
  File "/home/david/Programming/Jataware/facebook_diplomacy_searchbot/heyhi/run.py", line 166, in maybe_launch
    util.run_with_config(main, exp_handle, cfg, overrides, ckpt_dir, log_level)
  File "/home/david/Programming/Jataware/facebook_diplomacy_searchbot/heyhi/util.py", line 588, in run_with_config
    callable()
  File "/home/david/Programming/Jataware/facebook_diplomacy_searchbot/heyhi/util.py", line 371, in wrapped
    result = f(*args, **kwargs)
  File "run.py", line 98, in main
    return TASKS[task](cfg)
  File "run.py", line 39, in compare_agents
    agent_six = build_agent_from_cfg(cfg.agent_six)
  File "/home/david/Programming/Jataware/facebook_diplomacy_searchbot/fairdiplomacy/agents/__init__.py", line 51, in build_agent_from_cfg
    return AGENT_CLASSES[agent_stanza.which_agent](agent_cfg)
TypeError: RandomAgent() takes no arguments

[lightvector]

Thanks for reporting, these should be fixed now as well. The first error is due to "squeue" being present in our environment, but of course it may not be on more general environments - there may be some chance there are other similar places in the code that make some assumption about the environment the program is running in, so let us know if you see more instances of things like this.