AttributeError: module 'ncluster' has no attribute 'set_backend'

[wayne0908]

Hi,

Thank you for releasing the code. I got the error "AttributeError: module 'ncluster' has no attribute 'set_backend'" while I try to run the code. Could you please let me know how to solve this? Thanks

[yaroslavvb]

You can safely comment out that line. I'm planning to merge repo this week, the actual repo is this one https://github.com/cybertronai/imagenet18_

[wayne0908]

Thanks for the reply. If I understand correctly, the code shall run on a local machine and would automatically connect to our AWS account to set up everything? I am not sure if I understand how to run the code correctly but now I got the error as follows:
...
2019-07-27 22:47:24.629049 4.imagenet: Allocating p3.16xlarge for task 4.imagenet
2019-07-27 22:47:25.607062 5.imagenet: Allocating p3.16xlarge for task 5.imagenet
Exception are [KeyError('UserName',), KeyError('UserName',), KeyError('UserName',), KeyError('UserName',), KeyError('UserName',), KeyError('UserName',), KeyError('UserName',), KeyError('UserName',), KeyError('UserName',), KeyError('UserName',), KeyError('UserName',), KeyError('UserName',), KeyError('UserName',), KeyError('UserName',), KeyError('UserName',), KeyError('UserName',)]
Traceback (most recent call last):
File "train.py", line 196, in
main()
File "train.py", line 165, in main
install_script=open('setup.sh').read())
File "/usr/local/lib/python3.6/site-packages/ncluster/aws_backend.py", line 1291, in make_job
raise exceptions[0]
File "/usr/local/lib/python3.6/site-packages/ncluster/aws_backend.py", line 1272, in make_task_fn
**kwargs)
File "/usr/local/lib/python3.6/site-packages/ncluster/aws_backend.py", line 1102, in make_task
aws_user = u.get_iam_username()
File "/usr/local/lib/python3.6/site-packages/ncluster/aws_util.py", line 271, in get_iam_username
return iam_client.get_user()['User']['UserName']
KeyError: 'UserName'

[yaroslavvb]

Sorry, that seems to be a bug in ncluster, I will update it today and cut new version. Temporary workaround is to define an account alias, as described here -- https://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html

[yaroslavvb]

Should be fixed, do pip install -U ncluster
By the way, the latest most tested version is this one, can you try it instead? https://github.com/cybertronai/imagenet18_/tree/dev

Here's a log of a recent run from that branch

I will consolidate all the branches later this week

[yaroslavvb]

btw, understanding is correct, it will automatically setup all the machines. The machines will should shut down automatically after training is done, but check just in case

[wayne0908]

Hi, thanks for your update. It seems the problem still exists on my side. Not sure if it is the problem of the configuration of my computer. Thanks anyway!

[yaroslavvb]

Sorry for confusion, this repo is out of date, set_backend method no longer exists/isn't needed
I've consolidated branches/names under "imagenet18" repo: https://github.com/cybertronai/imagenet18

Just ran through this today on fresh account and seems to work, let me know if something doesn't work