MOCO V3 vit_small error: object has no attribute "num_tokens"

[shree-lily]

When I attempt to pre-train moco v3's vit_small model, I run into the following bug:

raise AttributeError("'{}' object has no attribute '{}'".format( AttributeError: 'VisionTransformerMoCo' object has no attribute 'num_tokens'

After changing the line
vits.py-line-66-LINK to
assert self.num_prefix_tokens == 1, 'Assuming one and only one token, [cls]' I don't see the bug anymore. It seems like the base class timm.models.vision_transformer has an argument named num_prefix_tokens but not num_tokens and hence vit_small is erroring out at the above mentioned line.

The command I used to run the code is:
python main_moco.py \ -a vit_small -b 1024 \ --optimizer=adamw --lr=1.5e-4 --weight-decay=.1 \ --epochs=400 --warmup-epochs=40 \ --stop-grad-conv1 --moco-m-cos --moco-t=.2 \ --dist-url 'tcp://localhost:8080' \ --multiprocessing-distributed --world-size 1 --rank 0 \ /data/

Please let me know if this is an accurate fix, or if I missed something. Thanks in advance!

[rohan1561]

I'm having the same issue using vit_small. I think it may be a typo and it was meant to be num_prefix_tokens. Kindly clarify, authors! The error is from line 66 in vits.py.

[ericosmic]

same bug

[755328872]

hello，you should follow the version of timm, it may work.

[gitillbr]

hello，you should follow the version of timm, it may work.

大佬您好，请问可以问您点问题嘛，mocov3可以用自定义数据集嘛

[755328872]

可以

[gitillbr]

可以

谢谢大佬，麻烦您了
[email protected]