Hello!I want

[SZW-zjbd]

Hello! Thanks for taking interest in EGG. We are glad you're here to help us improve it!
Please fill this template as thoroughly as possible, that would make things much easier for us :)

Expected Behavior

Current Behavior

Steps to Reproduce

Detailed Description

Possible Implementation

[SZW-zjbd]

I would like to ask where the paper(Cross-Domain Image Captioning with Discriminative Finetuning) code link is. Why the link is empty? Please provide me with this code if possible!Thank u!!

[robertodessi]

Hi,

The code for the paper is here https://github.com/robertodessi/EGG/tree/rll_refactor/egg/zoo/emergent_captioner
It's just in my private branch, I'll try to merge as soon as I can but I lost push rights since I am not longer at Meta.

The command to launch it and reproduce the clipcap experiments in the paper is

python -m egg.zoo.emergent_captioner.finetuning.train \
    --dataset_dir <PATH TO YOUR COCO DIRECTORY> \
    --clipcap_model_path <PATH TO A CLIPCAP CHECKPOINTS, YOU CAN GET IT FROM THEIR REPO> \
    --baseline mean \
    --n_epochs 20 \
    --batch_size 100 \
    --lr 1e-7 \
    --max_length 20

Hope this helps.

[JuneCly]

Hi,

The code for the paper is here https://github.com/robertodessi/EGG/tree/rll_refactor/egg/zoo/emergent_captioner It's just in my private branch, I'll try to merge as soon as I can but I lost push rights since I am not longer at Meta.

The command to launch it and reproduce the clipcap experiments in the paper is

python -m egg.zoo.emergent_captioner.finetuning.train \
    --dataset_dir <PATH TO YOUR COCO DIRECTORY> \
    --clipcap_model_path <PATH TO A CLIPCAP CHECKPOINTS, YOU CAN GET IT FROM THEIR REPO> \
    --baseline mean \
    --n_epochs 20 \
    --batch_size 100 \
    --lr 1e-7 \
    --max_length 20

Hope this helps.

Hi! I'm very insterested in your preety work! Could you please provide the pre-trained checkpoint file of the clipcap model? Thank u!!!

[nidongpinyinme]

Hi,

The code for the paper is here https://github.com/robertodessi/EGG/tree/rll_refactor/egg/zoo/emergent_captioner It's just in my private branch, I'll try to merge as soon as I can but I lost push rights since I am not longer at Meta.

The command to launch it and reproduce the clipcap experiments in the paper is

python -m egg.zoo.emergent_captioner.finetuning.train \
    --dataset_dir <PATH TO YOUR COCO DIRECTORY> \
    --clipcap_model_path <PATH TO A CLIPCAP CHECKPOINTS, YOU CAN GET IT FROM THEIR REPO> \
    --baseline mean \
    --n_epochs 20 \
    --batch_size 100 \
    --lr 1e-7 \
    --max_length 20

Hope this helps.

sorry to bother you, but I really don't know where to find those files that hard negatived in your utlis.py files
DATASET2NEG_PATHS = { "flickr": ( "/private/home/rdessi/EGG/egg/zoo/emergent_captioner/hard_negatives/flickr/train_flickr.emb.pt", "/private/home/rdessi/EGG/egg/zoo/emergent_captioner/hard_negatives/flickr/train_flickr.nns.pt", ), "coco": ( "/private/home/rdessi/EGG/egg/zoo/emergent_captioner/hard_negatives/coco/train_coco.emb.pt", "/private/home/rdessi/EGG/egg/zoo/emergent_captioner/hard_negatives/coco/train_coco.nns.pt", ), "conceptual": ( "/private/home/rdessi/EGG/egg/zoo/emergent_captioner/hard_negatives/conceptual/train_conceptual.emb.pt", "/private/home/rdessi/EGG/egg/zoo/emergent_captioner/hard_negatives/conceptual/train_conceptual.nns.pt", ), }

[robertodessi]

Hi, thanks for your interest in the paper.
For the hard negatives you need to recompute them and you can use the script here