Instruction to visualize hard attention

[vyouman]

Hi, thanks for sharing the code. I expect the alpha_visualization.ipynb to visualize hard attention just like what the paper represented. But I trained a stochastic one and visualize it , then I got visualization like the soft one. Has anyone successfully visualize the hard attention and give me some instrustions on how to do it? Thanks. :)

[vyouman]

Have got
bleu1 bleu2 bleu3 bleu4 meteor
67.0 49.0 35.0 24.9 22.8
on MSCOCO dataset. However, it doesn't seem to attend to the right place though the sentence is pretty good. I follow the instruction of @Lorne0 and modify the attention type to stochastic one. To be clear, the visualization is almost the same at all time steps. In other words, the attention weights doesn't seem to learn to attend to the specific place at timestep t. Do anyone have some ideas about it?

My setup is
{'lrate': 0.01, 'decay_c': 0.0, 'patience': 10, 'save_per_epoch': True, 'n_layers_init': 2, 'RL_sumCost': True, 'max_epochs': 5000, 'dispFreq': 1, 'attn_type': 'stochastic', 'alpha_c': 1.0, 'temperature': 1.0, 'n_layers_att': 2, 'saveto': 'coco_stochastic_model.exp1.npl', 'ctx_dim': 512, 'valid_batch_size': 64, 'lstm_encoder': False, 'n_layers_lstm': 1, 'optimizer': 'adam', 'validFreq': 2000, 'dictionary': None, 'batch_size': 64, 'selector': True, 'n_words': 10500, 'dataset': 'coco', 'use_dropout_lstm': False, 'prev2out': True, 'dim': 1800, 'use_dropout': True, 'dim_word': 512, 'sampleFreq': 250, 'semi_sampling_p': 0.5, 'n_layers_out': 1, 'saveFreq': 1000, 'maxlen': 100, 'alpha_entropy_c': 0.002, 'ctx2out': True, 'reload_': False}

I've run 21 epochs.

Or alpha_visualization doesn't support a stochastic one? The paper only mentioned the visualization of the soft attention.

[AMDS123]

I have the same problem too.

[Shanzaay]

@vyouman and @AMDS123 can you guys tell me how to start the code and train the dataset.