Keras Attention 机制.md

Keras Attention 实现

Attention 资料

如果你想进一步地学习如何在LSTM/RNN模型中加入attention机制，可阅读以下论文：

• Attention in Long Short-Term Memory Recurrent Neural Networks-by Jason Brownlee

• Attention and Augmented Recurrent Neural Networks

• Deep Language Modeling for Question Answering using Keras-codekansas

  简单介绍了Keras的使用，以及细致讲解了简单的 Attentional LSTM 模型实现！！！

• Attention and memory in deep learning and NLP

• Attention Mechanism

• Survey on Attention-based Models Applied in NLP

• What is exactly the attention mechanism introduced to RNN? （来自Quora）

• What is Attention Mechanism in Neural Networks?

Attention 实现

目前Keras官方还没有单独将attention模型的代码开源，下面有一些第三方的实现：

• Deep Language Modeling for Question Answering using Keras

• Attention Model Available!

• Keras Attention Mechanism

  • attention_lstm
  • attention_dense

• Keras Blstm+attention 的简单实现

• keras-language-modeling

  代码的讲解对应上面的资料《Deep Language Modeling for Question Answering using Keras》，但是Keras 1.x 的版本已经无法使用。

  • attention_lstm_
  • 词向量生成

• Attention and Augmented Recurrent Neural Networks

• How to add Attention on top of a Recurrent Layer (Text Classification)

• Attention Mechanism Implementation Issue

• Implementing simple neural attention model (for padded inputs)

• Attention layer requires another PR

• seq2seq library

• keras-attention

  Zafarali Ahmed an intern at Datalogue developed a custom layer for Keras that provides support for attention, presented in a post titled “How to Visualize Your Recurrent Neural Network with Attention in Keras” in 2017 and GitHub project called “keras-attention“.

  • custom_recurrents.py
  • tdd.py

• 基于Attention Model的Aspect level文本情感分类---用Python+Keras实现

• Keras 自注意力层实现

• 可视化您的递归神经网络以及关注Keras ” 的帖子提供了支持，GitHub项目名为“ keras-attention ”

• 《Attention is All You Need》中的Attention机制的实现

• dl-models-for-qa

  • QA-LSTM CNN
  • QA-LSTM with Attention
  • Incorporating External Knowledge
  • QA-LSTM with Attention and Custom Embedding

• ​