Beam Search 9.Debug attention_decoder

background

beam_search=True is not working yet. We'll compare method param and return types to see expected types there.

seq2seq method

• Input is exact the same
• Output
  • (A) bucker_output is
    • False: shape=(?, 256) float32
    • True: (10,) int64'
  • (B)_ is
    • False: (?, 768) float32
    • True: (10, 768), (?, 10), (?, 10)
• Added this, not sure if it's right loop_function = _extract_beam_search( embedding, beam_size, num_symbols, embedding_size, output_projection, update_embedding_for_previous) # if feed_previous else None
• What happen if feed_previous = True
  • I guess feed_previous should always be True for beam_search
• Where (A) come from
  • embedding_attention_decoder
    • beam_attention_decoder or attention_decoder
    • or
    • loop function difference returns
      • True: (10, 256)
      • False: : loop_function=None when feed_previous=False