Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scepticism about the correctness of the use of the LSTMCell #76

Open
alirezakazemipour opened this issue Feb 1, 2022 · 0 comments
Open

Comments

@alirezakazemipour
Copy link

Hi,

First of all, thank you very much for your easy-to-follow implementation! Very intuitive and simple. 👍
My question is about your use of LSTMCell to implement the recurrent version of A3C.
As it sounds from your code, you have used single timestep data (in your batch feeding to the model) to update your model including (the LSTMCell part) however, if it was intended to implement a recurrent procedure then you should have performed forward pass of the LSTMCell on each timestep in your recurrent part but it's missing thus, hidden states/cell states of the LSTMCell does not contribute to the gradient flow.

Simply put, I mean such a part is missing:

for i in inputs:
    # Step through the sequence one element at a time.
    # after each step, hidden contains the hidden state.
    out, hidden = lstm(i, hidden)

Otherwise, hx and cx should not have been detached:

cx = cx.detach()

hx = hx.detach()

Am I right? Or I'm missing something?
Thank you in advance.

@alirezakazemipour alirezakazemipour changed the title Sceptism about the correctness of the use of LSTMCell Scepticism about the correctness of the use of LSTMCell Feb 1, 2022
@alirezakazemipour alirezakazemipour changed the title Scepticism about the correctness of the use of LSTMCell Scepticism about the correctness of the use of the LSTMCell Feb 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant