Explain CNN+LSTM Classification

[gabarlacchi]

Hi!
I am developing a clip classificator using features (extracted from a CNN) fed into an LSTM for classification. It's an end-to-end system. I would like to know if it's possible apply eli5 visualization tool to such a network in order to display, at each classification, the area into the images of the clip that has been "important" for classify if.
Someone has already implemented such a system?
Thanks in advance,
Gabriele

[teabolt]

Hi @barloccia,
ELI5 can highlight what's important in an image for Keras CNN's, https://eli5.readthedocs.io/en/latest/tutorials/keras-image-classifiers.html.
Support for LSTM's and other libraries is still work in progress and not merged #325, #327.
Regards,
Tomas

[simplezhang57]

how to use "permutation_importance" on keras LSTM to find out feature importance?

[gabarlacchi]

@teabolt Thanks for the reply. I managed to make what desired with CONV-LSTM and visualize the activations through the time.