Global network (formed by 2 Diehl&Cook nets and connected by their excitatory layers) possible to be done?

[ManjaJRamon]

Hi, i'm trying to implement this paper "https://ieeexplore.ieee.org/document/8482490" which was originally done in Brian2, now in Bindsnet, using the MNIST Dataset. For the audio network I already have an implementation that transforms audio samples in images of the number spoken in the original sample, so the entrance for the image subnet would be the MNIST dataset, and for the audio subnet would be a pseudo-MNIST which come out from this transformation but has the same dimensions 28px X 28px.

I'm not quite sure if you can create a network with two diferent Input() declarations and if it maps the entries, for example inputs = {"X" : batch1["image"], "Y" : batch2["image"]} and then only the content in inputs["X"] goes to Input1() and only the content of inputs["Y"] goes to Input2() when you execute the command network.run(inputs=inputs,...).

Also if i'm not wrong, the paper i'm trying to recreate in Bindsnet, requires the ability to manipulate nodes individually within a layer, with the aim of creating connections between specific nodes from one excitatory layer to the other one. As I've seen, when you create a Connection object in Bindsnet, the behaviour you specify to one node extends to the rest of the nodes allocated in the same layer, so the connections expected to appear between excitatory layers would not be possible, as it doesn't follow a global behaviour for all the nodes in the origin layer of the connection.

The article talks about being able to detect that a excitatory neuron from one of the nets predicts a "5" using MNIST dataset, and create a connection with trainable weights (stdp rule) with another excitatory neuron from the other net that also predicts a "5", but only in one direction and a p2p connection.

That is, at the end we would have all the excitatory neurons from both layers connected to excitatory neurons of the opposite network that predicts the same label as them.

My issue here is if i could get some feedback about whether the paper above being able to be reproduced in Bindsnet or not, as I think it can't, but I would like to know if I'm giving it a fair try.

Thanks in advance, i'll leave below the issue the image that can be found in the article and defines the network connections.

[Hananel-Hazan]

Yes I dont see why not.

Also if i'm not wrong, the paper i'm trying to recreate in Bindsnet, requires the ability to manipulate nodes individually within a layer, with the aim of creating connections between specific nodes from one excitatory layer to the other one. As I've seen, when you create a Connection object in Bindsnet, the behaviour you specify to one node extends to the rest of the nodes allocated in the same layer, so the connections expected to appear between excitatory layers would not be possible, as it doesn't follow a global behaviour for all the nodes in the origin layer of the connection.

You can use masks with the connection object to enforce the topology you want.

[ManjaJRamon]

Where can i find those masks and how do I use them? I'm not familiar with the term

[Hananel-Hazan]

Currently BindsNET use a weight matrix to connect between two different layers. If you want to create a unique topology where a subgroup of a layer connect only to a different subgroup of other layer, you can use the masks option in the run method of the network object. The mask are a boolean matrix with the same dimension of the weight matrix that use to zero the unwanted connection and to enforce topology.