Using bindsnet for temperature prediction？

[Whale-xh]

Hello, I have never found any research on bindnet for regression tasks online, so I am trying to use bindnet for temperature prediction. However, when I was using bindnet, I found that the loss value of the network cannot be reduced. Could you please help me check the code for any errors and make modifications? I would greatly appreciate it!

Import package module, where CPUdata is a dataset I created myself
import torch import numpy as np import matplotlib.pyplot as plt import pandas as pd from torch.nn import DataParallel from torch.utils.data import DataLoader from torch.autograd import Variable from bindsnet.network import Network from bindsnet.network.nodes import * from bindsnet.encoding import PoissonEncoder from bindsnet.network.monitors import Monitor from bindsnet.learning import PostPre from bindsnet.network.topology import * from CPUdataset import CPUTestDataset, CPUTrainDataset

Load Dataset
`events_train_file = r'../train_events.csv'
temps_train_file = r'../train_temps.csv'
cpu_events_train = pd.read_csv(events_train_file, header=None)
temperature_train = pd.read_csv(temps_train_file, header=None)
cpu_events_train = torch.tensor(np.array(cpu_events_train), dtype=torch.float)
temperature_train = torch.tensor(np.array(temperature_train) / 1000, dtype=torch.float)

n_epochs = 10
batch_size = 16
dt = 1
n_time_steps = 100
n_cpu_events = cpu_events_train.shape[1] #Input feature number
n_temperature_features = 1 #Output feature number
n_hidden = 100
n_output = 1

data_train = CPUTrainDataset(events_train_file, temps_train_file)
dataloader_train = DataLoader(data_train, batch_size=batch_size, shuffle=False)`

Network structure
`nu=[1e-5, 1e-3]
theta_plus=0.05
norm=78.4
exc = 22.5
inh = 120

network = Network(dt=dt,batch_size=batch_size)
input_layer = Input(n=n_cpu_events,traces=True,tc_trace=20.0)
hidden_layer = DiehlAndCookNodes(n=n_hidden,traces=True,
rest=-65.0,
reset=-60.0,
thresh=-52.0,
refrac=5,
tc_decay=100.0,
tc_trace=20.0,
theta_plus=theta_plus,
tc_theta_decay=1e7)
output_layer = LIFNodes(n=n_output, traces=True,rest=-60.0,
reset=-45.0,
thresh=-40.0,
tc_decay=10.0,
refrac=2,
tc_trace=20.0)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
network.to(device)

if torch.cuda.device_count() > 1:
network = DataParallel(network)

w = 0.3 * torch.rand(n_cpu_events, n_hidden)
input_hidden_conn = Connection(source=input_layer, target=hidden_layer,
update_rule=PostPre,
nu=nu,
reduction=None,
w=w,
wmin=0.0,
wmax=1.0,
norm=norm)

w = exc * torch.rand(n_hidden,n_output)
hidden_output_conn = Connection(source=hidden_layer, target=output_layer,w=w, wmin=0, wmax=exc)

output_output_conn = Connection(output_layer, output_layer,wmin=inh,wmax=0)

network.add_layer(input_layer, name="Input")
network.add_layer(hidden_layer, name="Hidden")
network.add_layer(output_layer, name="Output")
network.add_connection(input_hidden_conn, source="Input", target="Hidden")
network.add_connection(hidden_output_conn, source="Hidden", target="Output")
network.add_connection(output_output_conn, source="Output", target="Output")

input_monitor = Monitor(input_layer, state_vars=["s"], time=n_time_steps)
hidden_monitor = Monitor(hidden_layer, state_vars=["s", "v"], time=n_time_steps)
output_monitor = Monitor(output_layer, state_vars=["s", "v"], time=n_time_steps)
network.add_monitor(input_monitor, name="Input Monitor")
network.add_monitor(hidden_monitor, name="Hidden Monitor")
network.add_monitor(output_monitor, name="Output Monitor")`

`def inverse_encoding(spikes, time, dt=1.0):
shape = spikes.shape[1:]
spikes = spikes.float().view(time, -1) # flatten
total_spikes = spikes.sum(dim=0)
rate = total_spikes / (time * dt)
datum = rate # firing rate = original intensity

return datum.view(*shape)

#Temperature mapping
def temp_map2(rates,batchSize):
pred_temp = []
for i in range(batchSize):
pred_temp.append(0 + 100 * rates[i])
return pred_temp

encoder = PoissonEncoder(time=n_time_steps, dt=dt)

criterion = torch.nn.MSELoss()
optimizer = torch.optim.Adam(network.parameters(), lr=1e-3)
`

Train
`real_temps = temperature_train[:, 0]
predicted_temps = []

for epoch in range(n_epochs):
total_loss = 0.0
total_accuracy = 0.0
for batch_data in dataloader_train:
x_cpu, y_temperature = batch_data
x_cpu = encoder(x_cpu)
x_cpu = x_cpu.to(device)
y_temperature = y_temperature[0].to(device)

    inputs = {"Input": x_cpu}
    network.run(inputs=inputs, time=n_time_steps,clamp=clamp)

    output_spikes = output_monitor.get("s")
    output_voltage = output_monitor.get("v")
    rates = inverse_encoding(output_spikes, time=n_time_steps, dt=dt).view(-1).tolist()
    y_pred = temp_map2(rates,batch_size)
    predicted_temps.extend(y_pred)
    y_pred=torch.tensor(y_pred)

    accuracy = torch.mean(torch.abs(y_pred - y_temperature) / y_temperature)
    total_accuracy += accuracy.item()
    loss = criterion(y_pred, y_temperature)
    loss_value = loss.item()
    total_loss += loss_value

    optimizer.zero_grad()
    loss = criterion(y_pred, y_temperature)
    loss = Variable(loss, requires_grad=True)
    loss.backward()
    optimizer.step()

    network.reset_state_variables()

mean_accuracy = total_accuracy / (cpu_events_train.shape[0] / batch_size)
mean_loss = total_loss / (cpu_events_train.shape[0] / batch_size)

print(f"Epoch {epoch + 1}/{n_epochs} Loss: {mean_loss:.4f} Accuracy: {mean_accuracy:.4f}")

print("Training complete!")`

[Hananel-Hazan]

If you're seeking assistance with your code, consider improving its readability and paste is here in a way that is make it easier for others to help you.

Question:

Why are you using "norm" here? The "norm" value is typically suitable for MNIST, but I'm unsure if it will work in this context. Is there a specific reason why you've applied "norm" to the input-to-hidden layer but not to the hidden-to-output layer?

As I understand your code, you are using a spiking network to encode your input and applying a linear classifier to distinguish between the different spiking patterns of each class. If that current, plot the spikes train of each class and eyeball the plots to see if there is any different between the two plots. My guess that the two plots are the same.

[Whale-xh]

Thank you for your reply. I just got to know bindnet and SNN, and there are many things I don't understand. The establishment of the network is modeled after supervised_mnist,How should I establish a network model for regression tasks?

[Hananel-Hazan]

Similar to how supervised_mnist.py utilizes the clamp feature to guide the designated neuron in the output layer, coupled with the STDP rule to adjust network weights in order to associate input with the designated output neuron, employ this process to instruct your network for regression tasks.