How to create a layer with 64 resonate and fire neuron

[zeinebCh]

Hello,
I am a beginner in this field and I am trying to build a network that generates spikes from radar data.
My input data shape is (1,192,100): 1 Batch/ 192 Samples/ and 100 Frames. The frames represents the temporal information.
I want to create a layer consisting of 192 resonate and fire neuron where each neuron has different frequency (to cover the Bandwidth).

I did the following implementation:
# initialize layers
self.neuron = rf.Neuron( threshold, period, decay, persistent_state=True)

I think that with this layer initialization, I am only capable of setting 1 resonation frequency (1 period).
And how to set the number of neuron correctly?

I would appreciate any help.
Thanks a lot.

[bamsumit]

Hi @zeinebCh, the period (also decay) can take a tuple as input which defines the minimum and maximum value to initialize the neuron with. The neurons will be initialized equally spaced linearly or in log domain of resulting frequency. This is controlled by log_init argument. Refer here for more details of the possibilities.

Here is a short snippet that does what you want.

import torch
from lava.lib.dl import slayer
input = torch.rand([1, 192, 1000])  # creating a random input
rf_params = dict(threshold=10, period=(10, 100), decay=0.9, shared_param=False, log_init=True, persistent_state=True)
block = slayer.block.rf.Dense(neuron_params=rf_params, in_neurons=192, out_neurons=192)  # a combination of dense synapse and rf neuron
out = block(input)  # this automatically determines the neuron shape from dense synapse output, which is 192
print(f'{block.neuron.shape=}')
print(f'{block.neuron.frequency=}')
print(f'{block.neuron.period=}')
print(f'{block.neuron.decay=}')  # This is also randomly initialized around 0.9. If (min_decay, max_decay) was given, the range would be used

Here are the outputs:

block.neuron.shape=torch.Size([192])
block.neuron.frequency=array([0.00984336, 0.01004609, 0.01042829, 0.01054274, 0.01046093,
       0.01047461, 0.01077368, 0.01085227, 0.01113035, 0.01095885,
       0.01137894, 0.01131909, 0.01173683, 0.01168155, 0.01179497,
       0.01180994, 0.01206946, 0.0123099 , 0.01235787, 0.01262912,
       0.01254913, 0.01290786, 0.01310807, 0.01336274, 0.01356322,
       0.01341275, 0.01378871, 0.0136793 , 0.01404417, 0.01419258,
       0.01423964, 0.01436238, 0.01477239, 0.014853  , 0.01505426,
       0.01524952, 0.01540366, 0.01559169, 0.01582442, 0.01594398,
       0.01605728, 0.01622539, 0.01649282, 0.01669168, 0.01693295,
       0.01709061, 0.0175258 , 0.01778587, 0.01777152, 0.01813289,
       0.01819071, 0.01845139, 0.0188488 , 0.01907342, 0.0192318 ,
       0.01933173, 0.0196333 , 0.01997475, 0.02024243, 0.02042327,
       0.02079177, 0.0208995 , 0.02114983, 0.02140033, 0.02170203,
       0.02202907, 0.02227969, 0.02258362, 0.02270212, 0.02290644,
       0.023146  , 0.02351366, 0.02375229, 0.02415418, 0.02438881,
       0.02471387, 0.02505367, 0.02514426, 0.02542537, 0.02609852,
       0.02609301, 0.02640595, 0.02704474, 0.02733648, 0.02738706,
       0.02803378, 0.02812409, 0.02866153, 0.02876674, 0.02908104,
       0.02947369, 0.02991663, 0.03045386, 0.03078127, 0.03119136,
       0.03157627, 0.03199824, 0.03207024, 0.03272368, 0.03282884,
       0.03354429, 0.03380157, 0.03411623, 0.03475824, 0.03488756,
       0.03543048, 0.03594438, 0.03636637, 0.03688712, 0.03725969,
       0.0377083 , 0.0381563 , 0.03843566, 0.03898957, 0.0393877 ,
       0.03992908, 0.04035903, 0.04097418, 0.04131022, 0.04197897,
       0.04267116, 0.04285327, 0.0435793 , 0.04406294, 0.04456994,
       0.04500441, 0.04570502, 0.04615118, 0.04680765, 0.04736342,
       0.04793086, 0.04854104, 0.04916785, 0.04960755, 0.0503385 ,
       0.05095676, 0.05146614, 0.05200263, 0.05265861, 0.0534709 ,
       0.05411601, 0.05454329, 0.05557415, 0.05599336, 0.05670235,
       0.05731749, 0.05799726, 0.05893481, 0.05942123, 0.06048873,
       0.06093501, 0.06198187, 0.06268141, 0.06320833, 0.0640972 ,
       0.06459332, 0.06554925, 0.06648534, 0.0669803 , 0.06799451,
       0.06887468, 0.06963775, 0.07046501, 0.07152081, 0.07241686,
       0.07296717, 0.07411301, 0.07508572, 0.07562877, 0.07693382,
       0.0777557 , 0.07850742, 0.07956718, 0.08056713, 0.0813039 ,
       0.0823069 , 0.08345232, 0.08437512, 0.08532576, 0.08670726,
       0.08745664, 0.08849702, 0.08984213, 0.09081106, 0.09194522,
       0.09297489, 0.09435079, 0.09534572, 0.09621868, 0.09765049,
       0.09901252, 0.09995103], dtype=float32)
block.neuron.period=array([101.59138  ,  99.54121  ,  95.893005 ,  94.851974 ,  95.59379  ,
        95.46897  ,  92.8188   ,  92.14664  ,  89.844406 ,  91.25044  ,
        87.88168  ,  88.34631  ,  85.20191  ,  85.605095 ,  84.78193  ,
        84.674446 ,  82.853714 ,  81.23542  ,  80.920105 ,  79.182076 ,
        79.68677  ,  77.47216  ,  76.28888  ,  74.83496  ,  73.7288   ,
        74.55592  ,  72.52309  ,  73.103134 ,  71.20395  ,  70.459335 ,
        70.22652  ,  69.62636  ,  67.69388  ,  67.326454 ,  66.42639  ,
        65.57585  ,  64.91965  ,  64.13673  ,  63.19348  ,  62.719597 ,
        62.277058 ,  61.631805 ,  60.632435 ,  59.910076 ,  59.056465 ,
        58.51165  ,  57.05874  ,  56.224407 ,  56.269814 ,  55.148407 ,
        54.973114 ,  54.19647  ,  53.05377  ,  52.428993 ,  51.997208 ,
        51.72843  ,  50.933884 ,  50.063198 ,  49.401176 ,  48.963745 ,
        48.095947 ,  47.848034 ,  47.28171  ,  46.728252 ,  46.078648 ,
        45.394558 ,  44.883923 ,  44.279884 ,  44.048744 ,  43.655834 ,
        43.20401  ,  42.528477 ,  42.101204 ,  41.40071  ,  41.00241  ,
        40.463108 ,  39.914307 ,  39.77051  ,  39.3308   ,  38.316357 ,
        38.324436 ,  37.87025  ,  36.97577  ,  36.581158 ,  36.513596 ,
        35.671257 ,  35.55671  ,  34.889977 ,  34.762367 ,  34.38667  ,
        33.92857  ,  33.42622  ,  32.83656  ,  32.487286 ,  32.060158 ,
        31.66935  ,  31.251717 ,  31.181559 ,  30.558914 ,  30.461023 ,
        29.811337 ,  29.584427 ,  29.311558 ,  28.770157 ,  28.663513 ,
        28.224283 ,  27.820765 ,  27.497932 ,  27.10973  ,  26.838657 ,
        26.519361 ,  26.20799  ,  26.017504 ,  25.647888 ,  25.388638 ,
        25.044405 ,  24.777603 ,  24.405615 ,  24.207085 ,  23.82145  ,
        23.435034 ,  23.335442 ,  22.946674 ,  22.694809 ,  22.436646 ,
        22.220045 ,  21.879433 ,  21.66792  ,  21.36403  ,  21.11334  ,
        20.863386 ,  20.601124 ,  20.338493 ,  20.158222 ,  19.86551  ,
        19.624483 ,  19.430252 ,  19.229795 ,  18.990248 ,  18.701761 ,
        18.478819 ,  18.334063 ,  17.993979 ,  17.859259 ,  17.635954 ,
        17.446682 ,  17.242193 ,  16.967901 ,  16.829    ,  16.532005 ,
        16.410927 ,  16.13375  ,  15.953692 ,  15.820701 ,  15.601305 ,
        15.481478 ,  15.255704 ,  15.04091  ,  14.929763 ,  14.707069 ,
        14.519124 ,  14.360028 ,  14.19144  ,  13.981944 ,  13.80894  ,
        13.704793 ,  13.492908 ,  13.318111 ,  13.222481 ,  12.998184 ,
        12.860794 ,  12.737649 ,  12.567996 ,  12.41201  ,  12.299533 ,
        12.14965  ,  11.98289  ,  11.851835 ,  11.7197895,  11.53306  ,
        11.434238 ,  11.299815 ,  11.130635 ,  11.011875 ,  10.876041 ,
        10.755591 ,  10.598746 ,  10.488148 ,  10.392993 ,  10.240604 ,
        10.099733 ,  10.0049   ], dtype=float32)
block.neuron.decay=array([0.8933508 , 0.8916299 , 0.9067824 , 0.9004164 , 0.9033558 ,
       0.8923451 , 0.8989385 , 0.8996692 , 0.8986781 , 0.90774035,
       0.90772325, 0.9072362 , 0.9072185 , 0.9001217 , 0.8944858 ,
       0.89791244, 0.9001035 , 0.9020509 , 0.89298815, 0.8983618 ,
       0.9008149 , 0.90055203, 0.8961323 , 0.9010185 , 0.9024781 ,
       0.89268553, 0.89560455, 0.9061477 , 0.9085798 , 0.89582634,
       0.9070991 , 0.89434546, 0.9078063 , 0.9004457 , 0.9043576 ,
       0.8928189 , 0.8989411 , 0.900156  , 0.89916015, 0.9047961 ,
       0.90061766, 0.9040428 , 0.9079535 , 0.9020497 , 0.8988421 ,
       0.892938  , 0.90002924, 0.9014851 , 0.89264196, 0.901215  ,
       0.90366834, 0.90291274, 0.9028842 , 0.9060645 , 0.8967078 ,
       0.8952241 , 0.90873027, 0.8946794 , 0.90376365, 0.8931646 ,
       0.89879584, 0.8955854 , 0.9041845 , 0.9071206 , 0.89403594,
       0.90090775, 0.8915169 , 0.8912409 , 0.90726125, 0.8978698 ,
       0.90228885, 0.9054643 , 0.89655524, 0.8950337 , 0.9056325 ,
       0.89423525, 0.9065595 , 0.89913446, 0.9033098 , 0.89680904,
       0.89977914, 0.8950431 , 0.9076048 , 0.9028676 , 0.90732265,
       0.8941136 , 0.902783  , 0.9059498 , 0.90221566, 0.90863526,
       0.9058611 , 0.9085453 , 0.9050045 , 0.89202994, 0.8984466 ,
       0.90462315, 0.90830934, 0.8950942 , 0.9067268 , 0.8974851 ,
       0.8961392 , 0.90734196, 0.8955589 , 0.90421903, 0.90120023,
       0.8993723 , 0.9040577 , 0.89221805, 0.8969015 , 0.9073857 ,
       0.8970256 , 0.89821184, 0.8979165 , 0.9023581 , 0.90531844,
       0.89969176, 0.9036572 , 0.9002779 , 0.89441234, 0.9054205 ,
       0.90784025, 0.90730464, 0.8952682 , 0.9071071 , 0.90372   ,
       0.8949999 , 0.9017377 , 0.9052228 , 0.8938811 , 0.9009183 ,
       0.898764  , 0.90324974, 0.8915966 , 0.8925265 , 0.90110016,
       0.9022596 , 0.9047301 , 0.90719956, 0.9068109 , 0.9022498 ,
       0.9070308 , 0.89683276, 0.8986702 , 0.89727217, 0.8971029 ,
       0.89538866, 0.8986161 , 0.89341   , 0.8922341 , 0.8979945 ,
       0.9039332 , 0.9086901 , 0.9013859 , 0.8964841 , 0.89597404,
       0.8998211 , 0.8987608 , 0.8976969 , 0.90557945, 0.8980646 ,
       0.901033  , 0.903777  , 0.9059729 , 0.90052414, 0.89889276,
       0.894637  , 0.89179945, 0.9059903 , 0.8959417 , 0.9059751 ,
       0.9079302 , 0.9004906 , 0.9006873 , 0.8922479 , 0.9051235 ,
       0.8938314 , 0.90100706, 0.89812547, 0.892938  , 0.9043685 ,
       0.90323496, 0.90888023, 0.9078179 , 0.9028179 , 0.90790623,
       0.8937584 , 0.9016804 , 0.90214956, 0.9059346 , 0.89399546,
       0.90867054, 0.9085828 ], dtype=float32)

If you need further customization, you can directly access the sin_decay and cos_decay of the neuron and customize your neuron settings.

[zeinebCh]

Hi @bamsumit,

Thank you very much for the detailed answer.
For the period array: I could only set a min and max value? Could I have an array of periods where I can specify 192 periods as an input?
As for the unity the period is in seconds right? If my frequency is in Ghz, my period will be very small and will not satisfy the condition of period>4.

I used to firstly calculate the internal dynamic of the neuron (re, im) and then generate the spikes to extract the relevant features in my date. In this case it is done internally in block function right?
I used to do the following:
re,im = self.neuron.dynamics(input)
sp = slayer.spike.complex.Spike.apply(
re, im,
spiking_threshold,
1, # tau_rho: gradient relaxation constant
1, # scale_rho: gradient scale constant
False, # graded_spike: graded or binary spike
0, # voltage_last: voltage at t=-1
1, # scale: graded spike scale
)

Another question: My input data is now real data should I convert it to complex data or does it also accepts real data?

Thank you very much for your support.

[bamsumit]

For the period array: I could only set a min and max value? I could not have an array, that I can specify of size 192 as an input period?
As for the unity the period is in seconds right? If my frequency is in Ghz my period will be vey small and will not satisfy the condition of period>4.

In the current API, you can only set one value or (min, max) value. The unit of period is discrete time-steps. In real units, it is going to be period / sampling_frequency
For a signal in 1GHz range, you sampling frequency must be more than 1GHz which translates to a time-step less than 1ns. This is too small if you are considering real-time processing on a neuromorphic hardware. If you are not looking at real-time processing, then it is fine to choose a very small sampling frequency.

Like I said before, if you want more customization, you can directly set the sin_decay and cos_decay as you need after the first initialization.

The variable out contains the spikes generated from the input_data right?

Yes

I thought I must firstly calculate the internal dynamic of the neuron (re, im) and then generate the spikes to extract the relevant features in my date. In this case it is done internally in block function right?

Yes, it is done internally. You typically do not have to run dynamics separately.

Another question: My input data is now real data should I convert it to complex data or does it also accepts real data?

The complex synapse accepts real data and produces real and imag weighted sum. This is then taken by complex neuron. That is the workflow.

[zeinebCh]

I really appreciate your help. Thank you very much.