Question about the relationship between input current and relu in the paper about snn_toolbox

[rbodo]

Hi

The relation

between input currents and activations holds only for the first layer (notice upper index 1 vs l in the equation below), because our implementation feeds the activation values as constant currents into the neurons of the input layer. In higher layers, there is no such clean relation anymore, which is why the equation

does not contain a reference to the activations a. Instead, we use this equation to establish a connection between a and the spike-rates r, because that will tell us whether the conversion is successful.

[victkid]

@rbodo Thank you so much for your reply. Does the "first layer" you mentioned refer to the input signal or the values after the first activation layer?

Please correct me if I'm wrong. From my understanding, a{i}{1} is the "input signal", for image, it is the normalized pixel value. z_{i}{1} is the spike train of the input. In the paper, it is defined as V_{thr} multiply by the input signal a_{i}{1}. We can also choose Poisson input to generate the spike train in the snntoolbox.

[rbodo]

Layer index 1 represents the first hidden layer (this is what I meant with "first layer"); index 0 would be the input layer, i.e. the neurons that encode the image pixel values a_{i}^{0}. After just having gone through the equations again, I think you have identified an inconsistency in the paper. The index in relation z_{i}^{1} = V_{thr} * a_{i}^{1} should in fact be 0 instead of 1; this equation holds strictly only for the input neurons. For the first hidden layer, the neuron input z would on average contain the same error terms as in Equ. 5. The overall result is unchanged, but the error terms we derive come in one layer earlier than shown in the paper.