For example, if I have the following architecture:
- Each neuron in the hidden layer has a connection from each one in the input layer.
- 3 x 1 Input Matrix and a 4 x 3 weight matrix (for the backpropagation we have of course the transformed version 3 x 4)
But until now, I still don't understand what the point is that a neuron has 3 inputs (in the hidden layer of the example). It would work the same way, if I would only adjust one weight of the 3 connections.
But in the current case the information flows only distributed over several "channels", but what is the point?
With backpropagation, in some cases the weights are simply adjusted proportionally based on the error.
Or is it just done that way, because then you can better mathematically implement everything (with matrix multiplication and so on)?
Either my question is stupid or I have an error in my thinking and assume wrong ideas. Can someone please help me with the interpretation.