weight matrix dimension intuition in a neural network

Question

I have been following a course about neural networks in Coursera and came across this model:

I understand that the values of z1, z2 and so on are the values from the linear regression that will be put into an activation function. The problem that I have is when the author says that there should be one matrix of weights and a vector of the inputs, like this:

I know that the vector of Xs has a dimension of 3 x 1 because there are three inputs, but why the array of Ws is of dimensions 4 x 3?. I can deduct that it has four rows because those are the weights w1, w2, w3 and w4 that they correspond to each ones of the values of a1...a4, but what is inside that array? Its elements are something like:

w1T w1T w1T
w2T w2T w3T
... ?

so when I multiply by x1, for example, I will get:

w1Tx1+w1Tx2+w1Tx3=w1T(x1+x2+x3)=w1TX

I have think about it, but I cannot really get a grasp about what this array contains, even though I know that at the end I will have a vector of 4 x 1 that corresponds to the values of z. Any help?

Thanks

Answer 1

As a thumb rule, weight matrix has following dimensions :

• The number of rows must equal the number of neurons in the previous layer. (in this case previous layer is input layer). So 3
• The number of columns must match the number of neurons in the next layer. So 4.

Therefore weight matrix = (3X4). If you take the transpose, it becomes (4X3).