When implementing a deep neural network, one of the debugging tools often used to check the correctness of the code is work through the dimensions and matrix involved.

In above figure, Capital L is equal to 5, i.e. not counting the input layer, there are five layers here, so four hidden layers and one output layer.

And so if you implement forward propagation, the steps will be: \( Z^{[1]} = W^{[1]}*A^{[0]} + B^{[1]} \\ A^{[1]} = g(Z^{[1]}) \\ Z^{[2]} = W^{[2]}*A^{[1]} + B^{[2]} \\ A^{[2]} = g(Z^{[2]}) \\ Z^{[3]} = W^{[3]}*A^{[2]} + B^{[3]} \\ A^{[3]} = g(Z^{[3]}) \\ Z^{[4]} = W^{[4]}*A^{[3]} + B^{[4]} \\ A^{[4]} = g(Z^{[4]}) Z^{[5]} = W^{[5]}*A^{[4]} + B^{[5]} \\ \hat{Y} = A^{[5]} = g(Z^{[5]}) \\ \)

Now this first hidden layer has three hidden units. So using the notation we had from the previously, we have that \( n^{[1]} \), which is the number of hidden units in layer 1, is equal to 3.

And here we would have the \( n^{[2]} \) is equal to 5, \( n^{[3]} \) is equal to 4, \( n^{[4]} \) is equal to 2, and \( n^{[5]} \) is equal to 1.

And finally, for the input layer, we also have \( n^{[0]} \) = n_x = 2.

So now, let's think about the dimensions of z, w, and x. z is the vector of activations for this first hidden layer, so z is going to be 3 by 1, it's going to be a 3-dimensional vector.

So I'm going to write it a \( n^{[1]} \) by 1-dimensional vector, \( n^{[1]} \) by 1-dimensional matrix, all right, so 3 by 1 in this case.

Now about the input features x, we have two input features.

So x is in this example 2 by 1, but more generally, it would be \( n^{[0]} \) by 1.

So what we need is for the matrix \( w^{[1]} \) to be something that when we multiply an \( n^{[0]} \) by 1 vector to it, we get an \( n^{[1]} \) by 1 vector

So you have a three dimensional vector equals something times a two dimensional vector.

And so by the rules of matrix multiplication, this has got be a 3 by 2 matrix because a 3 by 2 matrix times a 2 by 1 matrix, or times the 2 by 1 vector, that gives you a 3 by 1 vector.

And more generally, this is going to be an \( n^{[1]} \) by \( n^{[0]} \) dimensional matrix.

So what we figured out here is that the dimensions of \( w^{[1]} \) has to be \( n^{[1]} \) by \( n^{[0]} \).

And more generally, the dimensions of \( w^{[L]} \) must be \( n^{[L]} \) by \( n^{[L]} \) minus 1. So for example, the dimensions of \( w^{[2]} \) , for this, it would have to be 5 by 3, or it would be \( n^{[2]} \) by \( n^{[1]} \). Because we're going to compute \( z^{[2]} \) as \( w^{[2]} \) times \( a^{[1]} \) (let's ignore the bias for now).

So the general formula to check is that when you're implementing the matrix for layer L, that the dimension of that matrix be \( n^{[L]} \) by \( n^{[L-1]} \).

Let's think about the dimension of this vector b. The general rule is that \( b^{[L]} \) should be \( n^{[L]} \) by 1 dimensional.

So hopefully these two equations help you to double check that the dimensions of your matrices w, as well as your vectors p, are the correct dimensions.

And of course, if you're implementing back propagation, then the dimensions of dw should be the same as the dimension of w.

So dw should be the same dimension as w, and db should be the same dimension as b.

Now the other key set of quantities whose dimensions to check are these z, x, as well as \( a^{[L]} \) because \( z^{[L]} \) is equal to \( g(a^{[L]} \), applied element wise, then z and a should have the same dimension in these types of networks.