In "input" position to add the MP model neuron nodes, mark it as "input unit". The rest remains the same, so we have below: starting from this picture, we will weight w1, w2, w3 wrote, in the middle of the "line".
In the "sensors", there are two levels. Is the input layer and output layer respectively. "Input unit" of the input layer is responsible for data transmission, do not do calculation. The "output units" in the output layer requires the input of the previous layer. We call the level of the need to compute the computational layer, and calculate have a layer of the network is called "single layer neural network". There are some documents will be named according to the network with the number of layers, for example, called the "sensors" two layers of neural network. But in this article, we named according to calculate the amount of layers. If we want to predict the target is no longer a value, but it is a vector, such as [2, 3]. You can in the output layer to add a "output units". The figure below shows the single neural network, with two output units of output unit z1 calculation formula of the diagram below.
Figure 1 single layer neural network
Figure 2 single layer neural network (Z1)
As you can see, the z1 calculation with the original z and no difference. We know the output of a neuron can be passed to multiple neurons, thus z2 calculation formula of the following figure
As you can see, the calculation of z2 in addition to three new weights, w4, w5, w6, the other is the same with z1.The output of the entire network below
Figure 3 single layer neural network (Z2)
Figure 4 single layer neural network (Z1 and Z2)
Figure 5 single-layer neural network (extension)
If we look at the output calculation formula, we will find that the two formulas are linear algebraic equations. Therefore, the two formulas can be expressed by matrix multiplication.For example, the input variables are [a1, a2, a3] T (represented by a1, a2, a3) columns, represented by vector a; the left side of the equation is [z1, z2] T, represented by vector z.The coefficient is matrix W (the matrix with 2 lines 3 columns; the arrangement forms is same with the formula).As a result, the output formula can be rewritten as:
G = z (W * a);
This formula is the matrix manipulation of the neural network for calculating the later from the former layer.
Other than an input layer and an output layer, double layer neural network also adds an interlayer. During this time, the interlayer and output layer are calculating layers. We extend the single layer neural network above, add a new layer on the right side (containing only one node).Now, we increased to two weight matrix, we use superscripts to distinguish variables between different levels.For example, the ax(y) represents the xth node of y layer. z1，z2 turn into a1(2)，a2(2). The following figure shows the calculation formula of a1(2)，a2(2).
Figure 6 Single-layer neural network (interlayer calculation)
The method for calculating the final output z reaches the result by using of the a1 (2), a2 (2) of interlayer and the second weight matrix, as shown in the following figure.
Figure 7 Single-layer neural network (output layer calculation)
Suppose that the target we predict is a vector, and then it is similar to the value above, it only needs to add the node in the output layer.We use the vector and matrix to represent the variables in the layers. a (1), a (2), z is the vector data of network transmission. W (1) and W (2) is the matrix parameters of the network. It is as shown in following figure:
Figure 8 Single-layer neural network (vector mode)
Using matrix to express the whole calculation formula is as follows
G (W (1) * a (1)) = a (2);
G (W (2) * a (2)) = z;
Thus it can be seen that the use of matrix to express is not only very concise, but also not affected by the increasing of nodes (no matter how many nodes participate in the operation, both ends of the multiplication have only one variable). Therefore, in the tutorial of neural network matrix is widely used in description.It's important to note that, so far, our discussion of neural network structure has no mention of bias unit. In fact, existing of these nodes is the default. It is essentially only an unit containing storage capabilities with storage value always be 1. In each layer of neural network, in addition to the output layer, they will all contain such a bias unit as in linear regression model and logistic regression model.Bias unit are connected to all nodes of the later layer, we set the parameter values as the vector b, called bias. It is as shown in following figure:
Figure 9 Single-layer neural network (in view of the bias nodes)
You can see, it's good to offset the node recognition, because of its no input (no arrow pointing to it) in the previous layer. Some of the neural network structure will draw offset nodes significantly, some don't. In general, we will not draw a clear bias node.After considering the bias of a neural network matrix operations are as follows:
G (W (1) * a (1) + b (1)) = a (2);
G (W (2) * a (2) + b (2)) = z;
It's important to note that in the two layers of neural network, we no longer use SGN function as a function g, but use a smooth function sigmoid as the function g. We called the function g (active function) the activation function.In fact, the nature of the neural network is to fit real function relation between the characteristics and target through parameters and activation function. Beginners may think that Figure of the neural network structure is to realize these circles and lines in the program, but in a neural network program, neither the object “line” nor “unit” exists in this object. The most need for realizing the neural network is linear algebra library.