Home » Non-linear Boundary in Deep Neural Network

Non-linear Boundary in Deep Neural Network

by Online Tutorials Library

Non-linear Boundary in Deep Neural Network

In the perceptron model, we used a linear model to classify two regions of data. Realistic data is much more complex and not always classify by a straight line. For this purpose, we need a non-linear boundary to separate our data. Perceptron model is work on the most basic form of a neural network, but for realistic data classification, we used Deep Neural Network.

When our model is unable to represent a set of data, we use a non-linear model instead of it. The non-linear model is used in the following situation

Non-linear Boundary in Deep Neural Network

In the above image, there is a curve which perfectly classifies our data but how we can obtain this curve. For this, we combine two perceptrons into a third one. It’s quite typical to understand so that for better understanding, we take two linear models and combine them to form a single non-linear model.

Non-linear Boundary in Deep Neural Network

It is clear from the above pictures that both models are unable to classify our data. Following are some steps which are used to form a non-linear model from two linear models:

Step 1:

Let start by combining each linear model to form a non-linear model. If we have two linear models, then by combining them the resulted model will look like

Non-linear Boundary in Deep Neural Network

The output model is a linear combination of the two other models.

Step 2:

Now, what we have to do, we will treat both the linear model as an input node which contains some linear equation. We denote our first model as x1 and second as x2.

Non-linear Boundary in Deep Neural Network

Step 3:

In our next step, we will multiply our model with some weight such as w1 and w2, and we also consider bias so that we will also treat bias value as a node.

Non-linear Boundary in Deep Neural Network

Step 4:

Now, everything is added to obtain a linear combination. For this purpose, we will apply the sigmoid activation function, which gives us the expected curve.

Non-linear Boundary in Deep Neural Network

Step 5:

We will mathematically multiply all the nodes with its weight value such as w1=0.4, w2=1, and b=0.5 and apply the sigmoid then the resulting curve will be as follows:

Non-linear Boundary in Deep Neural Network

Step 6:

In the second linear model x2, if we will take the weight value 3, the resulting model will give us an unexpected curve, and it looks like a

Non-linear Boundary in Deep Neural Network

From step 5 and step 6 it is clear that the model which is created with weight value 1.5 and 1, classify our data best rather than the model which is created with weight value 1.5 and 3. Weight defined the non-linear boundary of the non-linear model.

The process of combining two linear models to form a non-linear model is not so simple. It is quite essential to understand the structure of the Neural Network to implement the non-linear boundary of deep neural network.


You may also like