In the last lecture, we discussed a single cell called Perceptron.
Now, in this lecture, we are going to extend the concepts that we learned in the last one.
I told you that a Perceptron takes in binary input that is one and zero and gives out a single binary
output.
But there is no logical reason to put this limitation.
We can easily extend this to any real input values.
So instead of having black and white only or zero and one only, we can have different shades of grey
as well.
That is, we accept any real value as input DVDs and trishaw still function in the same way.
Next, we will take a look at this equation of Perceptron will slightly modify it to Legia generally
used equation in this equation.
We are multiplying weight, adding these terms and comparing them with detail.
We will make a small change here, bring this threshold to the left and right.
This new term as B basically it means that we have B is equal to minus altricial.
People usually call this constant as the bias doesn't really make any difference.
But this is the mathematical representation of Perceptron, as you would find in most of the books.
Now, let's move on.
I look at the graphical representation of this function.
If you look at this graph, if the calculated value of this left part, that is summation of weight
multiplied by features, plus the bias, if the summation if this left part is less than zero, the
output comes out to be zero.
So you can see in the graph below zero, the output of the function is also zero.
When this left part is greater than zero.
This function suddenly activates and gives an output of one.
This type of function is called a simple step function.
This is one type of activation function activation functions are basically those functions which take
into account some able to racial value here.
The threshold value is zero.
And this function takes a sudden step at this threshold value, which is why it is called a step activation
function.
There are many other types of activation functions.
Most popular one is the sigmoid function.
It is a pictorial representation of how sigmoid function looks.
It is a smooth s shaped go.
It also has a minimum of zero at minus infinity and maximum of one at plus infinity.
But instead of having a step and raising, suddenly, this function rises gradually and continuously.
This function is also called logistic function and is also used in logistic regression, which is a
very basic classification algorithm.
Now, the sigmoid function solves a major problem that we have with this step function when we are training
our Perceptron using historical data to find the value of beads and treasure.
This step function is very sensitive to individual observations.
For example, when we are classifying fashion objects in our fashion m NASD dataset and that algorithm
is misclassifying a particular image of boots as trousers to rectify, this model will need to find
new ways and bias values.
This is where the problem comes.
Small change in dividend bias values will completely flip the output for a lot of the other observations.
This makes the step function very hard to control with sigmoid function.
The changes gradual, so it is easier to control the behavior.
Now, when we replace this step function with a sigmoid activation function, we call this new cell
as a sigmoid neuron or a logistic neuron instead of Perceptron.
Mathematically, a sigmoid function formula looks like this.
It is sigmoid.
A Z is equal to one upon one.
Plus it is to the power of minus C.
And if you plot this function on the graph, that is, if you have the Z on X axis and you calculate
the value of this function using this formula and plot it on the Y axis.
This is how this formula looks like.
Now we will replace the value of Z.
With these summation plus bias value.
So W.G. A, C plus B was the input to our activation function.
So we input this in place of zie.
So this is what the output of our neuron looks like.
It is one upon one plus exponential minus summation of words with features.
Minus B, if you calculate this value, it will always lay between zero to one and it will have a shape
like this.
So you can compare it with this step function also in step function.
We calculated output using this formula where we got zero.
If this summation was less than zero and we got one, if the summation was greater than equal to zero.
We have replaced astep with a sigmoid function.
This is a continuous function.
We do not need two parts to it.
So we just input the value of the Bluejays X and the bias to calculate the output, which is a continuous
function.
Now with this are artificial neural cell Israeli, which takes in any number of real value inputs and
gives an output between zero and one.
It is time to create an artificial neural network, which is basically a network of these individual
cells.
So just a brief recap of this class.
Initially, I said that we taken binary input and gave out one single binary output.
We replaced the input from binary to any real value, and we have replaced the binary output to a value
between zero and one.
So in this generalized form, we taken any input which have any real value and we get one output with
lies between zero and one.