In the last lecture we discussed a single cell called perception.
Now in this lecture we are going to extend the concepts that we learn in the last one.
I told you that a perception takes in binary input that is 1 and 0 and gives out a single binary output.
But there is no logical reason to put this limitation.
We can easily extend this to any real input values.
So instead of having black and white only or zero and one only we can have different shades of grey
as well.
That is we accept any deal value as input DVDs and threshold still function in the CMB.
Next we will take a look at this equation of perception.
We will slightly modify it to lead to generally used equation in this equation.
We are multiplying without adding these terms and comparing them with the threshold we will make a small
change here.
Bring this threshold to the left and right.
This new term as B.
Basically it means that we have B is equal to minus or a threshold.
People usually call this constant as the bias doesn't really make any difference.
But this is the mathematical representation of perception as you would find in most of the books.
Now let's move on and look at the graphical representation of this function
if you look at this graph.
If the calculated value of this left part that is summation of weight multiplied by features plus the
bias if this summation if this left part is less than zero.
The output comes out to be zero.
So you can see in the graph below zero the output of the function is also zero.
When this left part is greater than zero.
This function suddenly activates and gives an output of 1
this type of function is called a simple step function.
This is one type of activation function activation functions are basically those functions which take
into account some people.
Threshold value here the threshold value is 0.
And this function takes a sudden step at this threshold value which is why it is called a step activation
function.
There are many other types of activation functions.
Most popular one is the sigmoid function.
It is a pictorial representation of how sigmoid function looks.
It is a smooth S shape go.
It also has a minimum of zero at minus infinity and maximum of one at plus infinity.
But instead of having a step and raising suddenly this function arises gradually and continuously.
This function is also called logistic function and is also used in logistic regression which is a very
basic classification algorithm
not the sigmoid function solves a major problem that we have with this step function.
When we are training our perception using historical data to find the value of wheat and threshold this
step function is very sensitive to individual observations.
For example when we are classifying fashion objects in our fashion M NIST dataset and our algorithm
is mis classifying a particular image of boobs as trousers to rectify this a model will need to find
new weight and biased values.
This is where the problem comes.
Small change in the weight and biased values will completely flip the output for a lot of the other
observations.
This makes the step function very hard to control with sigmoid function.
The change is gradual so it is easier to control the behavior.
Now when we replace this type function with a sigmoid activation function we call this new cell as a
sigmoid neuron or a logistic neuron instead of perception.
Mathematically a sigmoid function formula looks like this.
It is sigmoid or z is equal to one upon one plus it is to the power of minus.
And if you plot this function on the graph that is if you have the Z on x axis and you calculate the
value of this function using this formula and plotted on the y axis this is how this formula looks like.
Now we will replace the value of Z with this summation plus bias value.
So WG a C plus B was the input to r activation function so we input this in place of Z.
So this is what the output of r neuron looks like.
It is one upon one plus exponential minus summation of words with features minus b..
If you calculate this value it will always lay between 0 to 1 and it will have a shape like this.
So you can compare it with this that function also in step function we calculated output using this
formula with regard 0 if the summation was less than zero and we got one if the submission was greater
than equal to zero we have replaced this step with the sigmoid function.
This is a continuous function.
We do not need two parts to it.
So we just input the value of WD X JS and the bias to calculate the output which is a continuous function.
Now with this are artificial neural cell is ready which takes in any number of real value inputs and
gives an output between 0 and 1.
It is time to create an artificial neural network which is basically a network of these individual cells.
So just a brief recap of this class.
Initially I said that we taken binary input and gave out one single binary output.
We replaced the input from binary to any real value and we have replaced the binary output to a value
between 0 and 1.
So in this generalized form we taken any input which have any real value and we get one output with
lies between 0 and 1.