In this lecture, we will learn how to remove trend and seasonality from our data.
We have already discussed the fencing technique to remove Grendon seasonality.
Now we will implement the fencing in Python.
I am going to use the same dataset that we are using, which is US airline miles data and saving this
dataset into a data frame named Miles into score B.F..
Let's run this.
Let's take a look at first flight values of our dataset.
We update value in the column name month.
And we have the value of four time cities in the column name miles and millions.
Now, as we have discussed, in order to re-elect her, to remove train the applier defensing fourth
period equal to one.
So we will subtract value of all time cities from its lagged value of previous spread.
So let's also create.
And another them for leg one.
We will be using shift method with argument as one to create like one column.
Now again, let's look at the head.
You can see we have Miles data and then we have a leg, one data of mice.
So in the second period of leg, one for them, we have a value of mice of the first row.
So this leg, one called them, consists of leg one values of mice and men.
Call them.
Now, to remove Crane, we need to subtract this leg one value from this minus value.
The resultant series should not contain any grain.
After subtracting out defensing this legman value.
Now we can either subtract, these two lose, or there is a separate method or function available and
find us that can do that automatically.
So the function name is def essense for the forensic and in the function you have to mention the time
period for which you want to play defensing.
So if you want to defensing with leg one variable, you can write time period equal to one a few on
defensing with time period or fell.
That is defensing to remove seasonality.
You can write period equerry to do an.
So here I am really going to use this method instead of subtracting these two aloose.
So I'm creating another column.
That is my assignment.
Underscore def, underscore one.
And first, we have to mention the column name, which is which contains the value of all time cities,
which is Mice and Men.
And then we have to call the method def with period equal to one.
So let's run this.
Let's look at the word Dufrene once again.
So now you can see that VM another column minus.
I mean the one which is the difference of these two values Miles.
And it's lagman value since the legman value for the first row was and then the diffs value is also.
And then for the second row you can see that 649 is the difference of these two aloose for the third
row, nine zero six is the difference of these two values, Miles.
And it's legman, one value and so on.
So this newly created defensed series should not contain any trend.
Let's prove that by decomposing our mild cities and decomposing these different cities.
So first I am creating decompose plot or forward or regional my cities.
And then we will plot the decomposed plot for our different cities.
We know how to plot decompose plot.
First, we have to change the indexes.
So right now, the indexes are in numeric format.
We have to change this indexes to date.
Time for me.
That's why I'm changing my index to month values in this line of command in the second line.
I'm floating decompose plots using seasonal underscore decompose function from stack smarter.
I have already imported this in our previous lecture.
So if we are using this for the first time, you have to import it from is start Smardon.
I am passing Miles M-m data as values and we are going to blow 10 a..
More than I'm saving this result and result underscored it and then I'm floating desert.
Underscore to stand this.
So this is over.
Decomposed rough.
You can see this was a word or regional series V have consent.
Increasing trend, somewhat of a linear time.
And we have seasonality and these are the solutions.
So now let's plot the graph off over different cities.
Now, one thing to notice before plotting the decomposed graph for different cities is that our first
value of four different cities does not contain any value.
It's in there.
And so we have to avoid this particular record before alerting the cities.
So how to do that?
We will be using a lock.
We are using a lot because we want to apply condition on both rows and columns.
Earlier, we were only selecting the columns and we were selecting all the rows.
Now we want to select all the rows except this first row and we want to select this for them.
So we will be using I to segment our dataset.
We'll be writing, Miles, underscored D.F..
Then I look.
Then we will start what rose from the second row.
That's why we are writing one and then colon indexing.
It starts from zero.
So we're ignoring the rule that is present at the index zero, which is.
This role, we are selecting all the rules from indexing equate to one.
So you can see that index here is one.
So we are selecting all the rules with indexing.
Equate the one below the end.
And that's why we are using one column.
One is the starting point.
And since there is no end point after column, it will take all those iteration after the first observation.
So the first argument we are providing Rose, and in the second argument we are providing three because.
Because we want the fourth column.
So this is the zero column.
This is the first column.
This is the second column.
This is the third column.
That's VI v writing comma three.
Now, this is how we are selecting over the tough frame that we need to decompose.
And again, we are using an alliterative model.
Let's run this.
So this is their decomposed model for a word.
Different cities.
You can see earlier we had the linear trend.
Now there is no trend.
We are getting random values centred around zero in our trend line.
But you can see that we have successfully removed Craine from our data using like one defensing.
Now, one thing you can also notice is that seasonality is still present.
There is a seasonal pattern around here as well.
And in the difference, Florida as well, you can see there is some kind of seasonal pattern that is
occurring in each cycle.
So we have to remove seasonality also to remove seasonality.
We do defensing with leg off.
Well, it's so first legs, blood, the lifeblood for our date as well.
You can see there is a visible seasonality present in our data.
Even in our word difference, you can notice the seasonality for the first three periods.
You can clearly see there is a pattern that is repeating after every time interval.
Now, to remove seasonality, let's apply defensing for period equate to people will be using same bandos
function.
That is the method.
And this time we are applying defensing on our newly created column.
That is my step on that score one.
Let's apply this.
And.
Lord Elgin,
so does the plot.
After the debate, the Flensing First Free defends using Lego fun.
Now we're differenced using leg off twelve.
So at the starting work time series was like this.
After one defensing we removed the train.
And after w defensing we removed the seasonality as well.
Now let's look at the decompose graph of our newly created series and see whether we have remove tendencies
naledi or not.
So here I am calling this object as Cesari underscore.
See this?
So similarly, we will be using I love only and this time we will have around say first for L values
as and then N in our dataset.
So we would be starting of a tough dream from indexical to quell, that is a tough thing for Lou.
And since this is the fifth column, that is it does present at indexical before.
So this is zero.
One, two, three and four.
That's why I've been writing my quote before.
So let's load this with.
Be getting that dysfunction does not and does missing values because we are somehow including an end
and value in our data.
So he does the thing we applied defensing on this underscore one which already had one.
And then value.
So now I guess we have 13 rolls in with the last column is.
And then so I guess we have to change this 20 to 30.
Let's run this again.
This should work.
So you can see there is no crime right now and there is no seasonality as well.
So earlier we were getting a graph like this.
Remember, one year is around five units on our x axis.
And if you see, we still have the pattern.
But in the five month period, there is no seasonal pattern.
There is no permanent ups and downs.
This may be the one, but this may be December.
This may be January.
But there is no clear pattern saying that of winter or summer.
It's performing good or bad for our problem.
You can also look at the values of over Y-axis here.
So for seasonality, we have a value from minus 250 to 250.
Whereas when we had seasonality in our data, the values were between minus two thousand five one day,
two plus two thousand five hundred year, minus two thousand two plus two.
So you can see the scale as well.
Definitely, we have removed some seasonality from our data.
The scales has also changed from 2000 to 250, which signifies that there is a significant reduction
in the amount of seasonality of type of fencing.
This is how we remove seasonality and trend from our data to remove grey.
We apply defensing with period equal to one.
And we use the function of pandas and to remove seasonality.
We applied defensing of Berard equal to dwell on this on this detour and the data.
So that's all for this video.
Thank you.