In this video, we will learn how to do feature engineering for daytime data.
Feature engineering is one of the most important step for any forecasting related machine learning project.
Feature engineering means modifying, billeting or combining existing raw features from our data to
create some new features.
These newly created features will help us in the Hansing the performance of Overfocusing more than.
Let's first start with creating new daytime related features from our date column.
We are going to use the same dataset that we imported last time we named it B.F., too.
So let's look at the content of this data frame.
So here in our first column, we have the dates and in the second column we have, but.
So we have the same data.
You know, at first column and numeric data in the second column.
Now, suppose if I just want to create a column for here.
So here the year is 1959.
So I want the column that says 1959 as the year value in next column.
Similarly, for all these dates, I want a and column for mentioning that year.
So how to do that?
First, let's create another day.
Daphne, that is feature length of time.
We are using B to copy Manteau to do this.
Now, here we can use the same like properties to do this.
So if you select your column that contains a date and if you like dot duty.
And after that dot the year or dot man, you will get corresponding either man or day for that date.
For example, here, if I am creating a new column here and I am writing B F to date dot booted out
here, I will get the ear information in my new column.
Which is it?
Similarly, if I want to extract the month information.
So for example, for all these dates, the month is one.
And if I want this information in another column, I can write this.
I'm creating a new column which is Month.
And again, I'm using the F to date.
And then the date time like features.
And in that I am calling month.
So if I on this I can do some Leard thing if I want a separate column for days as well.
So here I want another column with zero one zero two zero three which contain the days.
If I run this and if I take the first five values of my new data frame that this features, you can
see that we are getting the desired result.
We have separated the year information, the month information and the days information from this combined
field to feed new fields.
That is year, month and days.
So this is the first week of being featured in the movie from a column that contains daytime FORTMAN
data.
We can create separate feeds for year, month, days.
Our mandate time and so on.
If you want to look at some more information on this, you can click on this oficial documentation of
Fundus.
Here you will see all the properties that are available in Thane Lake properties.
So we just used it months and days.
But there are other additional methods as well that you can use.
Now, the next types of feature like we want is the leg features.
So, for example, suppose for this rule number two, I want to get the values of the previous thumb.
So, for example, for this row three, I want the additional column which sees number 32 here, because
this is the value for the previous month.
Similarly, for the second row, I want the value of 35 since 35 is the value for the previous month.
So this type of features are Kollek features, and these types of features are very important for any
forecasting problem.
So let's see how to clear this leg features spiriting leg features is very simple.
We can use dot show method of pandas.
So here I am creating a feature name, leg one.
And again, I'm using the birth values and I want to have a birth value of the previous time period.
So to do this, we can use Dortch method.
We can just call the column for it, the values are needed and then we can use dot shift.
And in the beckert we can mention the difference of the time period.
So here we want the information of previous time period.
That is the differences one.
That's right.
If I write Larcher and then record if I write one, I will get the value of previous time.
Now to get the value off.
Same day last year.
We can use 365 since there are 365 days in a year.
So if we do a large ship and then record, if we mention 365 days, we will get the value that was there.
At the same day.
But the last fear, and that is one more thing, if you see there is no previous month value available
for this first store.
So this is the first entry in our data.
So in this case, the output will be there.
And that is not a number.
Similarly, if we are creating a lag feature with one year of delay for the first 365 values we will
get.
And in this variable.
So we have created this pool Legba variable one with the previous day value.
And another one with the last year value.
So let's look at our data frame once again.
You can see now we have two more variables.
Leg one, leg two.
As I said earlier, for the first record we have.
And then since the previous day, rally was not available.
And for the leg, two variables we have.
And then for the first 365 records, because the previous year, values are not available for the first
365, because here you can also see that we are getting to define value in the second row since the
birth on of first row was 35.
You can see that there is a leg off on one day within this, but when you like, when you.
So this is how we create leg features in Python.
Using pandas.
Now let's look at some other features.
The first one is the window feature.
So suppose I want another variable here where I want the average of these two aloose.
And here I want the average of these two values.
So we want the average of this one.
But data and the last month.
But data.
How can you do that?
For this, we have to use the ruling method.
So the method is on most similar to creating lag variables here.
We have to provide not groaning method.
We have to mention the number of periods we want to consider.
So if I just want to consider only two periods.
So suppose here I want the average of these two I lose my window size will be equal to two.
If I want to consider three butut, I would no size will be equal to three.
Now, after this, I have to use another aggregating function.
So since I have three values.
So if you are considering number of buttons, we have 35, 32 and 31, Lucía.
Now we have to mention what we want from these values.
We can either take a mean of all these values.
We can also take Max Trelew of these values.
We can also take minimum of these three values.
So now in this case, I also have to provide the aggregating function that we are going to use.
So let's create a mean variable in which we will be the mean of disparate value.
And the last filtered value.
So again, we are creating another variable in our detasseling features with the name Gool underscore
to me.
And here we are again taking B.F. to but data.
We are setting the ruling window of two periods and then we are taking them mean of these two values.
So let's clear this and let's look at our new data.
So you can see we have another variable that is roll under.
For me, again, the Foreswear Lewis.
And then since the previous period, data is not available for the first record.
For those second reward, we are getting a value of thirty three point five.
Thirty three point five is the average of 35 and 32.
Similarly, for the third record.
We are getting a value of 31, which is.
An average of Tegretol and 30 and so on.
So here our window size was off to the courts and I was aggregating function was mean.
Similarly, let's create another variable where we want to take the maximum value of previous three
periods.
So we want to consider this current period.
We want to consider previous.
We did.
Also, we want to consider previous to previous.
So for example, for this third record, I want the value of 35 since 35 is the maximum of 35, 32 and
30.
So I want to consider three time periods and I want to get the max out of those three.
Again, I am creating another variable that is Roll. underscore Max.
We are using D.F. two but column.
And in this case, my ruling window is of three units.
So I'm using darkling methods that we know equal to three.
And as an aggregating function, I am using max method.
So here I don't want the mean of these three were loose.
I want the maximum of these three values.
Let's find this.
Let's look at the word frame once again.
You can see this is our data.
We have another column for Roll Underscore, Max.
For the first two records, the value is.
And then since this time, the window sizes three.
And even for the second record, we don't have the three values to perform this operation.
So for the first two courts, Will Lewis and then and from the third record, we are getting our desired
result for the third record.
The values which we are considering is 35 prior to doing 30.
The maximum of all these three values are pretty face.
So that's why we are getting to the five year for this fourth record.
The values we are considering are today to 30 and 31.
The maximum of all these values is to.
That's what we are getting at here for this last record.
That is the record.
The maximum value is 44.
That's why we are getting 44.
So you can use mine as well, Max, as well, or minimum or average value as well.
Here, if you want some more details about this, you can again click at this link to look at the official
documentation of Pinder's.
Now, suppose we don't want to use a window, but we want to consider all the values from the setting
of our data.
For that, we can use expanding features now in expanding feature.
We don't give a window size.
We just consider all the values from the starting in that record.
So, for example, if I want the maximum value of birth till 1959, I will consider if I went 32 to
get the maximum of these two aloose.
If I want to get the maximum value till 1959, 01 03, I have to consider these three values.
For creating these type of records, we are we are considering all the values that we have to use the
expanding method of pandas.
So how to use that expanding is similar to ruling in ruling.
We have to create a regional parameter for the size of window.
And next, finding there is no need of that because we are considering all the values fill that time.
So again, we are creating another where we will that is expand on that score, Max.
And here we want to get the maximum value of.
But to date.
So I am using B, too.
And again, the data then I can use dot X funding method and as I aggregating function, I can use dot
max on this.
Let's look at the heart of first values here.
You can see here the maximum value of but first record is 35.
So we are getting 35 for the second quarter.
We are considering today if I went 32 again, the maximum is 35.
So we are getting 35 for the third record as well.
The maximum is 35.
Since we are considering these three values.
So we are getting 35.
Now, let's move this 52 court here.
We are considering all these five values.
We have 35 to do today, 31 and 44.
Again, the maximum is 44.
Let's say here the value is 44.
Now, for this record, say 10 through court.
We are consolidating all the values from the starting of pain.
So if I had to do 30.
So on 43, 45, 38, 27.
And the maximum out of all these values is for debate.
That's why we are using for every.
So that's all for featuring unity here.
We all know discuss how to do featuring an Iranian pattern.
We have not discussed the application, but in the later part of this course, we will learn how to
apply this feature in unity to generate some new data or some meaningful, useful data.
Thank you.