In this video, we will learn how to perform Upsampling and Downsampling in Biton.
We have already covered downsampling Upsampling in order to read lecture's downsampling means.
Converting our data from higher frequency data to some lower frequency data.
Which means converting monthly data and to easily data, converting daily data into weekly data or converting
any weekly data into monthly data.
So let's look at how to perform downsampling in Python.
We will be using the same dataset that we were using earlier, which is US airline miles data.
This data is available in the resources section of this video.
So let's first import the data.
I am already mentioning which column contains the data, and I am also mentioning the hiders are present
in the Post.
So let's look at top five rows of this data.
So here you can see this is the monthly data in the first stroke.
We have a data of Jan 1963.
And in the second row, we have the data of Fab 1963.
The dates columns are available in a column named Month.
And the values are available in the column.
Miles and miles and millions.
So let's first convert this data to quarterly data.
The function we are going to use is very simple.
So here we are creating a new data stream that is quarterly miles data stream.
And we are mentioning Miles, dear.
And we are using these simple function in resemble function.
You have to provide at least two arguments.
First, you have to provide the new frequency of your data.
So since we are going to generate quarterly data.
I have mentioned Cuvier.
And next, you have to mention the date column of your data.
So in our data, the data column is present in this month's column.
So here we are right on equal to then the name of the column.
So the name of the column is Month and then we need some aggregate function.
So in the quarter there are three values.
So suppose this is the first quarter.
These three values.
Now we need to use an aggregate function to summarize this data.
We can either take mean of these three values.
We can take max of these three values or minimum value of these three values, or we can take total
sum of all these three values.
So here we are planning to take mean of the values.
That's when we are using Daudt mean aggregate method.
So again, we are creating a new dedapper, importantly minus D.F. from our minds data frame.
Then we are using this simple function for aggregating data.
And we want to add new data.
To be, of course, a live frequency, the date column is the month column.
And we want to get the mean of the values.
So let's run this and let's look at first.
Five values of my dataset.
So this is the value.
You can see we have downsampled our data from monthly data to quarterly data.
Here we have the data our first quarter.
The mean value of these three values is around six six nine six point three three for the next quarter
to the mean value is this.
And so on.
So overall, we have reduced the frequency of forward data.
So, again, these are key words, you can use other key words as well.
So for annual, you can use it instead of you for weekly data, you can use the blue inside of Q and
so on.
So let's again convert this monthly data into yearly data.
So we are creating another data frame that is a total minus data thing.
We are taking value from minus the doubling.
And again, we are reassembling it at a new level.
So a sense for year or annual.
And the data column is the month column.
And here we want the sum of all the values.
So for the first year, we won some of all these values of Twellman in the next year.
We won some of all the twelve values of next year.
So let's run this.
Also, look at first five venues for decoupling.
So you can see we have the last day of 1960, three years, and we have the total minus of that year.
So these are the two examples of downsampling.
First, we have converted our monthly return to port elevator.
And here we have converted our monthly data and do and the old data that the sum of all the monthly
values.
Now, here is a list of all the arguments that you can pass.
So this is the list.
If you have a daily data, you can convert it into weekly data using W.S. as an argument.
We have already used quarter and a Q4.
You sense for quarter and a sense for here.
And you can also try the arguments as well for this data.
So if you notice in the last video, we also summarized our data using group by function.
So I guess when we were converting over monthly data into you alluded to here, we use grew by function.
So here we first created the user column.
And then we use this year for them to summarize this, Miles, M-m data and getting the mean value on
an yearly level.
So you can also use these simple and sort of these school functions to get the same desert.
So this is down something.
Next, we have Upsampling up seven Pling means increasing the frequency of your data.
So if you have monthly data and you want to deliver data.
Then you will convert your monthly return to daily data.
And for each man, you will have thirty 31 or 28 values depending on the month.
So decreasing the frequency of your data means downsampling and increasing the frequency means upsampling.
So for Upsampling also, we are going to use same minus data.
And here we are going to convert our monthly data into daily data.
The function is similar.
We will use three sample.
And earlier when we were down, something we were using quarterly or annually.
Here we are using B B sense for daily.
And we will use the same statement.
We have just changed the frequency from A to daily.
So let's run this.
Let's look first to define values of what you did say.
We are calling this new day to set up some minds to frame this on the first 35 values.
You can see earlier we only had 1963 zero one zero one.
And the next well, it was 1963 zero two zero one.
So this one of the first four values.
Now we have our own party, more values between these two dates.
So you can see the second row here is of 2nd January 1963 and so on.
And now this is the second value of what?
Your column, first Fab 1963.
You can also notice that for the newly created growth we have and they're and as values of minds.
So oral you can see that we have only created the structural forward and you the frame.
We have not fully values of miles in our new data.
So the next step of upsampling is to fill this empty values with some values using the already present
values.
So just look at this Faustman value here.
Starting with Lewis six thousand eight hundred twenty seven.
And the ending with Lewis, six thousand one hundred and seventy eight.
Now, how can we feel this ambivalence?
Now, it would make more sense to go straight line between these two values.
Six thousand eight hundred twenty seven and six thousand one hundred and seventy eight.
And feel all these values.
According to that straight line.
So suppose this value can be six thousand eight hundred and twenty two.
This well, you can be six thousand eight hundred and seventeen.
This value can be six thousand eight hundred and twenty and so on.
Then we reach this value.
Six thousand one hundred and seventy eight at the end of this value.
Basically we are trying to fit a linear line between these two values in that way.
These newly created values will make a lot more sense to us.
So let's see how to do that.
To fill this one news, we will use Interpol alert function.
We are creating a new data frame.
That is Interpol alerted Miles.
The Dufrene and we have used this data frame up simple miles the Dufferin and we are using Interpol
alert function and we want to fill this well use using the linear line between the available values.
So let's run this.
Let's again take a head of all the 35 values.
So now we have filled all this empty values with a linear line between the available values.
So you can see the first value is same six thousand eight hundred and twenty seven.
And the next value is six thousand eight hundred and six.
So this are Degreaser 21 units in the next role as well.
You can see there is a decrease of 21 units, 21 units and so on.
Then we reach this value.
Six thousand one hundred and seventy eight.
Here you can see there is an almost surgically look one where Luzier.
So we have separated the defense of these school values.
Six thousand eight hundred twenty seven and six thousand one hundred and seventy eight and coopetition
equally spaced values.
So the difference between A and IT two values will be around twenty one units.
Let's plot the graph husband.
So this is the graph, you can see that they run linear lines in between any two points in our data.
So just look at these two values.
So this may be a value of December 1965.
And this may be a value of January 1966.
We have that straight line between these two values to create interpolated values at daily level.
Now, you can also look at.
There are sharp edges on this maps.
This is due to the linear model that we are using.
We are just passing two values and we are growing the straight line between between those school values.
What we can do is we can also use polynomial function in sort of this linear function.
So if somewhere there is a decrease and then an increase, our model will try to fit a curved line and
sort of a straight line to smooth out the edges of this graph.
So let's look at how to do that.
So earlier we were using mentor to equate to linear, since we wanted to fit linear line between the
existing points.
So now to fit a polynomial or stein curve, we can use method, equal this plane.
And then we can provide the order of that plane.
So if if we want to offer TopCoder decline, we can use all that equal to do.
If we want to tell Kubic line, we can use all that equal to three and so on.
So we will be using the same function interpolate function just instead of mentally.
What do you mean yet?
We are using metallic plug this plane and then we are mentioning the order of that plane.
Let's for this as well.
We are saving over newly created be detrimental poly interpolated miles to frame.
Let's look at.
No.
You can see, no, there are no edges on this graph.
We have a splain which is smoothing out this edges.
You can also compare the values of this newly created plane with Lina did nothing, so.
So you can see the difference between any two values at the start is larger than the difference.
The friends of Anita will lose a day or so of our interpolate model is trying to fit a curve and remove
edges from our graph.
Now, let's look at what all these different aggregate functions are available and our reship method.
So you can see earlier we were using mean, but you can also use some.
You can also use men.
You can also use Macs.
So all these options are available.
You can look at all these values and try this parameters on your own.
Thank you.