All right.
So in this lesson what we're going to talk about are some of the problems that are associated with neural
networks.
Some of the disadvantages that these types of models have.
We've already talked about their black box nature.
We don't really know why it is that a neural network might give a particular output.
And this is actually very important when we care about the rationale for a particular decision that
is made by neural network.
So imagine that a neural network is being used in the legal system.
And this computer programs job is to set the amount of bail or the amount of jail time for a particular
person.
Now surely in this situation you'd want to be able to ask wait well why is this guy getting two years
and this other guy getting 10 years.
This is just one example of a situation where an in transparent and intractable model is a real disadvantage.
But in this lesson I want to spend time talking about the other really big disadvantage of big complex
neural networks that second really big disadvantage is cost.
And this is a really weird one to wrap your head around at first because what kind of costs are we talking
about here.
This cost actually comes in two forms.
The first is the amount of data required and the second kind of cost is the amount of compute required
to train a neural network.
And when I say compute what I'm talking about is the amount of hours a model has to be trained on something
like a GP.
So what you're gonna see is that these two costs are related in a minute and they all kind of go back
to the structure of a neural network.
Now with any kind of model a pretty good proxy for the degree of complexity in the model are the number
of parameters.
So in this slide for this example neural network where we're just estimating the weights how many parameters
do you think we actually need to estimate in total.
Because that's our training this neural network what we're doing is adjusting the weights for our different
connections.
Right.
So let's work out the total number of weights that we've got on the very left.
We've got six input nodes and then we've got the six nodes for our first hidden layer.
So that's six times six or a total of 36 connections right and then we can do the calculation for the
rest of the layers too.
So between the second layer and the third layer we've got six times five plus five times six for the
third and fourth layer and then for the output layer we've just got four times one adding this all up
we've got about 90 different connections.
So 90 different weights that we need to estimate.
The issue is that the more parameters you have the more data you need to figure out how to tweak each
of them.
So a real life example would be something like baking the perfect cake like the one that your grandma
used to make.
So you try this and at the end you find out it's not tasting quite the same.
It's not just quite right.
But where did things go wrong.
Did you use too much flour or too few eggs or not enough sugar.
Or did you leave it in the oven too long to get this cake tasting just right the way your gran used
to make it.
You might have to experiment quite a bit and bake it a few more times to get it just right.
Tweaking each of these little parameters along the way.
Now of course the more complex the problem the more parameters you have the more you'd have to experiment
to figure out what their value should be.
And this is really a similar story with the neural network or any model the more complex the model the
more data it needs to chew through in order to get sensible parameter estimates now the question you
might ask at this point is well how much data do we need to train this network.
And the answer is It depends.
But in the industry people tend to use a rule of thumb.
We probably need about 10 times the amount of data as we have parameters that we need to estimate.
So in this case we've got 90 parameters so we'd need around 900 data points now say that we were working
with images right instead of our six input nodes.
We have a small image that we want to supply.
And this image is going to be 25 pixels by 28 pixels.
And this means that this black and white image would actually have around 700 inputs because 25 times
28 is 700 now even if we keep the rest of the network the same in that first hidden layer we've already
got four thousand two hundred different connection weights.
The point I'm trying to make here is that the number of parameters goes up massively with the size of
the network.
So if we have these 700 inputs and we just add one extra neuron to that first hidden layer all of a
sudden we're jumping from four thousand two hundred different weights to 4900.
And as the number of parameters goes up this network needs to be fed with more and more data to reliably
estimate all these weights during training.
Now in practice depending on the field of application it's actually not uncommon to have between 100000
or a million or even 10 million parameters in a neural network that need estimating and at that point
the amount of data that you need to get your hands on actually starts to become enormous.
And in addition it has to be an enormous amount of good quality data.
And this is why the biggest and most complex neural networks always have the backing of large organizations
right like Google Facebook Microsoft or a government institution.
Now the good news is that there's a lot of good data out there for the likes of you and I.
We're gonna be able to build some pretty interesting things with the resources that we have available.
So suppose that we manage to actually get a hold of say half a million data points to train the ultimate
cat detector.
Now what.
Well we have to process this data.
We have to store the data we have to clean the data and then eventually we have to train our neural
network.
And it's at this point that the second part of the costs hits you to that was talking about because
training means a lot of calculations.
It means making a lot of predictions estimating a lot of losses at each node in the neural network and
then making a lot of weight adjustments for every single piece of training data that you feed through
the network.
So there is an incredible amount of computation going on for a large neural network.
There's actually so much computation involved during training that your laptop just isn't going to be
up for that job anymore even with a desktop computer.
You're actually going to struggle so at this point you're going to need some very serious hardware.
And by that I mean you're probably going to need some GP use that's plural GP use.
One might not even be enough.
Let's actually go on Amazon and let's take a look at what top of the line GP use going for these days.
So if I fire up Amazon dot com and I type in say and video g force GTA X I don't know.
Two thousand eight hundred.
Right.
That's probably one of the most popular gaming graphics cards these days.
Taking a look here we can see that's about seven hundred and thirty dollars.
It's pretty hefty.
But I've actually got bad news for you this pricey graphics card might be fine for gaming at a four
K resolution but in the world of deep learning researchers actually tend to train their models on the
likes of and videos Taz like Katie.
How the video Tesla Katie actually cost about two and a half times as much.
And this graphics card actually does about eight more operations per second than that top of the line
gaming graphics card.
This is what you'd actually find in a data center.
So needless to say this is a pretty expensive and buying 10 of these just probably isn't going to happen.
Maybe we can.
I don't know.
But a really nice letter to Santa and hope for the best during Christmas.
But these graphics cards are pretty pricey and the reason is that people are buying these because of
cryptocurrency mining.
We've got bitcoin to thank for that.
And it's pretty funny how videos share price has been pretty much tracking the bitcoin value for the
past two years.
So that's the world we live in nowadays but not all hope is lost because there's always the cloud.
Maybe we can use somebody else's computer to train a large neural network.
And you know if this isn't even such a bad solution going to someone like Amazon or another company
like Floyd hub or Microsoft I'm sure it's going to be a lot cheaper than buying your own GP use.
The last time I checked the price per hour varied between 50 cents and about a dollar.
So training a model that requires 100 GP you hours will set you back 50 to 100 dollars.
But what if your model requires 10000 gp hours to train.
Well it still might be worth it depending on your situation punt long training times do mean that you're
going to be probably a little more thoughtful when it comes to iterating and tweaking your models to
retrain it.
So what I'm saying is that Amazon NWS and these other services they lower the barriers of entry for
sure.
And it's still probably cheaper than buying your own GP use and there's even some good deals around
these days but even cloud services can get very expensive very quickly.
And this brings me back to the questions I asked earlier can you use a neural network to solve almost
any problem.
Yes.
Should you use a neural network to solve every problem.
Mm hmm.
Probably and hot the data requirements training time and the amount of computation it takes is why many
people try to solve simple problems with simple methods.
And even though neural networks can pretty much do anything it's not always the best approach.
When you factor in the amount of data required the amount of compute required and the lack of transparency
at the end as a machine learning professional what you really want to do is use the right tool for the
job.
But I'm not going to leave this lecture on such a sad note because neural networks are awesome and there
is a way forward.
So as part of this cause we're gonna do a couple of things to make Neural Networks a bit more accessible.
First off we're gonna be sticking to simpler models and smaller datasets so that our computers can handle
the computation.
This is so we can learn the ropes and not have to pay a large cost.
But in addition there is a way that we can actually play with a GP you for free.
And so it is my pleasure to introduce to you the Google collab notebooks Google collab is actually based
on the Jupiter notebooks that we've been using all along but you can basically access it through your
browser.
You don't have to install anything all you need is a Google account and access to Google Drive and all
the calculations will then run on Google's servers.
All you have to do to open a collab notebook is to go to a drive dot google dot com thing go to new
go to more.
And then here you see collaborate story.
So you click on that and it starts a Jupiter notebook essentially.
And after your page is loaded you can go to runtime change runtime type and then here you see hardware
accelerator and you can change this to GP you.
So that's the first good news a free GP you hurray.
This means that we can do some pretty cool stuff regardless of how old our own personal computer is
because we can piggyback off Google's hardware.
And I read that when you select GP you hear you're in fact running it on one of those Tesla Katis.
The second piece of good news that I've got for you is that you don't always have to train your own
models.
That's right.
We can use other people's pre train models that they've decided to share in a place called A Models
zoo essentially some generous organizations and researchers have open sourced their models and more
importantly their weights for others to use.
One website where you might find some of these models is model zoo dot co.
And another is tensor flow hub.
Both of these are libraries of machine learning models.
So if I go to modules and then go to image classification you can pick and choose among these image
classification models.
And I know that some of these like nascent it have been trained on thousands of GP you hours.
I was actually listening to Regent Mongo explain on a podcast how they're best in class models for image
classification was trained on over a million images from Image net and it took them around 60000 GP
hours to train.
And that's great news for us right because it saves us having to rack up an enormous Amazon AWOL spill.
I still find it incredible that these kind of models are available for us to use.
And on that note let's jump into the next lesson so I can show you how.