Welcome back to Backspace Academy. In this lab we're going to be creating a 2
python application using the serverless application model. 3
Now the aws documentation on the SAM model can be a little bit 4
confusing, so what I'm going to do is I'm going to 5
guide you through a process that I use and it's quite simple. I don't use 6
SAM Init because that creates a hello world application that has a 7
whole heap of things that you don't really need. 8
What I'm going to do is start off with a github repository that has 9
simply two files, one of them will be the SAM 10
template, the other one will be our application python code 11
and from there we're going to use SAM build to build and prepare our 12
application and then SAM deploy to deploy that application quite simply. 13
So let's get into it. So let's go to the github repository. 14
The links for this are in the lab notes so make sure that 15
you download those lab notes and do all this yourself. 16
So first thing I would like you to do is to just star this. So just click up here 17
and star so that makes it easier for you to find it again when you 18
go to your your github account and also it makes it easier for other people to 19
find this code as well. So let's have a look 20
at it so we've got our template.yaml we'll open that up and have a look at it. 21
So the first thing there we've got is the template format version we need to 22
have that. AWS needs to know what what the format 23
version is. We also need to have transform and 24
serverless. So that instructs Cloudformation that 25
we're going to be using the serverless application model. So then 26
we've defined some resources. So the first resource here is called 27
create thumbnail and that is a lambda function. So the properties for 28
this lambda function is that the code is going to be stored 29
in a folder called code and our handler will be lambda function.py 30
and inside of that lambda function.py file will be a lambda 31
handler our runtime is going to be python 3.6 32
and we have our timeout and we're going to use an AWS 33
managed policy there which is aws lambda execute 34
and then we're going to define a s3 event that's going to 35
trigger this lambda function. So here we go it's a type s3 36
and the properties. So we need to know what the bucket is that's going to be 37
triggering this s3 event. So we're referencing a 38
source bucket so that source bucket is over here, 39
defined here and the event is going to be objects created. So 40
whenever we put a object inside of this or upload a file to this bucket 41
this source bucket then this s3 event will be triggered 42
and that will invoke this lambda function. 43
So the next thing we need to do is that we need to have a resource 44
source bucket and so all we've done here is defined it as a type we haven't given 45
a bucket name so cloudformation or the service 46
application model will create this bucket and it will give 47
it a unique name for us. So we also need a destination 48
bucket so what this application is going to be doing 49
is that we are going to upload images to a source 50
bucket that will invoke our Lambda function 51
and our lambda function will grab that image out of that source bucket 52
it's going to resize our image to a thumbnail 53
and then it's going to upload that thumbnail to a destination bucket and 54
then the invocation of that lambda function 55
will close. So we need to also have a destination 56
bucket and so what we want is this destination bucket to have the same 57
name as the source bucket but with -resized on the end. So what we're doing 58
is again we've got type s3 bucket but this time we're saying it 59
depends on source bucket. So we can't create this 60
destination bucket unless that source bucket exists. 61
So cloud formation will create the source 62
bucket once that's done it will create the destination bucket 63
and so the property of bucket name we need to join 64
the name of that source bucket with dash resized 65
on the end. So the way that we do that is we use the cloudformation function 66
join and the way that works is that first 67
first off we define what character we're going to put between these two 68
strings that we are joining. So we're not going to put anything between it they're 69
not going to put a dash or anything we've already defined the dash down here 70
as dash dash resized. So we're just going to have two inverted 71
quotes with nothing in the middle there. So that's just empty. 72
There's no space. There there's no dash there's nothing 73
and so the two strings that we're going to join the first one will be 74
referencing the source bucket. So the name of that 75
source bucket and then we're going to join that with -resized 76
So this destination bucket will be the same 77
name as a source bucket with -resized 78
on the end. So that's pretty cool. So what we'll do now is that we will go 79
back to our cloud 9 ide and we're going to clone this 80
repository. So what we do is first off iIm going to 81
copy, go back to the start of this and copy 82
the url of that repository and i'm going to do git clone 83
and then paste in the name of that or the url for that repository 84
and so there we can see it's created or it's cloned that repository. So we've got 85
our template.yml and we've got our code 86
folder there as well which is going to have our lambda function.py 87
Now before we go any further we need to make sure that the python or 88
the python version that is running on our 89
ec2 environment in cloud9 here that the python version is going to be 90
the same as that we define in our template.yaml 91
So in our template.yml we'll just open that up and now 92
we're defining python 3.6 so the current version of python is far 93
more advanced than that currently but cloud9 tends to 94
lag a little bit so i'm just going to do python 95
and --version and then we can see we're running python 96
3.6 on cloud9 even though the current version of python is 97
well advanced from that already. I think it's about 3.8 or something like that. 98
So we'll leave that as it is so 3.6 is fine. 99
So what we need to do now is that we need to create our requirements.txt file 100
and that will define all of the or we'll list 101
all of the modules that or packages that our application requires. 102
We need to first off jump into our into our code directory. So i'm just 103
going to cut and paste from the lab notes. 104
Now with python unlike with node.js you need to create a 105
virtual environment in which to install any modules that are outside of the 106
standard python code. So what we need to do is we 107
need to create that and it's just an anomaly of python. 108
It's not a lambda thing. It's purely a python thing. So what we 109
need to do is we need to create this virtual environment 110
and i'll just cut it cut and paste from the lab notes here 111
and then we need to activate this and again just copying and pasting from 112
the lab notes. So activate that virtual environment. So now 113
we can see that we're inside this virtual environment here. 114
What we need to do now is that we need to install those modules that our 115
application code needs so it needs pillow. 116
Now pillow has an image module in there or an image package or a module inside 117
of the pillow package and that can help us to resize 118
these images and also boto3 which is required for 119
python coding on aws. 120
Okay so they've all been installed. So what we need to do now is that we need 121
to create this requirements.txt file that will list all of these 122
requirements or these additional packages that are required. 123
The way we do that is that we have them installed now on this ec2 124
environment. So what we can do is we can do a 125
pip freeze which will have a look at what our current environment is and it will 126
put that all into a requirements.txt So again copy and paste from the lab notes 127
pip freeze and so now we can see on the left 128
hand side here we've got a requirements.txt file that's been 129
created. So we double click on that and we can 130
see that now we've got pillow in there we've got boto in there everything that 131
we need to run this application. So just close out of that and so what we 132
can do now is deactivate this virtual environment we don't need 133
it anymore 134
and what we can do now is delete this this folder with a virtual environment 135
here as well so just select it and delete it. 136
Okay so that's everything we need for sam to build this application. We've got 137
the template.yaml. We've got our Lambda function.py and 138
we've got our requirements.txt So simply three 139
files is all we need to to to build this. so we just go to sam 140
and actually first of all we need to go back up to where our template.yaml is. 141
We just go cd and up one file or one folder. We'll do 142
an ls here make sure that it's in there. So that's where our template.yaml file 143
is. So we need to do our sam build from where that is. 144
So do sam build and sam will now build that application and prepare it for sam 145
deploy. So that that was successful. All we 146
can do now is we just need to deploy that so sam deploy and we're 147
going to do --guided and that will ask us a few questions 148
about how we want this to be deployed, so our stack name is going to be sam app 149
that will be fine we'll just select that that's okay, 150
region us-east-one that's fine, confirm changes before deploying, that's always a 151
good idea and we wanted to create a role yes, save 152
the arguments to sam config so that means that next time 153
we can just type in sound deploy and i'll pick up whatever's in our sam 154
config, we don't have to do guided again. Now it's going to be creating this 155
changeset of what we're going to be deploying 156
and yes we want to deploy this change set 157
so now it's going to start to create all of those resources so it'll create the 158
IAM role and the permissions, it'll create the 159
Lambda function and it'll create those two buckets, the 160
source bucket and the destination bucket. 161
Okay so after about five minutes or so everything has been deployed so let's 162
scroll up and have a look and see what's being created here. We've created 163
a role then we've created that lambda function create thumbnail, 164
we've created some permissions for that and we've also created two buckets, 165
the s3 bucket there called source bucket and also our 166
destination bucket and there is our cloudformation stack 167
called sam app. so we go into cloudformation we'll 168
be able to find that so what we'll do now is we're going to 169
the s3 management console and what we're going to do is find these 170
two buckets, so i'm just going to refresh the screen 171
here 172
so here we go we've got a sam app dash source bucket and that's obviously the 173
one that was created for us and we've also got the same name with 174
-resize so that's our destination bucket, so we're going to 175
go into this source bucket now and we're going to upload an image to it 176
and just click next for everything and upload that, 177
so that's uploaded fine, so if we now go back to s3 we'll just open it up in a 178
new window and we open up the resized one we should 179
have that resized object in there, so there we go 180
it worked, so let's have a look, so we've got 181
data.jpg which is 150k or 147k 182
and our resized one is only 23k so that's our thumbnail, 183
it was created successfully, so everything worked fine 184
so let's have a look at lambda, so what we've got here in lambda, so if i go to 185
the dashboard of Lambda and go to applications this time and there is our 186
sam application called sam app, it's created 187
complete, 188
so if we click on it we can see the resources here, so we've got our create 189
thumbnail which is our lambda function and our two 190
s3 buckets, so let's have a look at our lambda 191
function, so just click on here and that will open up our lambda 192
function for us and we can see there it is, 193
so let's let's click on monitoring and see what 194
what happened when when we ran it, so because we've instantiated that 195
that that Lambda function once there will be logs in there 196
until we actually instantiate a lambda function there will be no cloudwatch 197
logs created, so now there will be because we've 198
actually used it once 199
and just scrolling down we can see we've got that log stream, we'll just click on 200
that, okay so there we can see we've got the 201
print statements that were in our code have now been 202
outputted here and stored on cloudwatch logs, 203
so i've got here the function loaded successfully, downloaded the source image 204
from s3, resizing image and then uploading the 205
resized image to amazon s3, so that's all looking like it worked 206
perfectly, so what we'll do now is have a look at 207
the code and see how it all worked, so just going in here to the lambda 208
function.py and let's have a look at that, so the 209
first line here import boto3, so boto3 is the aws 210
python software development kit, then we've got some imports here for some 211
things that we're using or some packages that we're using within the 212
code, now that is all part of the standard 213
python with with lambda so we didn't have to 214
have that in our requirements.txt but we are now importing pillow and in 215
particular the image module from pillow, so that was 216
we needed to install that and put that into our requirements.txt, so that was 217
part of our build package and so we're importing PIL.image 218
and we're going to be using that to resize these images that were uploaded 219
to amazon s3 and then we're just 220
doing s3 or creating a variable s3 client which is going to be 221
creating an object to work with amazon s3 through the boto 222
software development kit, so what we've got here 223
is outside of our lambda handler and that is always good practice, 224
so by having that outside of the handler it 225
means that our lambda function is going to load quite 226
quickly, if we have our lambda function quite or 227
a lambda handler quite large then it's going to slow down 228
the invoking of that lambda function, so 229
what's happening here is it's resizing the image and so that 230
image will be located in temporary storage 231
in that lambda function and then it's going to resize our image to a width of 232
200 and then it's going to save that 233
again in temporary storage in another location in the Lambda functions 234
temporary storage, so when the lambda function is 235
terminated that temporary storage and even 236
anything in that temporary storage will also be 237
terminated, so here we've got the lambda handler and 238
it's going to get the bucket name from the 239
event that is passed to this Lambda function, 240
So where do we get that information from? We need to go to the 241
s3 documentation, so i'm just going to get that up now 242
and so if we go to the s3 developer guide, 243
if we go to notifications and then we go to event message structure, 244
it will give you the format of the json object 245
that is passed to the Lambda function when it is invoked from 246
this s3 event, so here we can see we've got an array of 247
records, so we're only interested in the first record in that array and so 248
that will be records zero and then we're interested in s3 249
bucket and name to get the the name of our source 250
bucket and we're also interested in s3 object and key which will be the name of 251
the object or the file that we uploaded to that to that bucket, 252
so now we can go back to cloud9 and have a look, so there we can see 253
so we've got records and the first record in that array 254
s3 bucket and name and then here we've got 255
records again s3 object and key for our key name, 256
so the first thing we need to do is that we're going to remove any bad or 257
any non-ascii characters and spaces from that key, so we're going to do that there 258
and then once we've done that, what we can do is we can 259
download that image from amazon s3 and then we can store it 260
in temporary storage, so that's what we're doing here, so we're defining where 261
that temporary storage is, so the tmp folder of this lambda 262
function is where temporary storage is and so 263
that lambda function when it's terminated 264
that temporary storage will also be terminated and lost, 265
so what we do is s3 client, so again we've defined that 266
as part of our boto software development kit 267
and then we do download file, the name of the bucket, 268
the key and then the path the which will be the temporary storage that 269
we're going to be storing this this image when we download it, 270
once we've got it into our temporary storage, 271
we're going to create an upload or define an upload path which will 272
also be temporary storage and then we're going to 273
resize that image, so we're going up here to the resize 274
image function there and we're going to resize that to 200, 275
doing image.resize and then we're going to do image.save to save it to 276
that temporary storage on this lambda 277
function, so once we've resized the image, put it 278
into temporary storage, then we can upload it to our destination 279
s3 bucket, so here we go so s3 client and then then we're going to do upload 280
file, so the upload path which will be the 281
temporary storage location that we've got our resized image in 282
and then the bucket that we want it to go to 283
and then the actual name that we're going to save it as there which is a key, 284
so that's what we're doing. 285
We're getting an object and we're downloading it to 286
storage, getting an image from amazon s3 we're downloading it to temporary 287
storage, we're resizing that image to width of 200 288
and then downloading that again to temporary storage 289
and then once we've got that all done and ready we're going to upload that 290
from temporary storage up to that amazon s3 destination bucket 291
and that's how it all works. So I hope you understand all that. It's 292
pretty straightforward i think and what we'll do now is 293
we'll finish up with the lab and we'll clean up everything so the first thing 294
we need to do is to empty these s3 buckets so go into the s3 295
bucket and actions and delete so delete that 296
object there but we don't delete the actual bucket we let cloud formation do 297
that for us and that one there again we'll delete 298
that one as well 299
okay so now we can go into into cloud formation and there is our 300
sam app and we can delete that 301
so that will delete that stack and any resources that have been created by that 302
stack so it will delete those s3 buckets it will delete the sam application in 303
lambda and also the functions and also the iam role that was 304
created so that brings us to the end of a pretty 305
advanced lab and i hope i've broken all down to 306
something quite simple because it can be quite confusing when you're reading the 307
aws documentation so this is i think quite a simple way of doing things 308
and i hope you enjoyed it and i look forward to seeing you in the next labs