1 00:00:07,550 --> 00:00:13,170 Welcome back to Backspace Academy. In this lecture we're going to be creating 2 00:00:13,170 --> 00:00:18,539 a nodejs application and we're going to use the X ray software development 3 00:00:18,539 --> 00:00:24,900 kit to send performance data about that application over to AWS X ray and we're 4 00:00:24,900 --> 00:00:29,130 going to do it all using elastic Beanstalk and take advantage of the 5 00:00:29,130 --> 00:00:35,450 integration between elastic Beanstalk and a AWS x-ray. 6 00:00:35,450 --> 00:00:37,440 In order for us to 7 00:00:37,440 --> 00:00:42,989 use the AWS x-ray service to monitor our application we need some permissions set 8 00:00:42,989 --> 00:00:48,480 up for our application that is using the AWS x-ray software development kit so 9 00:00:48,480 --> 00:00:54,510 what it will need to have permission to send segment document information to the 10 00:00:54,510 --> 00:00:59,100 daemon and also the x-ray daemon that is running will need to have permission 11 00:00:59,100 --> 00:01:05,129 after it has collected that data to forward that on to the AWS x-ray service 12 00:01:05,129 --> 00:01:11,670 now if we're using Elastic Beanstalk we can automatically in our configuration 13 00:01:11,670 --> 00:01:16,770 file enable x-ray integration with elastic Beanstalk and what elastic 14 00:01:16,770 --> 00:01:22,710 Beanstalk will do is it will not only run our application with its AWS x-ray 15 00:01:22,710 --> 00:01:27,840 software development kit it will also run as a separate process on the same 16 00:01:27,840 --> 00:01:34,650 ec2 instances it will run the AWS x-ray daemon and so that makes life a lot 17 00:01:34,650 --> 00:01:40,079 easier for us and what will happen is that our our application will pass its 18 00:01:40,079 --> 00:01:44,369 segment documents over to the x-ray daemon and it all be happening on the 19 00:01:44,369 --> 00:01:45,784 same ec2 instances 20 00:01:45,784 --> 00:01:50,450 now for that to happen we need some permissions to be set up 21 00:01:50,450 --> 00:01:55,560 so first of all we need to have permission for the software development 22 00:01:55,560 --> 00:02:00,840 kit that is running on our application to pass a segment documents over to the 23 00:02:00,840 --> 00:02:05,430 daemon so that the daemon knows what information to collect and then the 24 00:02:05,430 --> 00:02:10,830 daemon needs to have permission to forward what it has collected over to 25 00:02:10,830 --> 00:02:12,183 the AWS x-ray 26 00:02:12,183 --> 00:02:15,720 now fortunately for us the elastic 27 00:02:15,720 --> 00:02:20,220 beanstalk already has those permissions set up but we can check that by going to 28 00:02:20,220 --> 00:02:29,580 services and then into our IAM management console we go to roles and 29 00:02:29,580 --> 00:02:32,920 we're going to search for an ec2 role 30 00:02:32,920 --> 00:02:35,190 and there we have the elastic 31 00:02:35,190 --> 00:02:40,320 Beanstalk ec2 roll that elastic Beanstalk uses for its ec2 32 00:02:40,320 --> 00:02:42,470 instances that it launches 33 00:02:42,470 --> 00:02:46,770 we'll click on that it has four policies attached to 34 00:02:46,770 --> 00:02:51,780 it one of those policies is the elastic Beanstalk web tier so let's have a look 35 00:02:51,780 --> 00:02:52,280 at that one 36 00:02:52,280 --> 00:03:00,420 okay so in that policy we've got some permissions for Amazon s3 to 37 00:03:00,420 --> 00:03:07,500 list and put objects in there we've also got x-ray access so what we've got there 38 00:03:07,500 --> 00:03:12,240 are two permission so we've got put trace segments that will be used by the AWS 39 00:03:12,240 --> 00:03:17,400 x-ray software development kit and it will be used to send segment documents 40 00:03:17,400 --> 00:03:23,370 over to the x-ray daemon that will also be running on that same ec2 instance so 41 00:03:23,370 --> 00:03:29,640 that ec2 instance will also need permission to put telemetry records for 42 00:03:29,640 --> 00:03:36,120 the daemon to pass those telemetry records over to the x-ray server so 43 00:03:36,120 --> 00:03:40,769 it'll pass that trace data that it is collected about our application over to 44 00:03:40,769 --> 00:03:44,790 the x-ray service so that is setup fine but of course make sure that you go in 45 00:03:44,790 --> 00:03:48,630 and make sure that it is already set up for you but it should by default be 46 00:03:48,630 --> 00:03:54,269 there now before we actually start our elastic Beanstalk application we're 47 00:03:54,269 --> 00:04:01,620 going to go to the repository or our starting branch of the repository for 48 00:04:01,620 --> 00:04:07,170 our application code and that will be in the lab document so make sure that you 49 00:04:07,170 --> 00:04:11,459 download those lab notes as always and get the link to that and let's start 50 00:04:11,459 --> 00:04:17,459 having a look at this code so we've got our EB extensions folder and that will 51 00:04:17,459 --> 00:04:23,010 contain all of the configuration information that we're going to use to 52 00:04:23,010 --> 00:04:26,040 set up Elastic Beanstalk so let's have a look at that one 53 00:04:26,040 --> 00:04:31,620 and we've got to there we've got an options config and an xray config now 54 00:04:31,620 --> 00:04:36,300 all of these configurations could be in the one config file it doesn't have 55 00:04:36,300 --> 00:04:39,690 to be in separate ones but it is a obviously a good practice to separate 56 00:04:39,690 --> 00:04:46,110 them into the service and the actual elastic Beanstalk core options that you 57 00:04:46,110 --> 00:04:47,230 are configuring 58 00:04:47,230 --> 00:04:50,610 so we're just going to the options there for elastic Beanstalk the 59 00:04:50,610 --> 00:04:54,660 first one there is we're creating an application environment so this is an 60 00:04:54,660 --> 00:05:01,020 an environment variable that our NodeJS application can reference 61 00:05:01,020 --> 00:05:07,710 and that is called AWS underscore region and it's going to reference the AWS 62 00:05:07,710 --> 00:05:14,370 region that our ec2 instance is running on so that will appear to our 63 00:05:14,370 --> 00:05:19,980 application as an environment variable called region and we've also got there 64 00:05:19,980 --> 00:05:22,470 that we're going to be running nginx for nodejs 65 00:05:22,470 --> 00:05:27,420 and we're also going to be putting all of our static files into a static folder 66 00:05:27,420 --> 00:05:32,640 here as well so let's go back to the application and we can see there we've 67 00:05:32,640 --> 00:05:36,380 got that static that static directory it's going to have our bootstrap and 68 00:05:36,380 --> 00:05:43,290 jQuery libraries in there not going to be changing their static and it's going 69 00:05:43,290 --> 00:05:48,810 to have our view which is going to have a pug document that's going to have the 70 00:05:48,810 --> 00:05:54,040 details of our HTML web application 71 00:05:54,040 --> 00:05:56,730 and finally there we've got our app,js 72 00:05:56,730 --> 00:06:00,440 which is going to have our nodejs or javascript code 73 00:06:00,440 --> 00:06:02,010 now the first thing we 74 00:06:02,010 --> 00:06:08,220 see there is we've got X ray require the AWS x-ray software development kit now 75 00:06:08,220 --> 00:06:12,630 that is different to the AWS software development kit also if we want to 76 00:06:12,630 --> 00:06:19,590 capture any calls to the or through the AWS software development kit for example 77 00:06:19,590 --> 00:06:23,400 if further on in this application we're going to meet making some calls to 78 00:06:23,400 --> 00:06:29,610 Amazon s3 if we want to capture those as well we we need to require the AWS 79 00:06:29,610 --> 00:06:33,878 software development kit inside of this capture AWS 80 00:06:33,878 --> 00:06:35,970 and finally there we also 81 00:06:35,970 --> 00:06:42,210 need to have capture HTTP and we're going to require HTTP and that 82 00:06:42,210 --> 00:06:51,360 will monitor all of our HTTP requests to and from our application and further 83 00:06:51,360 --> 00:06:55,890 down here we can see here that we've got AWS config dot region and as we said 84 00:06:55,890 --> 00:07:00,180 before with we set that up in our configuration file and we can reference 85 00:07:00,180 --> 00:07:05,040 it and that's going to get the current region for for this application that it 86 00:07:05,040 --> 00:07:10,170 is running on and then we also need to set our sampling rules so there's going 87 00:07:10,170 --> 00:07:15,650 to be a sampling rules.JSON file we'll have a look at that soon and 88 00:07:15,650 --> 00:07:21,300 there's going to be here we've got our start x-ray segment so this is where we 89 00:07:21,300 --> 00:07:25,680 define our segment that we're going to be collecting data with so we've got 90 00:07:25,680 --> 00:07:30,120 here we're going to call our segment myfrontend and we're going to finish that 91 00:07:30,120 --> 00:07:34,770 at the end of our application here we're going to close that segment there as 92 00:07:34,770 --> 00:07:38,190 well so within here and here we're going to have a segment called my front end 93 00:07:38,190 --> 00:07:45,960 and that will appear on the AWS x-ray when we look at the dashboard we've got 94 00:07:45,960 --> 00:07:51,000 a sub sub segment here that we're going to be calling page render and so what we 95 00:07:51,000 --> 00:07:56,970 do is we do x-ray capture async function and we call it page render and then 96 00:07:56,970 --> 00:08:03,410 we're going to put that segments or that segment that we're capturing inside here 97 00:08:03,410 --> 00:08:09,030 so there we can do we're just going to be rendering that application and giving 98 00:08:09,030 --> 00:08:12,960 it a title and then we're just going to be closing that sub segment there's a 99 00:08:12,960 --> 00:08:17,730 segment.close and that's what's going to happen so we've this will have 100 00:08:17,730 --> 00:08:23,360 a very simple application it's just going to display a HTML web page and 101 00:08:23,360 --> 00:08:27,660 we're going to have that as a segment and inside that segment we're just going 102 00:08:27,660 --> 00:08:35,880 to have one single sub segment and just go back to the main root directory here 103 00:08:35,880 --> 00:08:41,310 and as we said before we have sampling rules JSON so there we've got our 104 00:08:41,310 --> 00:08:46,920 default sampling rules we're going to have a target of one sample per second 105 00:08:46,920 --> 00:08:53,280 and the rate is going to be one so what that means is that when we've 106 00:08:53,280 --> 00:08:59,640 when we've got over or any-any samples that go over one sample per second we're 107 00:08:59,640 --> 00:09:03,540 going to get all of those additional samples as well so we're 108 00:09:03,540 --> 00:09:05,388 going to get 100% of those 109 00:09:05,388 --> 00:09:08,160 so if we had a target of one sample per 110 00:09:08,160 --> 00:09:14,730 second and we owned it only once it gets a 10% of those additional all those 111 00:09:14,730 --> 00:09:18,570 additional traces that were coming in all those additional requests we can 112 00:09:18,570 --> 00:09:23,880 have the rate of 0.1 and it'lll only collect the extra 10% that for that period of 113 00:09:23,880 --> 00:09:24,380 one second 114 00:09:24,380 --> 00:09:27,840 so that's how we set up those sampling rules but there's a lot more to 115 00:09:27,840 --> 00:09:33,090 sampling rules but goes beyond what you need to know for an associate level 116 00:09:33,090 --> 00:09:33,590 certification 117 00:09:33,590 --> 00:09:38,580 okay so that's enough on the application let's get in and run it 118 00:09:38,580 --> 00:09:43,980 so what we do is we first need to download it now this zip file that I'm 119 00:09:43,980 --> 00:09:49,380 downloading here we can't cannot use it directly with with the elastic Beanstalk 120 00:09:49,380 --> 00:09:52,650 and don't try and do it because you'll get an error you'll get a 502 error when 121 00:09:52,650 --> 00:10:01,530 you go to do it so what we need to do is that we just need to show in folder okay 122 00:10:01,530 --> 00:10:06,839 so the reason we can't can't use this directly this zip file is if we double 123 00:10:06,839 --> 00:10:10,140 click on have a look at it we can see that all of our code is inside this 124 00:10:10,140 --> 00:10:14,040 folder so if we upload this directly unless your beam starts I'm not going to 125 00:10:14,040 --> 00:10:17,460 be able to find that code is gonna be expecting it here not hidden inside this 126 00:10:17,460 --> 00:10:21,870 folder so we need to get it outside of this folder so first of all it will 127 00:10:21,870 --> 00:10:24,440 extract it 128 00:10:30,410 --> 00:10:34,620 okay so that's extracted that what we do to do is just open this up and what 129 00:10:34,620 --> 00:10:39,950 we're going to do now is just create another zip file in here so sent to 130 00:10:39,950 --> 00:10:44,940 compressed folder okay that's good alright so now we can use that to upload 131 00:10:44,940 --> 00:10:53,339 to elastic Beanstalk and we're ready to go okay so back into elastic Beanstalk 132 00:10:53,339 --> 00:10:59,670 and management console and if we haven't got any applications up and running this 133 00:10:59,670 --> 00:11:02,190 is what we're going to get just a welcome screen so we click on create new 134 00:11:02,190 --> 00:11:06,690 application in the top right or right hand corner here and we'll give this a 135 00:11:06,690 --> 00:11:13,699 name and create that 136 00:11:13,730 --> 00:11:21,410 okay so let's create an environment first it's a web server environment and 137 00:11:21,410 --> 00:11:25,410 we'll just leave that name and we'll leave it for auto generating a domain 138 00:11:25,410 --> 00:11:30,269 for us that's fine and we select nodejs and we're going to upload our 139 00:11:30,269 --> 00:11:44,690 code and we'll upload that now so app.zip and we upload that and upload 140 00:11:46,820 --> 00:11:51,140 okay and create our environment 141 00:11:56,700 --> 00:12:02,020 okay so our environment will start chugging away there and soon it will be 142 00:12:02,020 --> 00:12:08,680 up and running okay so after quite some time we've got our environment up and 143 00:12:08,680 --> 00:12:11,890 running let's click on the end point for that environment and we can see there 144 00:12:11,890 --> 00:12:16,960 we've got our simple HTML page nothing too spectacular there so let's have a 145 00:12:16,960 --> 00:12:21,580 look and see where the x-ray actually collected any of that data that we that 146 00:12:21,580 --> 00:12:25,540 we requested so we're going to services we're going to x-ray just going to open 147 00:12:25,540 --> 00:12:34,300 that into a new tab and it'll be computing that map it might take some 148 00:12:34,300 --> 00:12:37,720 time depending on how much data you've got if you've got less than a minute it 149 00:12:37,720 --> 00:12:41,230 won't come up we've got a minute or so you should come up with a bit of data 150 00:12:41,230 --> 00:12:48,640 and then we can see we've got our our server set and node and it's myfrontend 151 00:12:48,640 --> 00:12:53,460 so remember we created a segment in our application called my front end and 152 00:12:53,460 --> 00:12:57,700 that's what's appeared here so we've we've got a problem here so we've got 153 00:12:57,700 --> 00:13:02,620 most of it's okay but one of those requests came of an error and that's 154 00:13:02,620 --> 00:13:06,460 going to be an orange so it's an client-side error so it must be 155 00:13:06,460 --> 00:13:09,940 something like a 404 error we've made a request for something that doesn't exist 156 00:13:09,940 --> 00:13:15,820 so we can click on this node here and we can see information about the response 157 00:13:15,820 --> 00:13:20,710 distribution of the requests I've got some requests there and we can see that 158 00:13:20,710 --> 00:13:26,890 they vary from up to one second down to you know 100 or 200 milliseconds here so 159 00:13:26,890 --> 00:13:31,360 what we can do now is we can filter our traces based on their error so we're 160 00:13:31,360 --> 00:13:34,240 just going to click on here on their arrow because we just want to see the 161 00:13:34,240 --> 00:13:39,610 problem traces and we're going to click on View traces here and here we can see 162 00:13:39,610 --> 00:13:45,760 our trace list so we click on the actual trace ID and we can see what the problem 163 00:13:45,760 --> 00:13:52,630 is so it was my front end and it gave us a 404 error what we can do now is that 164 00:13:52,630 --> 00:13:57,880 we can click on that again and that will give us more information about that 165 00:13:57,880 --> 00:14:03,580 segment which was my front end and what caused it to give an error so if we look 166 00:14:03,580 --> 00:14:10,209 down here for it further we've got the request URL and it was for our favicon.icon 167 00:14:10,209 --> 00:14:15,639 now we didn't actually have one in our application so it made a request 168 00:14:15,639 --> 00:14:19,720 to that and it didn't actually exist so that was where the problem was 169 00:14:19,720 --> 00:14:24,459 and not a major deal we can just not worry about but later on we can if we're 170 00:14:24,459 --> 00:14:29,139 going to create a real application we're going to put a favicon icon there for 171 00:14:29,139 --> 00:14:34,269 our application we can do that so we just close that out now what we want to 172 00:14:34,269 --> 00:14:40,420 do now is that we want to actually start looking at how we can trace cause to the 173 00:14:40,420 --> 00:14:43,990 software development kits the actual AWS software development kit so what we're 174 00:14:43,990 --> 00:14:48,189 going to do is that we're going to call the Amazon s3 service we're going to 175 00:14:48,189 --> 00:14:51,999 list the buckets and we're going to see how long that's going to take so we need 176 00:14:51,999 --> 00:14:56,589 to make some changes to our application code to make that happen so again we'll 177 00:14:56,589 --> 00:15:04,540 just jump back into our repository and we'll now go to the master node which 178 00:15:04,540 --> 00:15:10,869 will have the completed application then okay so let's have a look at some of the 179 00:15:10,869 --> 00:15:16,509 changes that I've made to this so that we can actually monitor those calls to 180 00:15:16,509 --> 00:15:19,445 AWS or using the AWS software developer kit 181 00:15:19,445 --> 00:15:21,220 so again we've already set 182 00:15:21,220 --> 00:15:26,649 this up previously but we have a require AWS software development kit inside of 183 00:15:26,649 --> 00:15:33,819 the x-ray doc capture AWS and that will capture all calls that are going to the 184 00:15:33,819 --> 00:15:39,670 software development kit or from the software development kit and what we do 185 00:15:39,670 --> 00:15:45,189 now is we just scroll down here and in our capture async function we're going 186 00:15:45,189 --> 00:15:51,069 to have our an s3 list buckets here and then at the same time we're also going 187 00:15:51,069 --> 00:15:55,929 to have our rendering out there as well and then we're going to close off that 188 00:15:55,929 --> 00:15:59,410 segment so what we'll do is we'll we'll see exactly the same as what we had 189 00:15:59,410 --> 00:16:03,549 before we'll see that that that segment my front end will come up as a no but we 190 00:16:03,549 --> 00:16:10,420 should have another node there as well for this call to Amazon s3 and it's just 191 00:16:10,420 --> 00:16:14,529 going to output that when it renders this it's going to do it to a variable 192 00:16:14,529 --> 00:16:19,029 there called bucket list and we're going to put that in to the view so I just go 193 00:16:19,029 --> 00:16:26,200 back into the view here and index if we see down there 194 00:16:26,200 --> 00:16:28,780 that we're just going to output that buck at least it's going to be a string 195 00:16:28,780 --> 00:16:33,460 of all of the buckets that I've got on this account so again we go back up here 196 00:16:33,460 --> 00:16:39,460 and we'll save that zip file or download that zip file and again we need to 197 00:16:39,460 --> 00:16:47,640 extract it and get it all into the right format show in folder let's have a look 198 00:16:49,350 --> 00:16:56,020 and I've just extracted that that that file that zip file just go into it and 199 00:16:56,020 --> 00:17:00,840 again we need to select everything and put it into a higher-level 200 00:17:00,840 --> 00:17:07,030 zip for them and we just open it up make sure that is fine there is not a problem 201 00:17:07,030 --> 00:17:11,650 so I'll just go back in there we can now update our elastic beanstalk environment 202 00:17:11,650 --> 00:17:16,240 to have our new code so go back into their elastic Beanstalk environment 203 00:17:16,240 --> 00:17:21,810 click on upload and deploy choose our file 204 00:17:31,990 --> 00:17:39,220 and we just give a version label say version 2 will be fine and deploy that 205 00:17:41,350 --> 00:17:47,810 ok again so after some time we've got our new version 2 up and running we'll 206 00:17:47,810 --> 00:17:52,670 click on that end point there so it's the same web page as before but we've 207 00:17:52,670 --> 00:17:57,740 got this list of our buckets here that I've got on this account so everything 208 00:17:57,740 --> 00:18:02,720 seems to be working so we should be able to see as a trace this call to Amazon 3 209 00:18:02,720 --> 00:18:08,900 s3 to list those buckets so we go back into AWS x-ray we'll click on the 210 00:18:08,900 --> 00:18:13,820 service map and we'll refresh it so we just click on the right hand side here 211 00:18:13,820 --> 00:18:22,370 and refresh and map is updating ok so there we can see that we've got again as 212 00:18:22,370 --> 00:18:27,500 before we've got our my front-end and we've also got this other service note 213 00:18:27,500 --> 00:18:32,180 here for Amazon s3 so the my front-end will be exactly same as before but we've 214 00:18:32,180 --> 00:18:38,150 we click on s3 we can see there that we've got a single call and it took 215 00:18:38,150 --> 00:18:42,830 about 150 or so milliseconds there was no errors on that 216 00:18:42,830 --> 00:18:48,590 so we've reviewed those traces and we click on the trace ID down here we can 217 00:18:48,590 --> 00:18:52,100 see some more information about it so there we can see we've got our my 218 00:18:52,100 --> 00:19:01,400 front-end segments inside our my front in my front end segment we have a page 219 00:19:01,400 --> 00:19:05,630 render sub segment and that took up 1.2 seconds 220 00:19:05,630 --> 00:19:14,330 and we've also got a call to Amazon s3 which was a list buckets call here and 221 00:19:14,330 --> 00:19:19,010 we can see there that the big thing that took the time was a page render but the 222 00:19:19,010 --> 00:19:24,230 cord Amazon s3 was quite quick no not a problem there and we can some some more 223 00:19:24,230 --> 00:19:27,860 information about the s3 them and we can see there that he took a hundred 224 00:19:27,860 --> 00:19:34,130 sixty-three milliseconds there he was two hundred he was a successful call if 225 00:19:34,130 --> 00:19:37,370 we click on that we can get some more information about it and there we can 226 00:19:37,370 --> 00:19:44,870 see no errors the resources and we can see there as it's a list buckets 227 00:19:44,870 --> 00:19:51,409 in the region the request ID all fine there so that's pretty good you pretty 228 00:19:51,409 --> 00:19:57,190 well have a good understanding now of how x-ray works how it can monitor calls 229 00:19:57,190 --> 00:20:04,100 HTTP calls to your request and how it can or how you can set up segments of 230 00:20:04,100 --> 00:20:09,409 your code and monitor those segments and sub segments if you want to look at the 231 00:20:09,409 --> 00:20:14,000 performance of different parts of your application and you can also monitor 232 00:20:14,000 --> 00:20:19,190 calls to AWS services as well so that brings us to an end what we need to do 233 00:20:19,190 --> 00:20:25,539 now is clean it all up so we just closed x-ray there we just need to go into our 234 00:20:25,539 --> 00:20:32,090 applications there in an elastic Beanstalk and click on the x-ray lab and 235 00:20:32,090 --> 00:20:36,559 actions and delete that application don't delete the environment delete the 236 00:20:36,559 --> 00:20:39,620 application and it would delete everything that is associated with that 237 00:20:39,620 --> 00:20:45,620 application so just delete that and we should be up and running so just click 238 00:20:45,620 --> 00:20:50,720 on that in from and again and we can see their elastic Beanstalk is terminating 239 00:20:50,720 --> 00:20:54,590 your environment so I hope you enjoyed it hope you lot got a lot out of this 240 00:20:54,590 --> 00:20:58,820 one it's a very good hands-on lab and I look forward to seeing you in the next 241 00:20:58,820 --> 00:21:01,120 lecture