1

00:00:03,460  -->  00:00:09,590
welcome back to back space Academy in
this lecture on simple queue service or sqs

2

00:00:09,590  -->  00:00:16,040
for short we'll go through and explain
exactly what the sqs service is it's a

3

00:00:16,040  -->  00:00:21,260
service for the queueing of messages and
how we can use that to the couple our

4

00:00:21,260  -->  00:00:27,140
processes we'll look at the queue types
and we'll also look at the life cycle of

5

00:00:27,140  -->  00:00:32,420
a message as it passes through the sqs
process we'll look at dead letter queues

6

00:00:32,420  -->  00:00:37,430
what we what do we do with messages that
cannot be processed but look at delay

7

00:00:37,430  -->  00:00:43,309
queues and message timers and we'll also
look at the process of long polling sqs

8

00:00:43,309  -->  00:00:50,329
it is a queue for storing messages and
it acts as a buffer of data for

9

00:00:50,329  -->  00:00:57,680
processing service sqs can smooth out
the peak demand up to 10 attributes or

10

00:00:57,680  -->  00:01:02,059
name and values can be added to a
message additional to the message body

11

00:01:02,059  -->  00:01:08,870
and the message size can be set from 1 k
to 256 K so just looking at the diagram

12

00:01:08,870  -->  00:01:12,770
there we have messages coming in so
those messages will have peaks and

13

00:01:12,770  -->  00:01:16,400
troughs of demand there will be more
coming in at one stage and then there

14

00:01:16,400  -->  00:01:21,590
will be gaps where there won't be so
much so between those peaks and troughs

15

00:01:21,590  -->  00:01:25,580
the instance needs to cope with that and
we don't want the instance to be taken

16

00:01:25,580  -->  00:01:30,080
out by too much of a demand so we allow
those messages to build up in our sqs

17

00:01:30,080  -->  00:01:36,080
queue and our instance will process
those messages one by one and it will

18

00:01:36,080  -->  00:01:40,610
every time it finishes a message it will
go back and grab another one by polling

19

00:01:40,610  -->  00:01:48,290
the queue and then it just keeps going
until the queue is is completed it's

20

00:01:48,290  -->  00:01:52,400
curious it allows us to decouple our
processes so it allows us to cope with

21

00:01:52,400  -->  00:01:57,290
those peaks and troughs but one thing we
need to understand is that if the

22

00:01:57,290  -->  00:02:04,400
average demand exceeds our processing
capacity so over time is those peaks and

23

00:02:04,400  -->  00:02:08,539
troughs still average out to be more
than what our instance can cope with

24

00:02:08,539  -->  00:02:13,310
then that queue will grow indefinitely
and it will continue to grow until such

25

00:02:13,310  -->  00:02:20,330
point in time that the sqs service can't
handle it anymore so sqs it can provide

26

00:02:20,330  -->  00:02:24,770
cloud watch metrics that can be used
with auto scaling and that's a very

27

00:02:24,770  -->  00:02:29,930
important thing because that will allow
us to introduce elasticity that is

28

00:02:29,930  -->  00:02:35,690
linked to the size of our sqs queue  and so
that can also allow us to have upper and

29

00:02:35,690  -->  00:02:41,390
lower set points so that when our queue
gets hit by a large amount of demand and

30

00:02:41,390  -->  00:02:46,790
it grows to a significant size that will
trigger a cloud watch metric that will

31

00:02:46,790  -->  00:02:53,450
then alert our auto scaling group to add
more instances and with our lower set

32

00:02:53,450  -->  00:02:57,890
point when that is reached it will do
the same thing but in Reverse so in the

33

00:02:57,890  -->  00:03:02,540
auto scaling group will start to
terminate entrances and reduce our

34

00:03:02,540  -->  00:03:09,080
capacity to save costs there are two
different types of queue types there

35

00:03:09,080  -->  00:03:13,700
used to be in the past only the standard
queue but now we have a FIFO or first in

36

00:03:13,700  -->  00:03:19,880
that first out queue so standard queue
is the default queue type and it

37

00:03:19,880  -->  00:03:24,019
provides nearly unlimited number of
transactions per second so you need

38

00:03:24,019  -->  00:03:28,760
speed the standard queue is a way to go
it guarantees that a message is

39

00:03:28,760  -->  00:03:34,810
delivered at least once although you'd
need to be aware that although really

40

00:03:34,810  -->  00:03:39,230
duplicates can occur and so your
application needs to be aware of that

41

00:03:39,230  -->  00:03:46,160
and have checks and balances in place in
case a messages processed twice and it

42

00:03:46,160  -->  00:03:53,180
does best-effort ordering so although
most of the time the messages will come

43

00:03:53,180  -->  00:03:57,319
in in the order that they're received
it's not always the case so then we look

44

00:03:57,319  -->  00:04:03,260
at first in first out queue type and
that complements the standard type and

45

00:04:03,260  -->  00:04:09,200
it's limited to 300 transactions per
second so it is slower than a standard

46

00:04:09,200  -->  00:04:12,819
queue type but it provides first-in
first-out

47

00:04:12,819  -->  00:04:20,090
delivery of our messages and it provides
exactly once processing one thing to be

48

00:04:20,090  -->  00:04:23,900
aware of its are currently not available
in all regions so you need to check your

49

00:04:23,900  -->  00:04:29,250
region to make sure that FIFO queues are
available

50

00:04:29,250  -->  00:04:34,800
it's very important to understand the
lifecycle of a message as it passes

51

00:04:34,800  -->  00:04:40,470
through the sqs service so that our
application can work properly for the

52

00:04:40,470  -->  00:04:46,470
first part of the message life cycle is
that the message is received by the sqs

53

00:04:46,470  -->  00:04:53,340
queue then the message will be received
by the processing server when the

54

00:04:53,340  -->  00:04:58,800
message is received by the processing
server the visibility timeout period

55

00:04:58,800  -->  00:05:06,410
starts so at that point it's not visible
to the message is not visible to other

56

00:05:06,410  -->  00:05:11,730
instances that may be polling that sqs
queue so once the message is processed

57

00:05:11,730  -->  00:05:17,640
by the processing server the processing
server sends a message delete request

58

00:05:17,640  -->  00:05:24,690
back to the sqs service at that point
the visibility timeout period ends so

59

00:05:24,690  -->  00:05:30,780
the visibility timeout specifies the
time which some message is invisible in

60

00:05:30,780  -->  00:05:35,430
the queue and has not been deleted and
then once it has been processed and

61

00:05:35,430  -->  00:05:40,530
deleted then the visibility timeout
period ends so if a message has not been

62

00:05:40,530  -->  00:05:47,250
deleted by the visibility timeout period
that the message then becomes visible

63

00:05:47,250  -->  00:05:51,780
and can be received again by a
processing server so that's another

64

00:05:51,780  -->  00:05:54,690
thing that you need to take in
consideration with your application

65

00:05:54,690  -->  00:06:00,240
making sure that you can process
messages within that visibility timeout

66

00:06:00,240  -->  00:06:07,680
period dead-letter queues can be used as
a target for messages that cannot be

67

00:06:07,680  -->  00:06:14,669
processed successfully so that allows us
to remove these messages from our queue

68

00:06:14,669  -->  00:06:19,620
and at the same time not lose those
messages and allows them to be analyzed

69

00:06:19,620  -->  00:06:25,980
or reprocessed at a later time you need
to remember the dead-letter queues must

70

00:06:25,980  -->  00:06:32,960
be in the same region and from the same
account as the original cube

71

00:06:33,490  -->  00:06:38,620
delay queues and message timers allow
for timing changes to be made to a

72

00:06:38,620  -->  00:06:44,800
messages lifecycle so delay queues they
postpone the delivery of new messages

73

00:06:44,800  -->  00:06:50,139
and the message is not visible when it
is first added to the queue for a

74

00:06:50,139  -->  00:06:55,660
defined period of time so that thought
defined period of time is set with

75

00:06:55,660  -->  00:07:00,009
delay seconds parameter with the
create queue or set queue attributes

76

00:07:00,009  -->  00:07:06,940
command for the existing queue you need
to remember that there is a 120,000

77

00:07:06,940  -->  00:07:11,919
limit for the number of in-flight
messages per queue so make sure that if

78

00:07:11,919  -->  00:07:17,949
you implement a delay queue that that
limit is not going to be exceeded we

79

00:07:17,949  -->  00:07:23,490
also have message timers and they
implement an initial invisibility

80

00:07:23,490  -->  00:07:29,229
invisibility period for an individual
message and they can be created using

81

00:07:29,229  -->  00:07:36,370
the console or with the delay seconds
parameter of the send message command we

82

00:07:36,370  -->  00:07:40,330
need to understand the different types
of polling so when our instance has

83

00:07:40,330  -->  00:07:46,659
completed its process and it is ready to
receive another message for processing

84

00:07:46,659  -->  00:07:53,110
it will pull the sqs service and there
are two ways for an instance to poll the

85

00:07:53,110  -->  00:07:58,270
sqs service for new messages and we have
short polling and long polling so short

86

00:07:58,270  -->  00:08:03,550
polling is very simple it just sends the
SQL Server simply sends a response back

87

00:08:03,550  -->  00:08:10,810
immediately whether or not a message is
in the queue so that obviously will

88

00:08:10,810  -->  00:08:17,199
require the instance to regularly keep
polling the sqs service until such point

89

00:08:17,199  -->  00:08:21,550
in time as a message is available the
other option we have available is long

90

00:08:21,550  -->  00:08:30,340
polling and that will return a response
to the instance and it will wait until a

91

00:08:30,340  -->  00:08:33,669
message is available before it does that
so we'll wait until a message is

92

00:08:33,669  -->  00:08:40,050
available in the queue before sending a
response so that significantly reduces

93

00:08:40,050  -->  00:08:44,440
the workload on our instant cell
instance is no longer sending all of

94

00:08:44,440  -->  00:08:50,620
these messages or these requests to the
sqs service it will send one request and

95

00:08:50,620  -->  00:08:57,580
then on an hour synchronous basis it
will have that sent back when there is a

96

00:08:57,580  -->  00:09:01,450
message available so it reduces the
number of empty responses when there are

97

00:09:01,450  -->  00:09:07,300
no messages available to return so we
set the wait time seconds parameter in

98

00:09:07,300  -->  00:09:12,760
the received message call or they
receive message wait time seconds in the

99

00:09:12,760  -->  00:09:18,580
create queue or set attribute score and
that is a parameter between 1 and 20

100

00:09:18,580  -->  00:09:25,960
seconds so that brings us to the end of
our lecture on sqs I'll see you in the

101

00:09:25,960  -->  00:09:28,350
next lecture