‫So now, let's talk about Amazon Athena.
‫So Athena is a serverless query service
‫to help you analyze the data stored in Amazon S3 buckets.
‫And to analyze this data,
‫you're going to use the standard SQL language
‫to query the files.
‫Behind the scenes,
‫Athena is built on the Presto engine,
‫which uses the SQL language.
‫So the idea is that users are going to load data
‫into your S3 bucket,
‫or you are going to load data into your S3 bucket,
‫and then you would use the Athena service
‫to query and analyze this data in Amazon S3
‫without moving it.
‫So Athena is serverless,
‫and it analyzes directly your data living in your S3 bucket.
‫So it supports different formats,
‫such as CSV, JSON, ORC,
‫Avro, and Parquet, and possibly others.
‫And the pricing is very simple.
‫You're just going to pay a fixed amount
‫per terabytes of data scanned.
‫You don't need to provision any database again,
‫because the whole service is serverless.
‫So Athena is commonly used with another tool,
‫called Amazon QuickSight to create reports and dashboards.
‫So we'll see Quicksight later in depth,
‫but Amazon Quicksight connects to Athena,
‫which connects to your S3 buckets.
‫Now, the use cases for Amazon Athena
‫are to do ad hoc queries,
‫business intelligence, analytics, reporting,
‫and to analyze and query
‫any kind of logs that originates from your AWS services.
‫So it could be your VPC flow logs,
‫your load balancer logs, your CloudTrail trails, and so on.
‫So as an exam tip,
‫anytime you need to analyze data in Amazon S3
‫using a serverless SQL engine,
‫you can think about Athena.
‫Now, I've talked about performance improvements,
‫and you can actually improve Athena performance,
‫and in this exam,
‫we'll test you on this as well.
‫So first of all, because you pay for the amount
‫of data scanned per terabyte,
‫you need to use a type of data
‫where you're going to scan less data.
‫And for this, you can use a columnar data type
‫for cost-savings,
‫because you only scan the columns you need.
‫So therefore, the recommended formats for Amazon Athena,
‫are going to be Apache Parquet and ORC,
‫and it's going to give you a huge performance improvement.
‫And to get your files into the Apache Parquet or ORC format,
‫you must use a service
‫that we'll see as well in this section,
‫for example, Glue.
‫Glue can be very helpful
‫to convert your data as an ETL job,
‫between, for example, CSV and Parquet.
‫Now, also because we want to scan less data,
‫we need to compress data for smaller retrievals.
‫So there are different compression mechanisms you can use
‫that have listed it right here.
‫Next, if you know you're going to query all the time
‫on some specific columns,
‫you can partition your datasets,
‫and partition datasets means that in your S3 bucket,
‫you're going to have the full path with slashes,
‫and each slash will be a different column name
‫with a specific value.
‫And so you're organizing,
‫you're partitioning your data in Amazon S3,
‫so that when you query it,
‫you can know exactly in which folder
‫at which path Amazon S3,
‫you need to scan for data.
‫So here's an example of data partitions.
‫So we have flight data in Parquet formats,
‫and then we do /year=1991.
‫So we partition by year,
‫and we'll have one folder for each year.
‫Then within each year we'll have month,
‫so month equals one,
‫and within each month we'll have days,
‫this equals one.
‫And so when I do a query on Athena,
‫and I filter for a specific year,
‫a specific month, and a specific day,
‫then we'll know exactly
‫to which folder in Amazon S3
‫to get the data from,
‫and therefore we'll only recover a subset of the data.
‫Therefore, we'll have really, really good partitioning.
‫Finally, the last performance improvement you need to do,
‫is to use bigger files,
‫it's to minimize your overheads.
‫So if you have many, many, many small files in Amazon S3,
‫Athena is not going to be as performance,
‫as if you had larger files,
‫for example, 128 megabytes, and over,
‫because larger files are easier to scan,
‫and easier to retrieve.
‫Another feature of Amazon Athena is the Federated Query.
‫So you know that Athena can query data in S3,
‫but actually you can query data anywhere,
‫for example, in relational or non-relational databases,
‫you can query objects and custom data sources,
‫would it be on AWS or on-premises.
‫How? Well, you use what's called a Data Source Connector.
‫It's a Lambda function,
‫and that Lambda function is going to run
‫the Federated Queries in other services.
‫So that could be, for example, CloudWatch Logs,
‫DynamoDB, RDS, and so on.
‫So it's very powerful.
‫For example, we have Athena here,
‫and we have a Lambda function,
‫and you'll have one Lambda function
‫per Data Source Connector.
‫And then through Amazon Athena,
‫you can run the query across ElastiCache,
‫Document DB, DynamoDB, Redshift, Aurora,
‫SQL Server, MySQL, HBase on the EMR service,
‫or any on-premises database.
‫They're clear from Athena,
‫as well as, of course, Amazon S3,
‫and you can do your drawings,
‫and you can do your competitions, and so on.
‫That's why it's called a Federated Query.
‫And then the results of this query
‫can be stored into your Amazon S3 buckets
‫for later analysis.
‫So that's it for Amazon Athena.
‫As you can see, it's a very powerful service.
‫I hope you liked it,
‫and I will see you in the next lecture.