AWS Athena and Glue

Amazon Athena and AWS Glue work together to turn raw files in S3 into data you can query with plain SQL. Athena runs the queries. Glue organizes the data and prepares it. Neither service needs a server that you manage.

Picture a huge library where books sit in boxes with no labels. Glue walks through the boxes and writes a catalog card for every book. Athena acts as the librarian who answers questions like "show me every book published after 2020" by using the catalog.

The Big Picture

Raw files in S3 (CSV, JSON, Parquet)
              |
              v
     Glue Crawler scans files
              |
              v
   Glue Data Catalog (tables)
              |
              v
   Athena runs SQL queries
              |
              v
   Results saved to S3 / shown on screen

Amazon Athena

Athena is a serverless query service. You point it at data in S3, write SQL, and receive results in seconds. There is no cluster to create and no software to install.

Key Facts About Athena

  • It reads data directly from S3 without loading it into a database.
  • It supports CSV, JSON, ORC, and Parquet files.
  • Pricing follows the amount of data each query scans.
  • It uses standard SQL, so analysts reuse skills they already have.

A Sample Query

SELECT country, COUNT(*) AS orders
FROM sales_data
WHERE order_year = 2025
GROUP BY country
ORDER BY orders DESC
LIMIT 10;

The query lists the ten countries with the most orders in 2025.

AWS Glue

Glue is a managed data integration service. It discovers data, records its structure, and moves or transforms it between locations.

Main Glue Components

  • Data Catalog: a central index that stores table names, column names, and data types.
  • Crawler: a tool that scans S3 folders and builds catalog tables automatically.
  • ETL job: a script that extracts data, transforms it, and loads it into a new location.
  • Glue Studio: a visual editor for creating jobs without heavy coding.

Setting Up Your First Query

  1. Upload a sales CSV file to an S3 bucket in a folder named sales.
  2. Create a Glue crawler and point it at the folder.
  3. Run the crawler. A table appears in the Data Catalog.
  4. Open the Athena console and choose the database.
  5. Write a SELECT query and run it.
  6. Check the results location in S3 for saved output.

Lower Your Query Costs

Athena bills for scanned data, so smart file design saves money.

Use Columnar Formats

Parquet and ORC store data by column. A query that needs two columns reads only those two. A CSV file forces Athena to read every column of every row.

CSV file (row-based)            Parquet file (column-based)
+----+------+-------+           +------+  +------+  +-------+
| id | name | price |           | id   |  | name |  | price |
| 1  | Pen  | 5     |           | 1    |  | Pen  |  | 5     |
| 2  | Book | 12    |           | 2    |  | Book |  | 12    |
+----+------+-------+           +------+  +------+  +-------+
Reads everything                 Reads only needed columns

Partition Your Data

Partitioning organizes files into folders by a value such as date.

s3://my-bucket/sales/year=2025/month=01/
s3://my-bucket/sales/year=2025/month=02/
s3://my-bucket/sales/year=2025/month=03/

A query filtered to January 2025 reads one folder and ignores the rest.

Compress Files

Compressed files take less space and less scanning. Formats such as Parquet support compression by default.

Common Use Cases

  • Searching application logs stored in S3
  • Analyzing website traffic from CloudFront or load balancer logs
  • Building reports from exported business data
  • Preparing data sets for dashboards in tools like Amazon QuickSight

Security Basics

  • Control access with IAM policies on S3 buckets and Athena actions.
  • Encrypt data in S3 and encrypt query results.
  • Limit who can run crawlers and ETL jobs.
  • Log activity with CloudTrail for audit trails.

Best Practices

  • Convert large CSV data sets to Parquet before running repeated analysis.
  • Name folders consistently so crawlers detect partitions correctly.
  • Select only the columns you need instead of using SELECT *.
  • Set a separate S3 location for query results and clean it periodically.

Athena and Glue give small teams a way to analyze large data sets without buying hardware or hiring database administrators.

Previous lesson
Back to course
Next lesson