Q22 — AWS SAA-C03 Ch.17

Question 22 of 89 | ← Chapter 17

Q1322. A company has an application that scans millions of connected devices for security threats and pushes the scan logs to an Amazon S3 bucket. A total of 70 GB of data is generated each week, and the company needs to store 3 years of data for historical reporting. The company must process, aggregate, and enrich the data from Amazon S3 by performing complex analytical queries and joins in the least amount of time. The aggregated dataset is visualized on an Amazon QuickSight dashboard. What should a solutions architect recommend to meet these requirements?

Correct Answer: A. Create and run an ETL job in AWS Glue to process the data from Amazon S3 and load it into Amazon Redshift. Perform the aggregation queries on Amazon Redshift.

Explanation

To meet the requirements of processing, aggregating, and enriching 70 GB/week of security scan logs (3 years retention) with complex analytical queries in the least amount of time, the best solution is:Correct Answer:A. Create and run an ETL job in AWS Glue to process the data from Amazon S3 and load it into Amazon Redshift. Perform the aggregation queries on Amazon Redshift.Why This is the Best Solution?\1. Amazon Redshift is Optimized for Complex Analytical Queries Redshift is a columnar, petabyte-scale data warehouse designed for fast aggregations, joins, and complex analytics.It uses massively parallel processing (MPP) to distribute queries across multiple nodes, reducing execution time.Best for:Large-scale aggregations (e.g., GROUP BY, JOIN, WINDOW functions). Historical reporting (3 years of data = ~10 TB, manageable in Redshift).Integration with Amazon QuickSight for dashboards.\2. AWS Glue Provides Efficient ETL for Structured/Semi-Structured Data AWS Glue is a serverless ETL service that:Crawls S3 data to infer schemas (or uses a custom schema).Transforms data (e.g., filtering, enrichment, normalization). Loads it into Redshift (or other targets like S3, RDS, etc.).Best for:Automating the data pipeline from S3 to Redshift.Handling incremental loads (e.g., daily/weekly updates).\3. QuickSight Integration with RedshiftAmazon QuickSight natively connects to Redshift for fast dashboarding without needing to move data again.Redshift's columnar storage and materialized views improve query performance for visualizations.Why Other Options Are Less Optimal?B. Use Lambda + DynamoDB for AggregationsDynamoDB is a NoSQL database, not optimized for complex analytical queries.Limitations:No native JOIN or GROUP BY support.High cost for large-scale analytics (scan operations are expensive).Not suitable for 3 years of historical data (10 TB+).C. Use Lambda + Aurora MySQL for AggregationsAurora MySQL is a transactional (OLTP) database, not optimized for large-scale analytics.Limitations:Poor performance for complex joins/aggregations on 10 TB of data.Higher cost compared to Redshift for analytics workloads.Not designed for columnar storage (Redshift is better for scans).D. Use AWS Glue + Athena for Direct S3 QueriesPartially correct, but not optimal for:Repeated complex aggregations (Athena scans raw S3 data each time, leading to high costs and latency). No materialized views (unlike Redshift, which can cache results).Best for: Ad-hoc queries, not high-performance dashboards.Use case: If the company only needs occasional queries, Athena is fine, but for daily/weekly aggregations, Redshift is better.Best Architecture for This ScenarioIngest: Scan logs are stored in Amazon S3 (cheap, scalable object storage). ETL: AWS Glue processes and transforms the data (e.g., parsing JSON, filtering, enriching). Load: Glue loads the transformed data into Amazon Redshift (optimized for analytics). Query & Visualize: Redshift runs aggregations, and QuickSight builds dashboards.Cost & Performance ConsiderationsRedshift is more cost-effective for large-scale analytics than DynamoDB/Aurora. Athena is cheaper for infrequent queries but expensive for repeated scans (10 TB ?many queries = high costs).Glue ETL is serverless, so costs scale with usage.ConclusionThe best solution is:A (AWS Glue + Redshift) for fast, scalable analytics on 3 years of security logs.Other options either lack analytical performance (B, C) or are too costly for repeated queries (D).