Q4 — AWS SAA-C03 Ch.16

Question 4 of 100 | ← Chapter 16

Q1204. A solutions architect needs to optimize a large data analytics job that runs on an Amazon EMR cluster. The job takes 13 hours to finish. The cluster has multiple core nodes and worker nodes that are deployed on large,compute-optimized instances.After reviewing EMR logs, the solutions architect discovers that several nodes are idle for more than 5 hours while the job is running.The solutions architect needs to optimize cluster performance.Which solution will meet this requirement MOST cost-effectively?

• A. Increase the number of core nodes to ensure there is enough processing power to handle the analytics job without any idle time.
• B. Use the EMR managed scaling feature to automatically resize the cluster based on workload. ✓
• C. Migrate the analytics job to a set of AWS Lambda functions. Configure reserved concurrency for the functions.
• D. Migrate the analytics job core nodes to a memory-optimized instance type to reduce the total job runtime.

Correct Answer: B. Use the EMR managed scaling feature to automatically resize the cluster based on workload.

Explanation

To optimize a large data analytics job running on an Amazon EMR cluster, where the job currently takes 13 hours to finish and several nodes are idle for more than 5 hours during the job's execution, the most cost- effective solution would be:B. Use the EMR managed scaling feature to automatically resize the cluster based on workload.Analysis:EMR Managed Scaling Feature: This feature automatically adjusts the cluster size to achieve the best possible performance at the lowest possible cost. By continuously monitoring key metrics related to the workload, EMR Managed Scaling uses algorithms to optimize the cluster size, ensuring that resources are utilized efficiently. It automatically scales up the cluster during peak periods and scales it down during idle times, reducing costs and optimizing cluster capacity.Cost-Effectiveness: Compared to manually adjusting the cluster size or migrating to another service like AWS Lambda, EMR Managed Scaling can more accurately match the workload requirements, avoiding resource waste. It eliminates the need to predict workload patterns or write custom logic to scale the cluster, making it easy to set up a scalable cluster that significantly reduces operational costs. Applicability: For the current scenario where nodes are idle for extended periods, EMR Managed Scaling can dynamically adjust the cluster size to ensure that only the necessary nodes are active during the job's execution, thereby reducing idle time and costs.Why Other Options Are Not the Best Choice:A. Increase the number of core nodes: Increasing the number of core nodes might improve processing power but would also increase costs. Since there are already nodes that are idle, adding more nodes is not the most cost-effective solution.C. Migrate the analytics job to AWS Lambda functions: AWS Lambda is suitable for short-lived, stateless tasks and is not designed for long-running, large-scale data analytics jobs. Additionally, migrating to Lambda might require rewriting the code and could face limitations such as the maximum execution time for Lambda functions (15 minutes).D. Migrate the analytics job core nodes to memory-optimized instance types: While using memory- optimized instance types might reduce the total job runtime, it does not address the issue of idle nodes. Moreover, memory-optimized instances are typically more expensive than compute-optimized instances, making them potentially less cost-effective.