Q53 — AWS AIF-C01 Ch.3

Question 53 of 100 | ← Chapter 3

A global telecommunications company is building an AI-assisted tool to enhance customer chat capabilities. All generated chat responses must be positive, friendly, and unbiased. Which evaluation method will identify a foundation model (FM) that satisfies these requirements?

• A. Use Amazon SageMaker Clarify to identify bias in a historical customer chat dataset
• B. Review each model's sample benchmark performance using AWS AI Services Cards
• C. Use Amazon Augmented AI (A2I) to review responses before sending them to customers
• D. Use Amazon SageMaker Clarify to quantify model toxicity on a test dataset ✓

Correct Answer: D. Use Amazon SageMaker Clarify to quantify model toxicity on a test dataset

Explanation

To ensure AI-assisted chat responses meet requirements for positivity, friendliness, and lack of bias, model evaluation is essential. Amazon SageMaker Clarify is a tool that quantifies model toxicity — i.e., evaluates whether a model produces inappropriate, biased, or harmful outputs — on a test dataset. Quantifying model toxicity enables identification of foundation models (FMs) satisfying the stated requirements. Thus, using Amazon SageMaker Clarify to quantify model toxicity on a test dataset is the appropriate evaluation method.