Domain 5 of 6

Monitoring and Incident Response

The stage teams skip. Drift, monitoring that watches quality rather than uptime, what a rollback criterion is for, and how to handle the day it produces something that reaches the press.

Concepts

~15%

Of the exam

Practice questions

Concepts in this domain

01AI incident responseWhat to do on the day it produces something it should not have, and the decisions worth making months before that day.02The machine learning lifecycleThe stages between a business question and a model in production, and why the work does not stop at deployment.03Transparency and explainabilityWhy an accurate model you cannot explain is sometimes the wrong choice, the difference between the two words, and what a model card is for.

Try a question from this domain

A live assistant has told several customers something incorrect about their entitlements. The system is up and almost every other answer is fine. What should happen first?

AReproduce the failure so the cause is understood before anything changes.
BDraft a customer communication for approval.
CStop the harm, by switching the feature off, restricting it or routing everything to human review.
DChange the prompt to prevent the specific wording that caused it.

7 questions on this domain.

One per page, with a worked explanation.

Start the set