Concept 1 of 2

Responsible AI in practice

6 questions test this

Responsible AI is often discussed as ethics and examined as engineering. The syllabus names six properties and expects you to know how each is detected and what is done about it.

The six properties

Bias is systematically different treatment of groups. It arrives from the training data, from how the problem was framed, and from who was in the room.

Fairness is whether that difference is defensible. The formal definitions conflict, so this stays a judgement about the situation rather than a threshold.

Inclusivity is whether the system works for the range of people who will use it, including those poorly represented in training data.

Robustness is whether it holds up on input it was not designed for, and under deliberate attempts to break it.

Safety is whether it can cause harm, directly through its output or indirectly through what people do with it.

Veracity is whether the output is true, which for generative systems is a property that has to be engineered rather than assumed.

Where bias comes from

The data. Historical records encode historical decisions. A hiring model trained on who was hired before learns who was hired before, including the reasons nobody would defend.

Representation. Groups thin in the training data get worse results, and an aggregate accuracy figure hides this completely.

Proxies. This is the one that catches people. Removing a protected attribute does not remove its influence, because other fields correlate with it. Dropping the column removes your ability to measure the gap while leaving the gap.

Feedback loops. A model that predicts where problems will be found sends people there, they find problems there, and that becomes tomorrow's training data.

The way you see any of it is by measuring performance separately for each group. A model at 94 per cent overall can be at 97 for one group and 71 for another, and the aggregate number will never tell you.

Bias and variance

The statistical pair, which the syllabus also names. High bias is a model too simple for the pattern, underfitting and doing poorly everywhere. High variance is a model too sensitive to its training data, overfitting and failing on anything new. Both damage subgroups first, because that is where the data is thinnest.

Datasets

The characteristics the exam names are inclusivity, diversity, curated sources and balance. The practical reading is that curation is where most of the improvement is available, and it is unglamorous work done before any model exists.

Legal exposure

Generative AI brings risks the syllabus lists directly.

Intellectual property. Models trained on scraped material can produce output resembling their training data, and who owns generated output is unsettled in several jurisdictions.

Biased output creates discrimination liability where the system touches employment, credit, housing or insurance.

Hallucination becomes misrepresentation once a customer relies on it.

End user risk is what somebody does with confidently wrong advice.

Loss of trust is the one with no legal remedy and the longest tail. A system caught being wrong in public rarely gets a second hearing.

Tools on AWS

SageMaker Clarify measures bias in data and in a trained model, before and after training, and produces feature attributions. SageMaker Model Monitor watches for drift and quality decay once the model is live. Guardrails for Amazon Bedrock filters harmful content, blocks named topics and screens both input and output. Amazon Augmented AI routes low confidence cases to people for review.

The exam also asks about sustainability, which is a responsible AI consideration. Model size, training runs and inference volume all carry an energy cost, and choosing a smaller model that does the job is the decision that follows.

Common misconceptions

Removing protected attributes from the training data removes bias.

Other fields stand in for them. Postcode carries ethnicity, first name carries gender, purchase history carries both. Dropping the column removes your ability to measure the disparity while leaving the disparity in place.

Fairness is a threshold a model either meets or fails.

The formal definitions of fairness conflict with each other, and it has been shown that several cannot hold at once except in degenerate cases. Which definition applies is a decision about the situation, not a setting.

Responsible AI is a review that happens before launch.

Bias appears as populations shift, so a system fair at launch can drift. It is a monitoring commitment with a named owner rather than a gate you pass through once.

6 questions test this concept

A lending model is found to approve applicants from one postcode group at a much lower rate. The team removes ethnicity from the training data and retrains. Approval rates remain skewed. What is the most accurate explanation?

  • AThe model needs more training data before the disparity will resolve.
  • BThe remaining skew reflects real differences and is therefore fair.
  • CThe model is underfitted and needs a more flexible architecture.
  • DOther fields act as proxies, so removing the attribute removed the ability to measure the disparity rather than the disparity itself.
Check whether it stuck.

One per page, with a worked explanation.

Start the set
Related material
Book
Weapons of Math Destruction, On models that scale harm quietly.
Book
Data and Goliath, On data collection and the asymmetry it creates.
Template
Experiment brief, The belief being tested, a hypothesis with a threshold and a date, how long the test runs and on how much traffic, and a table saying in advance what you will do with each possible result.