Domain 5 of 7
Interpretability and Transparency Interpretability research, what it has established and what it has not. The documentation that stands in for understanding while that research catches up.
Concepts in this domain
01 Interpretability What the field has established about the inside of a model, what it has not, and why a system that explains itself is not explaining itself. 02 Transparency and explainability Why an accurate model nobody can explain is sometimes the wrong choice, the difference between the two words, and what a model card is for. 03 Corrigibility and control The property of a system that lets people correct, interrupt or shut it down, why optimisation works against it, and what keeping it costs in a product. Try a question from this domain
A model is asked to show its reasoning before answering, and produces a clear step by step account. What has been obtained?
A Generated text about the answer, which can be fluent and unrelated to what actually drove the output. B A faithful record of the computation that produced the answer. C An audit trail sufficient to satisfy a right to explanation. D Nothing useful, since chain of thought does not improve accuracy. 9 questions on this domain.
One per page, with a worked explanation.
Start the set