Domain 2 of 6
Training and Alignment of Models Pre training, supervised fine tuning, reinforcement learning from human feedback and constitutional methods. What each stage adds, and why a helpful model is a trained artefact rather than an emergent one.
Concepts in this domain
01 How models are trained to behave Pre-training, supervised fine tuning, reinforcement learning from human feedback and constitutional methods, and why a helpful model is a manufactured artefact rather than an emergent one. 02 Foundation models and how they work Tokens, embeddings and transformers explained for somebody who has to make decisions about a model rather than build one. 03 Supervised, unsupervised and reinforcement learning The three ways a model learns, what each one needs from your data, and how to tell which problem you are actually holding. Try a question from this domain
What does a base model do before any further training is applied?
A It answers questions, but refuses nothing and holds no conversation. B It behaves as a helpful assistant, and less reliably than a tuned model. C It produces random output until it is given examples to imitate. D It continues text, and does not answer questions, refuse anything or hold a conversation. 9 questions on this domain.
One per page, with a worked explanation.
Start the set