Concept 2 of 4

Supervised, unsupervised and reinforcement learning

3 questions test this

A model learns in one of three ways, and which one applies is decided by the data you have rather than by the outcome you want.

Supervised learning

The training data carries the answer. Each example has a label, and the model learns to predict that label for input it has not seen.

Two tasks sit underneath it. Classification predicts a category, such as whether a transaction is fraudulent or which of five topics a support ticket belongs to. Regression predicts a number, such as next month's demand or the price a property will fetch.

Supervised learning is the workhorse, and its constraint is labels. Labels are expensive, they are often produced by people who disagree with each other, and a model can never be better than the labels it learned from.

Unsupervised learning

The data has no labels and the model finds structure on its own. Clustering groups similar records without anybody naming the groups, which is how customer segments emerge from behaviour rather than from assumption. Dimensionality reduction compresses many variables into a few that carry most of the signal. Anomaly detection learns what normal looks like and flags what is not.

The catch is evaluation. With no ground truth, there is no accuracy figure, and judging whether the clusters mean anything is a human job.

Reinforcement learning

An agent acts in an environment and receives rewards or penalties, and it learns a policy that maximises reward over time. There is no labelled dataset at all. There is a goal, a set of possible actions, and feedback.

It suits sequential decisions where each choice changes what comes next, such as robotics, game playing, or bidding. It appears in the syllabus mainly through reinforcement learning from human feedback, the step where people rank model outputs and those rankings train a reward model that shapes a language model's behaviour.

The data underneath

The exam separates data along two lines and expects you to place an example on both.

Labelled or unlabelled decides whether supervised learning is even available to you.

Structured or unstructured decides how it has to be handled. Structured data has a schema, which covers tabular records and time series where order carries meaning. Unstructured data does not, which covers text, images, audio and video, and is where deep learning earns its cost.

Choosing between them

The useful question is not which technique is most powerful, it is which one your situation admits.

If you have labelled examples of the thing you want to predict, use supervised learning. If you have data and no labels and you are looking for structure that might be there, use unsupervised learning. If the problem is a sequence of decisions with delayed consequences, and you can define a reward honestly, use reinforcement learning.

And if you can state the rule in a sentence, write the rule. A model that learns something you already knew costs more and explains less.

Common misconceptions

Clustering and classification are two words for sorting things into groups.

Classification puts items into groups you defined in advance and had labelled examples of. Clustering discovers groupings nobody named. Asking which one you need is really asking whether you have labels.

Unsupervised learning needs no data preparation because there are no labels.

It needs no labels, which is a different thing. Feature scaling, missing values and outliers matter more in unsupervised learning, not less, because there is no ground truth to reveal that the result is nonsense.

Reinforcement learning is what makes a chatbot helpful.

Reinforcement learning from human feedback is one step in aligning a language model, and it operates on a model that was already pre trained on text. The capability comes from the pre training. The reinforcement step shapes how that capability is expressed.

3 questions test this concept

A subscription business wants to understand whether its customers fall into natural groups it has not thought of, so that marketing can be tailored. It has three years of behavioural data and no existing segment labels. Which learning approach fits?

  • ASupervised learning, specifically classification.
  • BUnsupervised learning, specifically clustering.
  • CSupervised learning, specifically regression.
  • DReinforcement learning, because the marketing team will act on the result.
Check whether it stuck.

One per page, with a worked explanation.

Start the set
Related material
Book
Data Science for Business, On matching a business question to a learning task.
Book
Designing Machine Learning Systems, On what each learning type demands of your data pipeline.