Concept 2 of 3

What generative AI is good and bad at

6 questions test this

Generative AI has a shape. Knowing it saves you from the two expensive mistakes, which are using it where it does not belong and designing as though its failure modes were avoidable.

What it is genuinely good at

Adaptability. One model handles summarising, drafting, classifying, extracting and translating with no retraining. Before this you needed a separate model per task, each with its own labelled dataset.

Speed to a working thing. A prompt can stand in for what used to be a project. That collapses the cost of finding out whether an idea is worth pursuing, which is the largest practical change for a product team.

Unstructured input. Free text, images and audio were the hardest data to work with and are now the easiest. Extracting structure from a mess of support transcripts is the case where the technology is clearly the right tool.

The use cases the syllabus names follow from those. Summarisation, chatbots and customer service agents, translation, code generation, search, recommendation, and generating images, video and audio.

What it is bad at, permanently

Hallucination is confident output that is wrong. It is not a bug being worked on, it follows from the objective. The model generates the most plausible continuation, and plausibility is not truth. Newer models do it less and none of them stop, so anything where a wrong answer is costly needs grounding, citation, or a person in the path.

Nondeterminism. The same prompt can return different text. This breaks assumptions people carry over from ordinary software, including caching, exact match tests and reproducible support investigations. Lowering temperature narrows the variation and does not remove it.

Interpretability. You cannot get a faithful account of why the model produced what it did. A model asked to explain itself generates a plausible explanation, which is a separate act of generation rather than a window into the first one. Where a regulator or a customer is owed a reason, this is disqualifying on its own.

Arithmetic and exact recall. A system predicting text is a poor calculator and an unreliable database. Both should be delegated to a tool.

Cost and latency. Orders of magnitude more of both than a classifier doing the same job, which matters at volume.

Choosing a model, and choosing whether to

The syllabus lists the factors for picking between models. Model type and modality, performance against your actual task, capability against cost, and compliance constraints such as where inference happens and whether your input can be used for training.

The prior question is whether a foundation model is right at all. If the task is a fixed prediction over structured data, a classical model wins on every axis. If the output must be identical every time, this technology cannot promise it. If you cannot tolerate a wrong answer and cannot afford review, neither the model nor a better prompt will save the design.

Metrics that mean something

The exam asks for business value rather than model scores. Cross domain performance, efficiency, conversion rate, average revenue per user, accuracy against a human baseline, and customer lifetime value.

The useful framing is the counterfactual. Not whether output is impressive, but whether it beats what somebody was doing before, at a cost the result justifies.

Common misconceptions

Hallucination is a defect that will be fixed in a later model.

It follows from how the technology works. The model produces the most plausible continuation, and plausible and true are different properties. Better models hallucinate less often and none of them stop, so a design that depends on it never happening is unsound.

Setting temperature to zero makes a model deterministic.

It makes sampling greedy, which removes most of the variation. Identical output is still not guaranteed, because floating point behaviour, batching and the provider updating the model underneath you all move the answer.

Generative AI is the more advanced choice, so it is the better one.

For a fixed prediction on structured data, a classical model is cheaper, faster, more accurate and explainable. Reaching for generative AI there is a downgrade dressed as an upgrade.

6 questions test this concept

An assistant that answers policy questions occasionally states a refund window that does not exist in any company document, phrased with complete confidence. An engineer proposes waiting for the next model release to fix it. What is the most accurate assessment?

  • AThe behaviour is a defect in this particular model and a newer release will remove it.
  • BThe behaviour indicates the model was fine tuned on incorrect data and should be retrained.
  • CSetting temperature to zero will remove the behaviour entirely.
  • DHallucination follows from how the technology works, so the system needs grounding, citation or human review rather than a newer model.
Check whether it stuck.

One per page, with a worked explanation.

Start the set
Related material
Book
AI Engineering, On designing around the failure modes rather than hoping past them.
Book
Prediction Machines, On judging where cheap prediction actually changes a decision.
Template
Experiment brief, The belief being tested, a hypothesis with a threshold and a date, how long the test runs and on how much traffic, and a table saying in advance what you will do with each possible result.