Generative AI has a shape. Knowing it saves you from the two expensive mistakes, which are using it where it does not belong and designing as though its failure modes were avoidable.
What it is genuinely good at
Adaptability. One model handles summarising, drafting, classifying, extracting and translating with no retraining. Before this you needed a separate model per task, each with its own labelled dataset.
Speed to a working thing. A prompt can stand in for what used to be a project. That collapses the cost of finding out whether an idea is worth pursuing, which is the largest practical change for a product team.
Unstructured input. Free text, images and audio were the hardest data to work with and are now the easiest. Extracting structure from a mess of support transcripts is the case where the technology is clearly the right tool.
The use cases the syllabus names follow from those. Summarisation, chatbots and customer service agents, translation, code generation, search, recommendation, and generating images, video and audio.
What it is bad at, permanently
Hallucination is confident output that is wrong. It is not a bug being worked on, it follows from the objective. The model generates the most plausible continuation, and plausibility is not truth. Newer models do it less and none of them stop, so anything where a wrong answer is costly needs grounding, citation, or a person in the path.
Nondeterminism. The same prompt can return different text. This breaks assumptions people carry over from ordinary software, including caching, exact match tests and reproducible support investigations. Lowering temperature narrows the variation and does not remove it.
Interpretability. You cannot get a faithful account of why the model produced what it did. A model asked to explain itself generates a plausible explanation, which is a separate act of generation rather than a window into the first one. Where a regulator or a customer is owed a reason, this is disqualifying on its own.
Arithmetic and exact recall. A system predicting text is a poor calculator and an unreliable database. Both should be delegated to a tool.
Cost and latency. Orders of magnitude more of both than a classifier doing the same job, which matters at volume.
Choosing a model, and choosing whether to
The syllabus lists the factors for picking between models. Model type and modality, performance against your actual task, capability against cost, and compliance constraints such as where inference happens and whether your input can be used for training.
The prior question is whether a foundation model is right at all. If the task is a fixed prediction over structured data, a classical model wins on every axis. If the output must be identical every time, this technology cannot promise it. If you cannot tolerate a wrong answer and cannot afford review, neither the model nor a better prompt will save the design.
Metrics that mean something
What is asked for is business value rather than model scores. Cross domain performance, efficiency, conversion rate, average revenue per user, accuracy against a human baseline, and customer lifetime value.
The useful framing is the counterfactual. Not whether output is impressive, but whether it beats what somebody was doing before, at a cost the result justifies.
Practise this
You need any ordinary chat client, fifteen minutes, and ten questions from your own field where you already know the correct answer.
Write the ten before you open the client. Seven should be real questions with a definite answer you can verify, and three should be questions whose true answer is that no such thing exists, such as the clause number of a standard your industry does not have. Ask each in a fresh conversation, then score the reply against what you already knew.
Ten questions, written before the client is opened.
1 to 7 real questions in my field, answers I can verify
8 to 10 questions with no true answer, for example the number
of a standard that does not exist, the date of a
release that never happened, or the author of a paper
nobody wrote
Q Right Wrong, hedged Wrong, confident Said it
did not know
1 ____ ____ ____ ____
...
10 ____ ____ ____ ____
Wrong and confident, out of 10 ____
Of questions 8 to 10, how many invented ____
The three impossible questions carry most of the signal. A model that answers them fluently shows the failure mode in its purest form, since there was nothing to be right about and it produced something anyway. Notice where the wrong answers cluster as well, because the boundary is rarely your whole domain and is usually the recent, the internal and the very specific, meaning anything published after training, anything private to your organisation, and anything too niche to appear often.
Ten questions in your own subject show where the boundary sits for what you are actually shipping, which no general benchmark score can tell you.