A model does not contain a copy of its training data, and it can reproduce parts of it. Both statements are true and the gap between them is where the privacy problem lives.
What memorisation is
Training on a corpus produces a model that generalises. Along the way it also memorises, retaining some sequences closely enough to reproduce them.
The strongest driver is repetition. Text appearing many times across a corpus is far more likely to come back verbatim, which is why licence headers, common code and widely copied passages emerge readily.
Rarity matters too, for the opposite reason. An unusual record has nothing similar for the model to blend it with, so it is retained more distinctly than a typical one.
And larger models memorise more, so the property strengthens with the scale that also brings capability.
Extraction
Researchers have repeatedly pulled training data out of production models using nothing but prompting, recovering personal information, code and long passages of text. Some techniques are inelegant, including asking a model to repeat a word indefinitely until it drifts into reciting training data.
Two things follow. Training data is not private simply because the corpus is not published. And the same applies to anything you fine tune on, so a model tuned on customer records is a system that may recite customer records.
Membership inference
A weaker attack with a sharper edge. Rather than recovering a record, an attacker establishes whether a particular record was in the training set.
That can be the sensitive fact by itself. If the training set was patients with a diagnosis, membership discloses the diagnosis, and no content needs to be recovered at all.
Differential privacy
The principal formal defence. Noise is added during training so that the finished model is provably almost unchanged by any single record, which bounds what can be inferred about any individual.
It is a real guarantee rather than a heuristic, and it costs accuracy. Stronger privacy means more noise means a weaker model, and the tradeoff is a decision rather than a setting. It is used where the data is sensitive enough to justify the loss.
What actually reduces the risk
Not collecting it. The only complete defence, and the one most often skipped.
Deduplicating the corpus. Since repetition drives memorisation, removing duplicates reduces it substantially and costs nothing in capability.
Removing sensitive records before training rather than filtering output afterwards, because output filtering catches what it recognises.
Retrieval instead of fine tuning. Facts kept in an index rather than in weights can be deleted, access controlled, and corrected. This is the practical answer for most product teams, and it is why the choice between retrieval and fine tuning is a privacy decision as much as an engineering one.
Anonymisation and pseudonymisation, which help and are not sufficient, because combinations of ordinary fields identify people even after direct identifiers are gone.