A foundation model is a large model trained once on a very broad corpus and adapted afterwards to many different tasks. The lifecycle is the seven stages that carry such a model from raw data to a system in use, and the AWS exam guide names them as data selection, model selection, pre training, fine tuning, evaluation, deployment and feedback. Two of those stages belong to whoever built the model. A team building on somebody else's foundation model owns the other five, and knowing which five is what keeps a team from budgeting for work it will never do.
The sections below take the seven stages in order, saying at each one what the stage costs, who performs it and what it commits a team to.
Data selection
Data selection is the choice of what the model learns from. At foundation model scale that means trillions of tokens, and the decisions are about filtering, deduplication, quality and licensing.
The effects of data selection last longer than those of any other stage. Bias in the corpus becomes bias in the model, gaps in the corpus become gaps in what the model can do, and copyrighted material in the training data becomes a legal question that surfaces years later. A team that picks a model inherits all of it, which is why provenance is a reasonable thing to ask a vendor about.
Model selection
Model selection is the choice of an architecture and a size. For almost every team that means choosing among published models on capability, cost, licence, context length, modality and where the model is allowed to run.
Model selection is where the lifecycle starts for most teams, because whoever trained the model already chose its data.
Pre training
Pre training is the expensive stage, and the one almost nobody performs. The model learns language, and a great deal about the world, by predicting the next token across an enormous corpus, which costs millions of dollars and weeks of compute at scale. The model that comes out is knowledgeable and cannot yet follow an instruction, because predicting the next word is a different skill from answering a question.
Continued pre training is the affordable version of the same idea. An existing model is trained further on a large corpus from one field, so that it absorbs the vocabulary of medicine or law.
Fine tuning
Fine tuning adapts a pre trained model to behave a particular way, using a much smaller labelled dataset. Instruction tuning teaches the model to treat a prompt as an instruction and answer it. A model fresh from pre training treats the same prompt as text to carry on. Reinforcement learning from human feedback uses human rankings of output to train a reward model, and the reward model shapes later responses towards what people prefer.
Fine tuning changes behaviour, tone, format and domain fluency. Facts that change belong in retrieval, and the exam tests that boundary directly.
Evaluation
Evaluation covers benchmarks, human judgement and task specific measures. A foundation model is general, so no single score describes it, and a model that tops a public benchmark often behaves differently on one organisation's own work. The only evaluation that settles anything is the one run against the work the model will actually be given.
Deployment
Deployment puts the model where requests can reach it, either behind a managed API or on infrastructure the organisation runs itself. A managed API removes the operational work. Self hosting gives the team control over latency, cost at scale and data residency. The exam names four AWS services for this stage.
- Amazon Bedrock, for managed access to foundation models from several providers.
- SageMaker JumpStart, for deploying pre trained models into an account the organisation controls.
- Amazon Q, an assistant built on top of a model.
- PartyRock, a playground for trying prompts without writing code.
Managed access and a deployment the organisation runs itself differ on latency, on cost at scale and on where the data sits, and a scenario question usually asks which of those three matters most.
Feedback
Feedback is what real use sends back, and it feeds the next round of work. Explicit ratings, logged failures, human review of sampled output and drift in what people are asking for all belong to this stage.
Feedback is the stage teams most often leave out. Without a route that carries a bad answer back into the system, the only remaining way to improve the product is to change model.