Concept 3 of 4

The machine learning lifecycle

4 questions test this

A model is a small part of a machine learning system. The lifecycle describes everything around it, and most of the effort and most of the failures live in the stages before and after training.

Before the model

Data collection gathers what you will learn from, and decides the ceiling on everything that follows. A model cannot know something absent from its training data.

Exploratory data analysis is the look before the work. Distributions, missing values, obvious errors, and whether the data actually contains the signal you assumed. This is where most projects should stop and many do not.

Preprocessing cleans and shapes. Handling missing values, removing duplicates, normalising scales, encoding categories as numbers, and splitting into training, validation and test sets.

Feature engineering turns raw fields into things a model can use. A timestamp becomes day of week and hour. Two columns become a ratio. Domain knowledge enters the system here, and on classical machine learning problems it usually beats a cleverer algorithm.

Building the model

Training runs the algorithm over the training set to produce a model.

Hyperparameter tuning adjusts the settings you chose rather than the weights the model learned. Learning rate, network depth, number of trees. Tuning means training several times and comparing, which is why it costs what it does.

Evaluation measures performance on data the model has not seen. Judging a model on its training data tells you only how well it memorised.

Getting it to users

Deployment puts the model where requests can reach it. The syllabus names two shapes. A managed API service hosts the model for you and charges per call or per hour. Self hosting puts it on infrastructure you run, which costs more attention and gives more control over latency, data residency and cost at scale.

The other choice is when inference happens. Real time inference answers a request while somebody waits, and its constraint is latency. Batch inference scores a large set on a schedule, and its constraint is throughput. A fraud check at checkout is real time. Overnight churn scores for a mailing are batch, and paying for a low latency endpoint to produce them is waste.

After deployment

Monitoring watches the deployed model, and it watches two different things. Operational health covers latency, errors and cost. Model health covers accuracy against outcomes as they arrive, and drift, which is the world moving away from the data the model learned on. Drift is silent. Nothing breaks, the predictions simply get worse.

Where AWS puts the stages

Amazon SageMaker covers the pipeline end to end, and the exam expects a few components by name. SageMaker Data Wrangler for preparation, SageMaker Feature Store for reusable features, SageMaker Model Monitor for drift and quality after deployment, SageMaker Clarify for bias and explanations, and SageMaker JumpStart for pre trained models you can start from.

The instinct the exam rewards is to look for an existing model first. Sources of models run from a managed service that needs no training, through an open source pre trained model you adapt, to a custom model you train from scratch, and the cost rises steeply along that line.

Where Google Cloud puts the stages

The Google exam uses a shorter list of five stages and asks for the tool at each. Data ingestion into Cloud Storage or BigQuery. Data preparation with BigQuery and Dataflow. Model training and tuning on Gemini Enterprise Agent Platform, with Agent Platform AutoML where a custom predictive model is wanted without writing training code. Deployment to an Agent Platform endpoint. And model management, covering versioning, monitoring for drift and the Agent Platform Feature Store for reusable features.

The same instinct applies. Model Garden holds Google's models, open models and third party ones, and a prebuilt API frequently answers the problem before any of this is needed.

Common misconceptions

Deployment is the end of the project.

It is the point at which the model starts to decay. The world it learned from keeps moving, so monitoring for drift and retraining are part of the lifecycle rather than maintenance bolted on afterwards.

Hyperparameter tuning is part of training.

Parameters are learned during training. Hyperparameters are the settings you choose before it, such as learning rate or tree depth, and tuning means training repeatedly under different settings to compare them.

Building your own model is the normal path.

The exam expects the opposite instinct. A managed service or an open source pre trained model answers most problems, and custom training is what you reach for when neither fits.

4 questions test this concept

A team preparing to build a demand forecasting model converts a delivery timestamp into day of week, hour, and whether the date is a public holiday. Which stage of the machine learning lifecycle is this?

  • AFeature engineering.
  • BExploratory data analysis.
  • CHyperparameter tuning.
  • DModel evaluation.
Check whether it stuck.

One per page, with a worked explanation.

Start the set
Related material
Book
Designing Machine Learning Systems, On the lifecycle as an operational system rather than a project.
Book
Fundamentals of Data Engineering, On the data work that sits underneath every stage.
Template
Experiment brief, The belief being tested, a hypothesis with a threshold and a date, how long the test runs and on how much traffic, and a table saying in advance what you will do with each possible result.