Concept 6 of 7

The five layers of the gen AI landscape

3 questions test this

Google describes the gen AI market as five layers, from the hardware at the bottom to the finished products people use at the top. The layer an organisation buys at decides what it is responsible for, what it pays for and how quickly it can move, which is why the exam guide treats the five layers as a business question. The layers stack, so an application runs on an agent, an agent runs on a platform, a platform serves a model and a model runs on infrastructure.

The sections below take the layers from the bottom up, infrastructure, models, platforms, agents and applications, and close on how an organisation chooses the layer to buy at.

The infrastructure layer

Infrastructure is the compute, storage and networking that training and inference run on. For Google this means data centres, custom designed TPUs, GPUs and the AI Hypercomputer that arranges them as one system.

Almost nobody buys at this layer for a gen AI project. Infrastructure still decides two things a product team feels directly. It sets the price and the latency at which a hosted model is available, and it decides whether a given region can serve a given model at all.

The model layer

Models are the foundation models themselves. Google's are Gemini, Gemma, Imagen and Veo, and open and third party models sit alongside them.

Buying at the model layer means calling a model directly and building everything around it. A team that buys here has the most freedom on offer, and it owns the plumbing, the retrieval, the evaluation and the safety controls.

The platform layer

Platforms are the tooling around models, meaning everything a team needs to deploy them, tune them, ground them, evaluate them and govern what they do. Gemini Enterprise Agent Platform is Google's, and Model Garden, Agent Search and the tuning and evaluation tooling all sit within it.

Most engineering teams work at this layer, because the platform handles what every project has in common and leaves the team free to spend its time on the part that is particular to its own business.

The agent layer

Agents are systems that pursue a goal. An agent plans a sequence of steps, calls tools, reads what comes back and adjusts, so building one means deciding what it may do as well as what it may say.

The business implication is a change in risk. An agent that can act creates exposure a chat interface never had, because a prompt injection now reaches something that changes data.

The application layer

Applications are finished products people use without knowing a model is underneath. The Gemini app, Gemini for Google Workspace and the Customer Engagement Suite are all applications in this sense.

An application is the fastest layer to adopt and the one with the least to configure, and a whole organisation can see value from it within weeks.

Choosing a layer to buy at

The choice of layer follows from what an organisation is trying to differentiate on.

Where the differentiator is a workflow particular to the organisation, an application supplies the AI part and the effort belongs on the workflow. Where it is a customer experience no competitor can offer, the platform layer is the least the organisation needs and agents are usually part of it. Very few organisations differentiate on the model itself, and one that believes it does should be certain before committing.

The common error runs one way. Teams build at the model layer where an application would have done, and then spend a year rebuilding the retrieval, the evaluation and the safety controls that a layer above already had.

Common misconceptions

“Buying at a higher layer means giving up control.”

Buying at a higher layer means giving up control of things most organisations were never going to do well. What stays with the buyer is control of its data, its access rules and its use case, and those are the decisions that differentiate the result.

“An agent is just a model with a longer prompt.”

An agent plans a sequence, calls tools, reads what comes back and decides what to do next. That ability to act is what separates the layer, and it is why an agent raises security questions a chat interface never did.

“The layers are alternatives and an organisation picks one.”

They stack. An application runs on an agent, which runs on a platform, which serves a model, which runs on infrastructure. The choice is how far down the stack an organisation goes.

3 questions test this concept

A retailer wants a customer service assistant that answers from its own returns policy, checks an order in its warehouse system, and issues a refund where the policy allows it. Which layer of the gen AI landscape does this describe?

  • AModels, because a foundation model produces every response.
  • BAgents, because the system plans steps, calls tools and acts rather than only answering.
  • CInfrastructure, because it requires compute to serve requests.
  • DApplications, because customers interact with it directly.
Check whether it stuck.

One per page, with a worked explanation.

Start the set
Related material
Book
AI Engineering, On what a team actually owns at each layer.