Tokens, embeddings and transformers, what a foundation model can and cannot be relied on to do, and the lifecycle that takes one from raw data to a deployed system with feedback coming back.
A finance team is forecasting the cost of a customer support assistant built on a hosted foundation model. They plan to budget per active user per month. What should they be told about how the cost actually behaves?
ACost is billed per user session, so the per user assumption is broadly correct.
BCost is fixed once the model is deployed, because inference does not consume metered resources.
CCost depends on the number of parameters in the model rather than on usage.
DCost is billed per token of input and output, so it scales with conversation length and document size rather than with user count.