Data, Platform, and Cost
Module 3 · Operating Model and Platform

Data, Platform, and Cost

Assess data readiness, design a shared AI platform with a model gateway and evaluation, avoid lock-in in a fast-moving market, and manage AI costs before they surprise the CFO.

Data Is Still the Bottleneck

Modern AI models arrive pre-trained, so it’s tempting to think data matters less than it used to. It doesn’t. The model brings general capability; your data brings the value: the policies a support assistant cites, the contracts a review tool reads, the history a forecasting model learns from.

Ask four questions about the data behind any priority use case:

  1. Does it exist? Is the knowledge written down, or does it live in people’s heads?
  2. Can we reach it? Is it in systems AI can connect to, with usable access controls?
  3. Is it good enough? Current, consistent, and correct, or full of outdated versions and contradictions?
  4. Are we allowed to use it? Do privacy law, contracts, consent, or confidentiality limit this use?

For generative AI, “data readiness” often means content readiness: cleaning up document libraries, removing outdated policies, and fixing permissions so the AI doesn’t surface what users shouldn’t see. That last point is a common failure: an AI search tool that respects file permissions will still expose files that were shared too broadly years ago.


A Shared AI Platform

Without a platform, every team builds its own connections to models, its own logging, and its own guardrails. A shared platform gives you consistency and lets you change things once.

  1. 1 Apps and agents

    Business-unit solutions

  2. 2 Guardrails

    Content filters, data rules

  3. 3 Gateway

    One door to many models

  4. 4 Models

    Several providers and sizes

  5. 5 Data and retrieval

    Governed, permission-aware

  6. 6 Evaluation and logs

    Quality, cost, audit

Key components:

  • A model gateway. All applications call models through one layer that handles authentication, routing, logging, cost tracking, and policy. It also lets you switch providers without rewriting every application.
  • Approved models. A small menu of models for different needs (large and capable, small and cheap, specialized), reviewed for security and contract terms.
  • Retrieval and data access. Standard, permission-aware ways to connect AI to enterprise content.
  • Guardrails. Controls on sensitive data, harmful content, and, for agents, which actions they can take.
  • Evaluation. Test sets and automated checks so teams can measure quality before release and detect regressions after model updates.
  • Observability. Logs of inputs, outputs, and actions sufficient for debugging, audit, and incident investigation, retained according to policy.

Avoiding Lock-In

The AI market changes every few months: new models, price cuts, new capabilities. Protect your ability to change:

  • Route through a gateway rather than calling providers directly from every application.
  • Keep prompts, evaluation sets, and retrieval pipelines as your own assets, portable across models.
  • Prefer shorter commitments or flexible terms for fast-moving technology, and negotiate data-use, retention, and exit terms carefully.
  • Re-evaluate model choices on a regular cadence, using your own evaluation sets rather than vendor benchmarks.
Without a gatewayWith a gatewayApp 1Model AApp 2Model BApp 3Model CGatewayApp 1Model AApp 2Model BApp 3Model CEvery app wires itself to every providerOne place for logs, cost, policy, switching
⚠️ Common Pitfall: Every Team Picks Its Own Model and Vendor

Common mistake: letting each business unit sign its own AI contracts and connect directly to providers.

The result is fragmented spend, inconsistent data protection terms, no shared logs, and painful migrations when a better or cheaper model appears. Central contracts and a common gateway don’t slow teams down; they free them from repeating security and legal reviews.


Managing AI Cost

AI spending behaves differently from traditional software:

  • Usage-based pricing means costs scale with adoption, and a popular tool can blow through its budget.
  • Agents and long documents can use far more tokens per task than simple chat.
  • Seat licenses for AI assistants are often bought broadly and used narrowly.

Practical controls, borrowed from cloud FinOps:

  • Show cost per use case and per unit of work (cost per resolved ticket, per document reviewed), not just a total bill.
  • Right-size models. Use smaller, cheaper models where they meet quality needs; reserve the largest models for tasks that need them.
  • Cache and reuse where possible.
  • Review seat utilization quarterly and reallocate unused licenses.
  • Set budgets and alerts at the gateway so surprises are caught in days, not at quarter end.
🧪 Try It: Check Readiness for One Use Case

Pick your top-priority use case from Lesson 2.2:

  1. Answer the four data questions. Which is the weakest?
  2. List which platform components it needs that don’t exist yet.
  3. Estimate its cost per unit of work, roughly, and compare it with the current cost of the process.
  4. Identify one lock-in risk and how you’d reduce it.
🧠 Knowledge Check

1. An AI search tool respects file permissions but surfaces sensitive files to many employees. What's the likely cause?

2. What's a key benefit of routing all model calls through a gateway?

3. Which cost metric is most useful for managing AI spending?

Up Next

Module 4 turns to governance: how to manage AI risk in a way that enables adoption rather than blocking it.