Data, Platform, and Cost
Assess data readiness, design a shared AI platform with a model gateway and evaluation, avoid lock-in in a fast-moving market, and manage AI costs before they surprise the CFO.
Data Is Still the Bottleneck
Modern AI models arrive pre-trained, so it’s tempting to think data matters less than it used to. It doesn’t. The model brings general capability; your data brings the value: the policies a support assistant cites, the contracts a review tool reads, the history a forecasting model learns from.
Ask four questions about the data behind any priority use case:
- Does it exist? Is the knowledge written down, or does it live in people’s heads?
- Can we reach it? Is it in systems AI can connect to, with usable access controls?
- Is it good enough? Current, consistent, and correct, or full of outdated versions and contradictions?
- Are we allowed to use it? Do privacy law, contracts, consent, or confidentiality limit this use?
For generative AI, “data readiness” often means content readiness: cleaning up document libraries, removing outdated policies, and fixing permissions so the AI doesn’t surface what users shouldn’t see. That last point is a common failure: an AI search tool that respects file permissions will still expose files that were shared too broadly years ago.
A Shared AI Platform
Without a platform, every team builds its own connections to models, its own logging, and its own guardrails. A shared platform gives you consistency and lets you change things once.
- 1 Apps and agents
Business-unit solutions
- 2 Guardrails
Content filters, data rules
- 3 Gateway
One door to many models
- 4 Models
Several providers and sizes
- 5 Data and retrieval
Governed, permission-aware
- 6 Evaluation and logs
Quality, cost, audit
Key components:
- A model gateway. All applications call models through one layer that handles authentication, routing, logging, cost tracking, and policy. It also lets you switch providers without rewriting every application.
- Approved models. A small menu of models for different needs (large and capable, small and cheap, specialized), reviewed for security and contract terms.
- Retrieval and data access. Standard, permission-aware ways to connect AI to enterprise content.
- Guardrails. Controls on sensitive data, harmful content, and, for agents, which actions they can take.
- Evaluation. Test sets and automated checks so teams can measure quality before release and detect regressions after model updates.
- Observability. Logs of inputs, outputs, and actions sufficient for debugging, audit, and incident investigation, retained according to policy.
Avoiding Lock-In
The AI market changes every few months: new models, price cuts, new capabilities. Protect your ability to change:
- Route through a gateway rather than calling providers directly from every application.
- Keep prompts, evaluation sets, and retrieval pipelines as your own assets, portable across models.
- Prefer shorter commitments or flexible terms for fast-moving technology, and negotiate data-use, retention, and exit terms carefully.
- Re-evaluate model choices on a regular cadence, using your own evaluation sets rather than vendor benchmarks.
Common mistake: letting each business unit sign its own AI contracts and connect directly to providers.
The result is fragmented spend, inconsistent data protection terms, no shared logs, and painful migrations when a better or cheaper model appears. Central contracts and a common gateway don’t slow teams down; they free them from repeating security and legal reviews.
Managing AI Cost
AI spending behaves differently from traditional software:
- Usage-based pricing means costs scale with adoption, and a popular tool can blow through its budget.
- Agents and long documents can use far more tokens per task than simple chat.
- Seat licenses for AI assistants are often bought broadly and used narrowly.
Practical controls, borrowed from cloud FinOps:
- Show cost per use case and per unit of work (cost per resolved ticket, per document reviewed), not just a total bill.
- Right-size models. Use smaller, cheaper models where they meet quality needs; reserve the largest models for tasks that need them.
- Cache and reuse where possible.
- Review seat utilization quarterly and reallocate unused licenses.
- Set budgets and alerts at the gateway so surprises are caught in days, not at quarter end.
Pick your top-priority use case from Lesson 2.2:
- Answer the four data questions. Which is the weakest?
- List which platform components it needs that don’t exist yet.
- Estimate its cost per unit of work, roughly, and compare it with the current cost of the process.
- Identify one lock-in risk and how you’d reduce it.
1. An AI search tool respects file permissions but surfaces sensitive files to many employees. What's the likely cause?
AI search exposes existing over-sharing. Fixing permissions is part of data readiness.
2. What's a key benefit of routing all model calls through a gateway?
A gateway centralizes control and reduces lock-in.
3. Which cost metric is most useful for managing AI spending?
Unit costs connect spending to value and show where right-sizing helps.
Up Next
Module 4 turns to governance: how to manage AI risk in a way that enables adoption rather than blocking it.