Choosing an Operating Model
Compare centralized, hub-and-spoke, and federated AI operating models, decide what the center owns versus the business units, and design the core team a CAIO needs.
Who Does the Work?
Strategy says where AI should create value. The operating model says who builds, buys, runs, and governs AI, and how they work together. Get it wrong and you’ll either have a central team that’s a bottleneck or business units reinventing the same things with no guardrails.
Three Models
Centralized
A central AI team, often called a center of excellence (CoE), does most AI work for the whole organization.
- Strengths: consistent standards, efficient use of scarce talent, strong governance, easier reuse.
- Weaknesses: becomes a queue; distant from business problems; business units feel AI is “done to them.”
- Fits: organizations early in AI maturity, or with very scarce AI skills.
Federated
Each business unit runs its own AI teams with a light central function for policy.
- Strengths: close to the business, fast, strong ownership.
- Weaknesses: duplication, inconsistent risk practices, hard to share platforms or talent, difficult to see the whole portfolio.
- Fits: very large, diverse organizations with mature business units.
Hub-and-spoke
A central hub owns platform, standards, governance, and scarce expertise. Spokes, embedded in business units, own use cases and adoption.
- Strengths: balances consistency with business ownership; reuse through the hub, speed in the spokes.
- Weaknesses: needs clear division of responsibilities and strong relationships; can drift toward either extreme.
- Fits: most mid-to-large organizations once AI moves beyond early pilots.
Many organizations start centralized and move toward hub-and-spoke as skills spread and standards mature. Plan for that evolution rather than treating the first model as permanent.
What the Hub Owns vs. the Spokes
| The hub (central team) owns | The spokes (business units) own |
|---|---|
| AI strategy, portfolio view, and prioritization process | Use-case identification and business cases |
| Shared platform, model access, and approved tools | Process redesign and change in their area |
| Standards: security, evaluation, documentation | Day-to-day product ownership of their AI solutions |
| Governance, risk tiering, and review | Measuring and reporting value |
| Scarce expertise (ML engineering, evaluation, AI risk) | Domain experts and frontline adoption |
| Enterprise vendor contracts | Local champions and training |
Write this down and agree it with business-unit leaders. Ambiguity about who owns a failing AI product is a common source of conflict.
Common mistake: a strong central team keeps building every use case itself because it’s faster and higher quality than the business units’ attempts.
It works for a year. Then demand exceeds capacity, business units feel ignored, and the CAIO’s team becomes the reason AI is slow. The hub’s long-term job is to make the spokes capable: platforms, patterns, training, and review, not delivery of everything.
The Core Team
Whatever the model, a CAIO typically needs these capabilities, in-house or through partners:
- AI product management: turning business problems into well-scoped AI products with clear metrics.
- AI and ML engineering: building, integrating, and operating AI systems, including retrieval, agents, and evaluation.
- Platform and architecture: the shared infrastructure, model access, and integration patterns.
- Data partnership: close links to data engineering and governance, often in the CDO’s team.
- AI risk and governance: policy, risk assessment, model documentation, and regulatory tracking.
- Change and enablement: training, communications, champions networks, and adoption measurement.
Small, senior, and cross-functional beats large and specialized at the start.
- Describe your organization’s AI maturity in one sentence: early pilots, scattered production, or broad adoption?
- Pick the operating model that fits today, and the one you’d aim for in two years.
- Fill in a two-column hub-versus-spoke ownership table for your organization.
- List the three core-team roles you’d hire first, and why.
1. What's the main weakness of a fully centralized AI model?
Centralization gives consistency but can bottleneck delivery and lose touch with the business.
2. In a hub-and-spoke model, who typically owns process redesign in a business area?
Business units own their processes and adoption; the hub provides platform, standards, and expertise.
3. What's the hub's long-term job?
A hub that builds everything becomes the bottleneck. Its job is to enable.
Up Next
The next lesson covers the foundations the operating model runs on: data, platform, and cost.