Demoing AI is the easy part. Putting it in front of thousands of people, tying it to outcomes, and keeping it safe when models and data sprawl is the work. These five episodes from the Chain of Thought collection for enterprise AI leaders each point at one decision developers and product engineers control, even when leadership wants speed.
How do you deploy agents company-wide without a playbook?
How Block Deployed AI Agents to 12,000 Employees in 8 Weeks w/ MCP is the clearest story of breadth plus speed: Block put its open-source agent Goose in front of roughly 12,000 employees in about eight weeks, with Angie Jones describing deployment with no playbook already written.
For builders, the practical split is adoption versus control. Block leaned on security guardrails so wide rollout stayed defensible, and supported letting employees choose their own models (Claude, GPT, Gemini, plus open and local options in the discussion) because that choice drove uptake. MCP shows up as the connective tissue for hooking agents to real systems, with human-in-the-loop controls, OAuth, and tool annotations called out as part of the guardrail stack.
If you are past a pilot, I would treat model lock-in and missing guardrails as ship blockers before you chase more use cases. Ship one agent path with explicit approval steps for destructive tools, then expand connectors.
Why does so much enterprise AI spend fail to show ROI?
Why Most Enterprise AI Projects Fail to Show ROI still works as the straight ROI entry point. HP, ServiceNow, and Accenture walk through why heavy investment often ends in a vague "what did we get?" question instead of a number anyone trusts.
The builder takeaway is to pick use cases you can prioritize, implement, and measure, and to treat trust and explainability as part of delivery from the start. Alex Klug at HP frames risk in customer-facing terms: nobody wants to deploy something that steers customers wrong. That is a useful filter when product asks for AI in every surface.
Before you add another copilot button, write down who gets hurt if the model is wrong and whether you can detect and roll back bad outputs. If you cannot, you are still in pilot clothing.
How do you find where AI pays off instead of token maxing?
Stop Token Maxxing: Find Where AI Actually Pays Off with Jiaona Zhang (CPO at Laurel) targets a common failure mode: top-down "use AI everywhere" mandates that push teams to burn tokens without learning what returned time or revenue.
Zhang's through-line is measurement over mandate. Laurel runs its own product on itself and uses time data to see where AI helps versus where teams only assume it helps. The episode also ties efficient use to org design (lean teams, non-engineers shipping with agents, bottom-up plus top-down re-architecture), but the decision rule for developers is simpler: instrument time before you celebrate usage stats.
Sketch (illustration only, not Laurel's production code):
# Sketch: log task time before/after AI assist for one workflow
events = [] # {"workflow": str, "minutes": float, "used_ai": bool}
def median_minutes(workflow: str, used_ai: bool) -> float:
times = [e["minutes"] for e in events if e["workflow"] == workflow and e["used_ai"] == used_ai]
times.sort()
return times[len(times) // 2] if times else float("nan")
Compare medians per workflow with and without AI. Kill mandates that raise token charts but flatline time saved.
What if the agent calls the model fine but cannot reach the data?
Data Federation, Not Centralization, Is What Enterprise AI Needs with Jitender Aswani (Starburst) argues that stalled enterprise AI often loses a hidden fight: queries to the model work, access to data spread across dozens or hundreds of sources does not. The collection summary puts typical enterprises at 52 to 200 data sources; centralize-everything strategies pile ETL debt and brittle pipelines when SaaS schemas shift.
Aswani treats fragmentation as something you design around. Federation and query layers are his architectural direction, and MCP shows up in the episode as a driver of query volume. For application developers, the lesson is to stop blocking pilots on a mythical single lake and start exposing governed, federated access paths agents can call.
When an agent "hallucinates finance numbers," check data reachability before you fine-tune the prompt.
What separates enterprise AI that ships from AI that demos?
Inside IBM's watsonx: Building Enterprise AI That Ships with Dr. Maryam Ashoori, Head of Product for watsonx AI at IBM, focuses on trustworthy, scalable delivery across tools and regulations. Ashoori discusses simplifying the AI stack, automating pipelines, data management and lineage, scaling agentic systems, and effective agent evaluation, backed by survey findings from 1,000 developers on how enterprise AI implementation is going in the wild.
I'd take "ships" to mean repeatable pipelines, eval systems you trust enough to gate releases, and cost awareness as you scale agents. Demos skip lineage and eval; production cannot.
Checklist
- Pair wide agent rollout with explicit security guardrails and model choice where that choice drives adoption (Block/Goose pattern).
- Prioritize ROI-ready use cases only where trust, explainability, and customer harm if wrong are spelled out (HP/ServiceNow/Accenture panel themes).
- Replace "AI everywhere" mandates with time-based measurement per workflow so you stop token maxing (Jiaona Zhang, Laurel).
- Assume 52 to 200 data sources and plan federation and governed access instead of waiting for a finished centralization project (Jitender Aswani, Starburst).
- Treat simplified stacks, automated pipelines, lineage, and agent evals as shipping requirements (Maryam Ashoori, IBM).
More on this topic, with the related episodes, is on Chain of Thought. It draws on this episode.
Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.
Drafted with AI assistance from the episode transcripts.
Top comments (0)