Conrad Bengaluru | Notes from a day of keynotes, customer stories, and far too much coffee*
I spent a full day at Microsoft's Cloud and AI Innovation Day in Bengaluru, and it was one of those rare conference days where the vendor pitch and the practitioner reality actually lined up. As someone who lives inside Azure, Fabric, Logic Apps, and AI Foundry every working day, I went in looking for two things: where Microsoft's platform is heading, and what I can steal for my own stack. I came back with plenty of both.
Here's my writeup — the big picture first, then the sessions and speakers, and finally my personal learnings and the ideas I scribbled down on hotel notepaper between sessions.
The Big Picture: Microsoft IQ and "Intelligence You Own"
The framing for the entire day was the Microsoft IQ Platform — Microsoft's new umbrella for the intelligence layer that sits across its entire stack. The platform slide put Microsoft IQ at the centre, surrounded by Microsoft 365 Copilot, Copilot Studio, GitHub Copilot, Microsoft Fabric, Microsoft Foundry, and the new Agent 365 — all sitting on Azure with security as the foundation.
The line that stuck with me: "Intelligence you own, trust you control — one connected platform." Microsoft is now describing its stack as four layers:
- Copilot — where AI meets work (M365 Copilot, Copilot Studio, GitHub Copilot, Cowork)
- Microsoft IQ — the intelligence layer you own (Work IQ, Fabric IQ, Foundry IQ)
- Foundry + Fabric — model choice, your data and semantics, agents grounded in context ("rent or own the engine")
- Agent 365 + Security — identity, governance, and control for every agent
The message is clear: the era of bolting a chatbot onto your data is over. Microsoft wants the intelligence layer itself — the semantics, the ontology, the agent identity — to live inside your tenant, governed like any other enterprise asset.
Another keynote slide that resonated: "Where do organizations get stuck?" It showed the AI maturity curve — Intent → Proof of Concept → Production AI in Periphery → Production AI in Core Processes → Agentic AI Systems. The honest observation was that most organizations stall between PoC and production because governance, development, and infrastructure readiness lag behind enthusiasm. Only when all dimensions (benefits, governance, development, infrastructure, users) go green do you graduate to agentic systems. Having lived through exactly this journey internally, I found this uncomfortably accurate.
Theme 1: Fabric Is Becoming the Semantic Brain, Not Just the Data Platform
Fabric got the most stage time, and the positioning has clearly shifted. It's no longer "the unified data platform" — it's "the unified data platform for AI transformation." The new pieces layered on top of the familiar OneLake/Copilot/Governance foundation:
- Fabric IQ — the intelligence layer within Fabric
- Semantic Models — now positioned as the on-ramp to ontologies
- Ontology (public preview) — a live, unified model of how your business works
- Digital Twin Builder, Graph, Data Agents, and Operations Agents
The Fabric IQ – Ontologies session was the standout for me. The pitch: define how your business works once (entities, relationships, org-wide goals and rules), and every AI agent gets that context for trusted actions. You can jumpstart ontology creation from the 20M+ semantic models already out there. Ontology sits between your tables/streams and operational systems on one side, and AI agents and Teams on the other.
The "AI across Fabric workloads" slide showed Copilot capabilities now embedded across every workload — T-SQL generation in Databases and Warehouse, dataflow generation in Data Factory, notebook assistance in Data Engineering, KQL generation in Real-Time Intelligence, report generation in Power BI, and ontology access through Fabric data agents feeding into Copilot.
And OneLake as the anchor: the "Anchor AI on enterprise intelligence in OneLake" slide showed shortcut/mirroring connectivity from practically everything — AWS S3, BigQuery, GCP, Snowflake, Oracle, Databricks, Cosmos DB, SQL MI — up into one lake feeding M365 Copilot, Fabric, Azure ML, Foundry, and Copilot Studio.
My take: we already run our Global Data Lake on Fabric with Medallion architecture. The Ontology preview is the missing layer between our warehouse and our agents — instead of hand-feeding schema context into every agent prompt, the ontology becomes the shared brain.
Theme 2: Agents Are Getting Job Descriptions
Sandeep Alur's session on agentic AI was the most conceptually useful of the day. Two frameworks stood out.
The Agent Job Description. Think of every agent like an employee with a formal JD:
- What — Automations (workflows), Skills
- How — Integrations (MCP), Instructions (SOPs)
- Identity — Personality, Memory
This is a genuinely good mental model. When an agent misbehaves, you debug it the way you'd coach an employee: is the problem its skills, its instructions, its integrations, or its memory?
The Enterprise Learning Loop ("Hill Climbing Machine"). The architecture stacked three layers:
- Agentic System — memory, skills, traces, workflows, knowledge, instructions, tools, private evals
- Pluggable Model Layer — GPT, MAI, Claude, Llama, Gemini, DeepSeek, Mistral (models are swappable commodities)
- Optimization Engine — RLHF, RLE, RFT, distillation driven by reward functions
You can optimize the model, the agentic system, or both together — continuously hill-climbing. Notably, questions from the audience (and my own notes) touched on: are agents truly running on your behalf, what MAI (Microsoft AI's in-house models) means for the ecosystem, and pointers to check out o4-mini and Microsoft's notebook tooling.
Microsoft Foundry as the AI App and Agent Factory completed the picture — six spokes around the Foundry hub: orchestration, models, knowledge and tools, observability and agent control, fine-tuning and customization, and deployment to local and edge.
Theme 3: Frontier Tuning — Fine-Tuning Grows Up
A dedicated session defined Frontier Tuning as "the disciplined adaptation of frontier-class foundation models to a specific enterprise domain — combining advanced fine-tuning and parameter-efficient methods with high-quality data, rigorous evaluation and safety, then a continuous-improvement loop."
The 10-step lifecycle: define objectives → collect and prepare enterprise data → select frontier model → choose tuning approach (SFT / LoRA / QLoRA, reinforcement fine-tuning, preference optimization) → prepare training pipeline → configure training → train → evaluate and validate → deploy and integrate → monitor, learn and improve.
What I appreciated was the discipline of the "It IS / It is NOT / It COMPLEMENTS" framing:
- It is: teaching durable behaviour and domain expertise into the weights, grounded by data quality, evals, safety, and re-tuning
- It is not: a prompt trick, a bigger context window, a one-off training job, or pre-training from scratch
- It complements: RAG (for fresh, citable knowledge), agents (for planning and tool use), and responsible-AI guardrails at every layer
This directly answers a question I get internally: "should we fine-tune?" The honest answer for most of our use cases is still RAG + agents first, tuning only when behaviour needs to live in the weights.
Theme 4: Sovereign Cloud — AI Without the Internet
The sovereignty track was surprisingly meaty. Microsoft Sovereign Cloud now spans three pillars: Sovereign Public Cloud (built-in sovereign controls, regional data boundaries, in-country AI processing), Sovereign Private Cloud (continuous operations across hybrid or fully disconnected environments), and a Sovereign Partner Ecosystem.
The Sovereign Private Cloud slide showed Azure Local as the private cloud infrastructure running an AI suite (Foundry Local — models, IQ, tools, agents), a productivity suite (Microsoft 365 Local, GitHub Enterprise Local, Azure Virtual Desktop), and your own VMs/AKS — all connected or fully disconnected, with Azure Arc providing consistent management across Azure, AWS, GCP, Oracle, VMware, and Nutanix estates.
The showstopper was the Offline Containers case study — a full RAG pipeline successfully running in Indian Defense with zero internet connectivity: Document Intelligence containers for cracking documents, Mistral Small 3.1 + Jina Embeddings 2 for vectorization, SQL Server 2025/PostgreSQL as the vector DB, Azure Speech containers for STT/TTS, and Phi-4/Llama-4 containers doing generation. Every Azure Cognitive Service running as a disconnected Docker container on VMs.
Mandar Kulkarni's segment tied this to national-scale sovereign cloud requirements. For anyone serving regulated Indian customers — banking, defense, government — this is the reference architecture to watch.
Theme 5: Real Customer Stories — Voice AI, DHS, and BI Transformation
Kotak Mahindra — Production Voice AI (Vijay Narayanan)
The most technically detailed customer session. Kotak's Voice AI architecture: Genesys stays as the on-prem telephony system of engagement (PSTN → Genesys → SIP/WebSocket), while Azure Voice Live + LLM handles realtime voice — STT, TTS, VAD, noise suppression, barge-in — with GPT-4.1-mini doing reasoning and response generation, Azure AI Search for RAG grounding against their knowledge base, and everything running in a secured Central India landing zone over Private Link. Post-call analytics covers recording, transcript, summary, sentiment, and compliance checks; human agents get whisper/transfer/terminate via a Genesys extension.
The companion slide on making it production-grade covered the unglamorous work: finalizing the landing zone, clarifying service limits and region availability, optimizing latency and speech quality, Indic language PoCs, observability on the Voice Live integration, and vendor enablement with Engagely.
My take: the design pattern — keep the legacy system of engagement, bolt the AI runtime alongside over SIP/WebSocket + Private Link — is exactly how enterprise AI should be retrofitted. No rip-and-replace.
C5i — Domain Harnessing System (Chiranjiv Roy)
C5i presented their DHS (Domain Harnessing System) — a five-stage loop (Information Harnessing → Knowledge Harnessing → Inference Harnessing → Outcome Harnessing → Benchmarking) built on Fabric + Azure AI Foundry. Enterprise documents flow through Fabric governance into a knowledge graph, hybrid semantic search runs across graph + documents + data, AI model orchestration routes to the right model with guardrails and evaluations, and agents generate answers with citations, landing in Power BI.
The claimed business impact was aggressive: ₹1.2 Cr projected savings, 31% cost optimization, 25%+ accuracy improvement, 30–40% automation gain, 70–80% cycle-time reduction, and 22% token cost reduction. The from→to story — manual search to AI-powered semantic search, siloed knowledge to a unified knowledge graph, static reports to interactive intelligence, days to minutes — is the same story every enterprise data team is trying to tell.
Himalaya Wellness — BI Transformation
A grounded, refreshingly honest session. Their "Path to Continuous Improvement" for 2025: they found 25% of reports were not being used, plus duplicates — so KPI and report rationalization came first. Then CI/CD pipelines for BI (version control, automated deployments, simpler rollback), data quality standardization with validations and alerts, a data catalog with an ownership matrix, trained people (27 certifications), and data engineering governance with a "money saved is money earned" thrift mindset.
No AI hype in this one — just the boring, necessary groundwork that makes AI on top of BI actually trustworthy.
Speaker-Wise Quick Notes
From my notepad, session by session:
Nilesh Khattri (Data Intelligence / Fabric session) — Sparked most of my Fabric questions: why we'd need Microsoft IQ for our platform instead of directly querying the database from Fabric; how to use MCP protocols with Fabric; what pre-built models ship in Fabric; the Excel-in-Fabric experience; Teams integration from Fabric; ontology; and Real-Time Intelligence.
Kapish Chawala (Agent 365) — Walked through the Agent 365 framework with customer references including ClearTax and Physics Wallah. Agent 365 is essentially bringing employee-grade identity and governance to agents — every agent gets an identity, permissions, and oversight.
Sandeep Alur (Agentic AI / Enterprise Learning Loop) — The "agent job description" and hill-climbing frameworks above. Also worth following up: his pointers on Microsoft's notebook tooling, checking out o4-mini, and understanding what MAI (Microsoft's in-house frontier models) means for model choice.
Chiranjiv Roy, C5i — Inference harnessing, knowledge graphs, and the DHS five-stage system detailed above.
Vijay Narayanan, Kotak Mahindra — Voice domain on Genesys + Azure Voice Live, detailed above.
Mandar Kulkarni — Sovereign cloud for national-scale and security-sensitive workloads.
My Learnings
Ontology is the new semantic model. Fabric IQ's ontology layer (public preview) is where business context will live for agents. Semantic models are the on-ramp; ontology is the destination.
Agents need JDs, not just prompts. Splitting agent design into What (automations, skills), How (MCP integrations, SOP instructions), and Identity (personality, memory) is a cleaner design discipline than a monolithic system prompt.
The model layer is pluggable and disposable. GPT, Claude, Llama, MAI, DeepSeek — the architecture assumes you'll swap them. Investment should go into the agentic system around the model: evals, traces, memory, skills.
Frontier tuning is a lifecycle, not an event. If there's no eval pipeline and re-tuning loop, it's not tuning — it's a one-off training job.
Fully disconnected AI is real and production-proven. The Indian Defense offline container architecture proves the entire RAG stack — document intelligence, embeddings, vector DB, LLM, speech — runs air-gapped.
Retrofit, don't replace. Kotak kept Genesys and wrapped Azure Voice Live around it via SIP/WebSocket and Private Link. The best AI architecture respects the systems already in production.
Report rationalization before AI. Himalaya's finding that 25% of reports were unused is a reminder: pointing AI at a messy BI estate just automates the mess.
Most orgs stall at the PoC-to-production gap — and the blockers are governance, development maturity, and infrastructure, not model quality.
New Ideas I'm Taking Back
These went straight into my notebook during sessions:
- One central engineering observability solution — merge Application Insights, pipeline health, job status, and Azure cost into a single pane. Today these live in separate portals; the conference convinced me a unified internal dashboard (or a Fabric Real-Time Intelligence workload) is worth building.
- Benchmark code-review tooling — evaluate whether GitHub Copilot code review can be rolled out company-wide, and benchmark it against our current review process.
- MCP from Fabric — expose our Fabric warehouse to agents via MCP so internal agents can query governed data without custom plumbing.
- Semantic models for cost queries — build a Fabric semantic model over Azure cost data so cost questions become natural-language queries instead of manual Cost Management digging.
- Weekly work podcast — an AI-generated podcast summarizing the week's engineering notes and action items. Low effort, potentially great for team alignment.
- Candidate search latency — apply the hybrid semantic search patterns (AI Search + knowledge graph style grounding) from the C5i session to reduce latency in our recruitment candidate search.
- Plan a Fabric data agent demo — a working demo of a Fabric data agent over our warehouse, using ontology/semantic model grounding, to show internally what "agents grounded in context" actually looks like.
- OLTP vs OLAP split review — revisit which of our workloads belong on operational stores vs the Fabric warehouse, especially as ontology and data agents blur the line.
Closing Thoughts
The through-line of the day: the platform war has moved up a layer. It's no longer about who has the best model — models are pluggable commodities on every slide. It's about who owns the semantic layer, the agent identity layer, and the governance layer. Microsoft's bet is that if your ontology lives in Fabric IQ, your agents live in Foundry and Agent 365, and your data lives in OneLake, the models can be anyone's.
For practitioners like me, the takeaway is practical: invest in the durable layers — data quality, semantics, ontology, evals, observability — and stay loosely coupled to the model of the month.
If you attended and caught sessions I missed, or you're experimenting with Fabric IQ ontologies or Agent 365, I'd love to compare notes.
— Vignesh (Sinner)
Top comments (0)