DEV Community: Balaraj R

AgriSence: Building an AI Operating System for 140 Million Indian Farmers on Google Cloud

Balaraj R — Thu, 23 Jul 2026 11:07:19 +0000

India has more than 140 million smallholder farming households. Over 80% of them farm less than 2 hectares of land. Agriculture employs somewhere between 46% and 60% of the country's workforce depending on how you measure it, and contributes roughly 17–18% of national GDP.

And here's the number that actually kept me up at night: fewer than 1 in 100 of those households use any digital precision-farming tool today.

Not "an inferior tool." Not "an outdated tool." No tool at all. A farmer deciding when to irrigate, whether that yellowing patch in the field is drought stress or a fungal outbreak, whether they qualify for a government scheme, or whether they can get a fair loan against a harvest that hasn't happened yet — almost all of that is still guesswork, local word-of-mouth, or a once-a-season visit from an agricultural officer who is responsible for thousands of farmers.

## The number that started everything
Meanwhile, three technology waves have matured enough, at low enough cost, that none of this guesswork is actually necessary anymore:

Free, frequent satellite imagery (Sentinel-2, Landsat, MODIS) that can see a field's health without anyone visiting it.
Generative and agentic AI that can reason in a farmer's own language, on a ₹5,000 Android phone, over a patchy 2G connection.
Digitized government and financial rails — Aadhaar-linked direct benefit transfer, Kisan Credit Card, PMFBY crop insurance, eNAM markets — that are ready to be integrated into rather than navigated manually.

No existing platform — not CropIn, not Plantix, not DeHaat, not Microsoft FarmBeats, not IBM PAIRS — has combined all three into one farmer-first, India-first operating system.

That gap is what AgriSence is built to close.

What AgriSence actually is

AgriSence is an AI-powered Agricultural Operating System (AgriOS) — not a single app with one clever feature, but a connected platform where a farmer, an FPO, a bank, an insurer, and a government body can all interact with the same underlying intelligence. It's already a working, shipped product: a React Native/Expo mobile app for the field, a Next.js 16 web dashboard for farm management, live on Firebase, with 434+ commits and 15 orchestrated Genkit AI flows, supporting 7 Indian languages. It was recognized as a State Winner at Inferentia 2.0.

The product is organized into three intelligence layers.

Layer 1 — Farmer Intelligence

This is the everyday layer, the one a farmer touches directly.

AI Crop Doctor — a farmer points their phone camera at a diseased leaf, and Gemini's multimodal vision analyzes it against agricultural training data, returning a diagnosis, a confidence score, and actionable treatment steps — organic alternatives included, not just a chemical prescription.
Multilingual voice assistant — a farmer can simply ask, out loud, "What's the price of tomatoes in Bangalore today?" or "How much urea does a 2-acre corn field need?" and get an answer in the same language they asked in, across 7 Indian languages, with no typing required. This matters enormously in a country where literacy and smartphone-typing comfort can't be assumed for every user.
Offline-first design — fields famously have no signal. AgriSence caches data locally and silently syncs to Firebase Firestore the moment a connection reappears, so a farmer never loses a crop scan or an expense log to a dead zone.
Government scheme matching — the platform profiles a farmer's land size, location, and crop type, and automatically surfaces the subsidies and schemes (PM-KISAN, PMFBY, drip-irrigation subsidies, and more) they actually qualify for, instead of leaving them to discover it by chance.
Financial record-keeping — every rupee of expense and every harvest logged, with the platform calculating true season-level profitability and predictive ROI for the next planting cycle.

Layer 2 — Earth Intelligence

This is the layer that makes everything else trustworthy, because it doesn't rely on a farmer's self-report — it relies on what a satellite can independently verify.

AgriSence fuses three complementary satellite programs rather than depending on one: Sentinel-2 does the field-level heavy lifting, Landsat anchors the multi-season historical baseline every current reading is compared against, and MODIS's daily-but-coarse signal fills the gap during the long cloud-cover blackouts that Kharif-season monsoons routinely cause.

From that fused feed, the platform computes a whole suite of vegetation and moisture indices — NDVI, SAVI, EVI, NDMI, NDWI, GCI — because no single index tells the whole story. NDVI alone, for instance, can't tell you whether a field is drought-stressed or disease-damaged; NDMI (a moisture-specific index) is what lets the advisory engine tell a farmer "irrigate" instead of just "something is wrong, good luck."

The output is never a raw number shown to a farmer. It's a plain-language statement — "your field's moisture has dropped compared to the same point last season, consider irrigating within 3 days" — with the underlying index math kept in an auditable reasoning trace, because that same data becomes the backbone of insurance claims later.

Layer 3 — Financial Intelligence

This is the layer that turns "we can see the field is healthy" into "we can lend against it, insure it, or credit it for carbon."

Alternative-data credit scoring, using the satellite-verified cultivation signal instead of paperwork a smallholder farmer often doesn't have.
Parametric crop insurance, where a satellite-observed trigger — not a manual claims adjuster driving out to a field — determines a payout.
Carbon credit MRV (measurement, reporting, verification), so sustainable practice changes become a monetizable outcome, not just a good deed.

Each of these three layers is independently monetizable, but they all share the same underlying data spine — a farmer's disease-detection photo also quietly strengthens the confidence that the plot tied to a loan application is actually being cultivated.

The engineering underneath: why this was genuinely hard

A few of the hardest problems we had to design around, because they're the kind of thing that looks trivial until you actually try to ship it in an Indian agricultural context:

Cloud contamination. During the monsoon — exactly when moisture-stress and flood monitoring matter most — optical satellites get blocked by cloud cover for weeks. A naive system just goes blind at the worst possible moment. We mask cloud-probability per pixel, fall back to MODIS's daily coarse signal cross-calibrated against a field's own last clear reading, and reconstruct short gaps using the crop's own expected growth curve — not a naive straight-line guess.
Small, noisy, per-farmer data. Smallholder farmers, who dominate the market, each have thin historical data. Pooling across similar farmers and fields, rather than fitting one model per farmer, is what makes forecasting viable at all.
Literacy and connectivity, not just language. Multilingual isn't enough on its own — a voice-first interface and an offline-first architecture were non-negotiable design constraints, not nice-to-haves, because a huge share of the target user base can't be assumed to type comfortably or have a stable connection.
Explainability as a design rule, not an afterthought. Every recommendation has to carry a reason a farmer can verify against their own field, because trust — not accuracy in a lab benchmark — is what actually drives adoption.

Built end-to-end on Google Cloud

I want to be specific here, because "we use AI" is not the same claim as "we run production infrastructure on Google Cloud," and this project is the second one.

Firebase — the operational backbone since day one; hosting, and the platform's Firestore-based real-time data layer.
Firebase Genkit — 15 orchestrated AI flows. Instead of one monolithic chatbot, a router flow decides which specialist flow (disease triage, fertilizer advice, weather explanation, scheme matching) to invoke per query, then merges the results into one coherent answer — the same architectural direction the wider industry is moving toward, from single-shot chat to multi-agent, tool-calling systems.
Google Gemini — the reasoning and generation layer behind the conversational advisory and the multimodal crop-disease vision pipeline, grounded via retrieval-augmented generation against a knowledge base of agronomy science and government scheme documents, specifically to reduce hallucination risk on high-stakes questions like dosage or loan eligibility.
Vertex AI — training and serving the computer-vision disease-detection models and the time-series yield-forecasting models.
Cloud Run — stateless serving for the Genkit flows and APIs.
Cloud Firestore — the primary operational database, already handling real-time sync between the mobile app and the web dashboard.
Cloud Storage — raw satellite imagery and farmer-uploaded photos.
BigQuery — the analytical warehouse designed to hold satellite time-series and marketplace transaction data as the platform scales.
Pub/Sub — the event backbone connecting satellite-ingestion jobs, weather-alert triggers, and notification delivery.
Cloud Monitoring — SLOs on advisory-response latency, which matters more than it sounds: a farmer is often asking mid-outbreak, not browsing casually.

The design principle underneath all of it is strict layering: the AI layer never talks to a raw external API (weather, satellite, government schemes) directly. Everything sits behind business-layer service interfaces, so swapping a Vertex AI model version never touches data-access code, and switching a weather API provider never touches an AI prompt.

Why now, why India

Market-sizing figures vary by research house, but the direction is unambiguous across all of them: the dedicated Indian agritech market is estimated at roughly US$974 million in 2025, projected to reach around US$2.52 billion by 2034 — a small technology layer sitting on top of a roughly US$500 billion underlying agriculture economy that has barely been touched by digital tools. India's 2026-27 Union Budget allocated ₹1,62,671 crore to agriculture and allied activities, with PM-KISAN alone at ₹63,500 crore, and the government itself is now building AI-based multilingual advisory infrastructure (Bharat-VISTAAR) — a signal that the direction AgriSence is betting on is also the direction national policy is moving.

None of that greenfield closes itself. It closes when someone actually ships the product, on infrastructure that can scale to national reach without re-architecting from scratch. That's the bet AgriSence is making, and Google Cloud is the platform it's making that bet on.

What's next

The roadmap ahead includes IoT hardware integration (Bluetooth-connected soil-moisture sensors), drone-based orthomosaic field mapping for even finer-grained monitoring, a peer-to-peer equipment-sharing marketplace for smallholder farmers who can't justify owning machinery outright, and expanding language support further across India's linguistic diversity.

If you want to see the code, the architecture, or contribute:

GitHub: github.com/balaraj74/AgriSence

This project is being showcased at the Google Cloud Gen AI Academy — APAC Edition, Builders Meet 2026, and I'd genuinely welcome feedback, contributions, or just a conversation from anyone who cares about the intersection of AI and agriculture at scale.

How I Built OmniSence -A Multimodal AI That Streams Text, Images & Audio Together

Balaraj R — Mon, 16 Mar 2026 18:21:55 +0000

Disclosure: I created this piece of content for the purposes of
entering the Gemini Live Agent Challenge hackathon on Devpost.

GeminiLiveAgentChallenge

The Problem That Kept Me Up at Night

Every AI tool I've used thinks in documents, not experiences.

You get text here. An image there. Maybe audio if you switch tabs
and use a different tool entirely. But a real creative director doesn't
hand you a Word document — they paint a scene with words, sketches, and
emotion simultaneously.

That gap is what I built OmniSence to close.

What OmniSence Does

OmniSence is a Creative Director AI that takes a single idea — spoken
or typed — and streams text, images, and audio together in real-time
as one cohesive, interleaved experience.

You speak: "A girl who discovers she can paint the future."

OmniSence responds with:

📝 Narrative prose streaming word by word
🖼️ Watercolor illustrations appearing inline mid-sentence
🔊 Studio-quality narration reading the story back to you

All at once. All live. No switching tabs.

The Core Technical Innovation: Orchestrated Interleaved Streaming

This was the hardest problem to solve.

Gemini doesn't natively emit image bytes mid-text stream. So I designed
a pattern I call Orchestrated Interleaved Streaming using Google ADK:

User Prompt
    ↓
Google ADK Agent
    ↓
Gemini 3.1 Flash streams text with [IMAGE_DIRECTIVE: ...] markers
    ↓ (on marker detection)
Imagen 4 (async) ←→ Cloud TTS (async)
    ↓                    ↓
GCS Upload           GCS Upload  
    ↓                    ↓
SSE: {type:"image"}  SSE: {type:"audio"}
    ↓
React frontend renders everything inline, live

The key insight: image and audio generation run in parallel while
text continues streaming. The user never waits for one to finish before
the next begins.

The perceived latency formula:

$$T_{total} = T_{text} + \max(T_{imagen}, T_{tts}) - T_{overlap}$$

In practice, this means a full illustrated, narrated story appears in
under 30 seconds from a single voice prompt.

Building the ADK Agent

The Google ADK (Agent Development Kit) was the backbone of the entire
project. Instead of chaining API calls manually, I defined the agent
with 5 real async tools:

root_agent = Agent(
    name="omnisence",
    model="gemini-3.1-flash",
    description="OmniSence — Elite Creative Director AI",
    instruction=SYSTEM_PROMPT,
    tools=[
        generate_scene_image,   # Imagen 4 → GCS
        narrate_text,           # Cloud TTS → GCS  
        search_creative_references,  # Google Search grounding
        save_session_asset,     # GCS persistence
        get_style_constraints,  # Creative mode framework
    ],
)

What surprised me: the agent naturally decides when to generate
images vs. keep writing — creating organic pacing without me hardcoding
any rules. That emergent creative judgment was something I didn't expect.

Grounding: Preventing Hallucinations

For educational mode, I integrated Google Search grounding with a
single SDK addition:

if mode == "educational":
    generation_config["tools"] = [{"google_search": {}}]

One line. Dramatically more accurate factual content. The agent now
cites real sources before weaving them into the narrative.

The Google Cloud Stack

Service	What I Used It For
Gemini 3.1 Flash	Core text generation with image directives
Imagen 4	Scene illustration via Vertex AI
Cloud Text-to-Speech	Studio voice narration
Cloud Run	Serverless backend hosting
Cloud Storage	Asset persistence across sessions
Cloud Build	CI/CD with `cloudbuild.yaml`

One thing I want to highlight: Cloud Run's concurrency model was
perfect for SSE streaming. Each user gets a persistent async generator
that streams for up to 5 minutes — Cloud Run handles this gracefully
without the connection dropping.

The UX Decision That Changed Everything

Early in development, images appeared below the text after it
finished generating. It felt like "AI output."

The moment I moved images to appear inline — mid-paragraph,
exactly where the story described them — the experience shifted from
"AI output" to "living document."

That single UX change had more impact on how the product felt than
any technical improvement I made.

Challenges I Didn't Anticipate

Stream cancellation was deceptively hard. When a user hits
"Stop & Redirect" mid-generation, you can't just close the SSE
connection — there are async Imagen and TTS calls in flight on
GCP that need to be cleanly abandoned without leaving orphaned
uploads or billing surprises.

My solution: per-session cancellation flags checked at every
yield point in the async generator, combined with asyncio.shield()
for GCS cleanup tasks that must complete regardless.

Cloud TTS latency was the other beast. Studio voices take 2–4
seconds per paragraph. I solved this with a rolling generation pipeline:
generate audio for completed sentences while later sentences still stream.

What I'd Tell Someone Starting This Today

Use ADK, not raw SDK calls. The tool system makes multi-model
orchestration feel natural instead of spaghetti.
Design for streaming from day one. Adding SSE to a
request/response architecture later is painful. Build async generators
first.
Google Search grounding is a one-liner. Add it to any factual
mode. The quality difference is immediate and obvious.
The persona matters more than the features. OmniSence has a
distinct voice — warm, bold, cinematic. Users respond to that
personality more than any specific capability.
Deploy early to Cloud Run. Local dev hides async edge cases
that only appear under real network conditions.

Try It Yourself

🔗 GitHub: https://github.com/balaraj74/Omnisence

🔗 Live Demo: https://omnisence-518586257861.us-central1.run.app/

The entire project is open source. The README includes a one-command
deploy script — ./deploy.sh YOUR_PROJECT_ID YOUR_API_KEY — and
you'll have your own OmniSence instance running on Cloud Run in
under 10 minutes.

Built for the Gemini Live Agent Challenge · Powered by Google Gemini,
Imagen 4, and Google Cloud · #GeminiLiveAgentChallenge